Note: This is test shard 1 of 3.
[==========] Running 2 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 2 tests from CatalogManagerConfigurations/MasterStressTest
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/0
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260925 09:56:51.121598 10276 test_util.cc:268] Using random seed: -1136412355
2026-09-25T09:56:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:56:51Z Disabled control of system clock
I20260925 09:56:51.173128 10276 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu
/tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.9.62:34455
--webserver_interface=127.10.9.62
--webserver_port=0
--builtin_ntp_servers=127.10.9.20:38279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.9.62:34455,127.10.9.61:33047,127.10.9.60:36985
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260925 09:56:51.501319 10285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:56:51.501888 10285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:56:51.502288 10285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:56:51.532333 10285 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:56:51.532603 10285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:56:51.532795 10285 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:56:51.532982 10285 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:56:51.569653 10285 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.9.20:38279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-0/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.10.9.62:34455,127.10.9.61:33047,127.10.9.60:36985
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.9.62:34455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.9.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:06:08 UTC on ccycloud.quasar-lntfnh.root.comops.site
build id 4672078
TSAN enabled
I20260925 09:56:51.570824 10285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:56:51.572485 10285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:56:51.593986 10292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:56:51.595407 10294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:56:51.594161 10291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:56:52.792774 10293 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection time-out
I20260925 09:56:52.792902 10285 server_base.cc:1029] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260925 09:56:52.796980 10285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:56:52.799671 10285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:56:52.801080 10285 hybrid_clock.cc:648] HybridClock initialized: now 1790330212801043 us; error 72 us; skew 500 ppm
I20260925 09:56:52.801999 10285 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:56:52.818605 10285 webserver.cc:460] Webserver started at http://127.10.9.62:34589/ using document root <none> and password file <none>
I20260925 09:56:52.819671 10285 fs_manager.cc:359] Metadata directory not provided
I20260925 09:56:52.819905 10285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:56:52.820393 10285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:56:52.826066 10285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-0/data/instance:
uuid: "54c8485a7dcc47a1a75d08ef57647ece"
format_stamp: "Formatted at 2026-09-25 09:56:52 on dist-test-slave-4xvl"
I20260925 09:56:52.827234 10285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-0/wal/instance:
uuid: "54c8485a7dcc47a1a75d08ef57647ece"
format_stamp: "Formatted at 2026-09-25 09:56:52 on dist-test-slave-4xvl"
I20260925 09:56:52.834658 10285 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260925 09:56:52.840405 10301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:56:52.841553 10285 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.003s
I20260925 09:56:52.841902 10285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-0/wal
uuid: "54c8485a7dcc47a1a75d08ef57647ece"
format_stamp: "Formatted at 2026-09-25 09:56:52 on dist-test-slave-4xvl"
I20260925 09:56:52.842237 10285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:56:52.879318 10285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:56:52.880848 10285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:56:52.881381 10285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:56:52.964174 10285 rpc_server.cc:307] RPC server started. Bound to: 127.10.9.62:34455
I20260925 09:56:52.964262 10352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.9.62:34455 every 8 connection(s)
I20260925 09:56:52.966764 10285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-0/data/info.pb
I20260925 09:56:52.972150 10353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:56:52.976214 10276 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu as pid 10285
I20260925 09:56:52.976641 10276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-0/wal/instance
I20260925 09:56:52.980208 10276 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu
/tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.9.61:33047
--webserver_interface=127.10.9.61
--webserver_port=0
--builtin_ntp_servers=127.10.9.20:38279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.9.62:34455,127.10.9.61:33047,127.10.9.60:36985
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260925 09:56:52.983356 10353 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } has no permanent_uuid. Determining permanent_uuid...
I20260925 09:56:53.012655 10353 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } has no permanent_uuid. Determining permanent_uuid...
W20260925 09:56:53.014928 10303 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.9.61:33047: connect: Connection refused (error 111)
W20260925 09:56:53.019266 10353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.61:33047: Network error: Client connection negotiation failed: client connection to 127.10.9.61:33047: connect: Connection refused (error 111)
I20260925 09:56:53.068826 10353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } attempt: 1
W20260925 09:56:53.073365 10353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.61:33047: Network error: Client connection negotiation failed: client connection to 127.10.9.61:33047: connect: Connection refused (error 111)
I20260925 09:56:53.141762 10353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } attempt: 2
W20260925 09:56:53.146099 10353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.61:33047: Network error: Client connection negotiation failed: client connection to 127.10.9.61:33047: connect: Connection refused (error 111)
I20260925 09:56:53.237524 10353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } attempt: 3
W20260925 09:56:53.241434 10353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.61:33047: Network error: Client connection negotiation failed: client connection to 127.10.9.61:33047: connect: Connection refused (error 111)
W20260925 09:56:53.304499 10355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:56:53.305100 10355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:56:53.305570 10355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:56:53.335964 10355 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:56:53.336309 10355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:56:53.336570 10355 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:56:53.336822 10355 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:56:53.371480 10355 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.9.20:38279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-1/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.10.9.62:34455,127.10.9.61:33047,127.10.9.60:36985
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.9.61:33047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.9.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:06:08 UTC on ccycloud.quasar-lntfnh.root.comops.site
build id 4672078
TSAN enabled
I20260925 09:56:53.372745 10355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:56:53.374320 10355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 09:56:53.384826 10353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } attempt: 4
W20260925 09:56:53.390012 10353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.61:33047: Network error: Client connection negotiation failed: client connection to 127.10.9.61:33047: connect: Connection refused (error 111)
W20260925 09:56:53.394708 10365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:56:53.689512 10353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } attempt: 5
W20260925 09:56:53.693459 10353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.61:33047: Network error: Client connection negotiation failed: client connection to 127.10.9.61:33047: connect: Connection refused (error 111)
I20260925 09:56:54.241083 10353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } attempt: 6
W20260925 09:56:54.248412 10353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.61:33047: Network error: Client connection negotiation failed: client connection to 127.10.9.61:33047: connect: Connection refused (error 111)
W20260925 09:56:53.394824 10364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:56:53.395934 10367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:56:53.397315 10355 server_base.cc:1034] running on GCE node
I20260925 09:56:54.615317 10355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:56:54.617919 10355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:56:54.619313 10355 hybrid_clock.cc:648] HybridClock initialized: now 1790330214619261 us; error 99 us; skew 500 ppm
I20260925 09:56:54.620232 10355 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:56:54.626691 10355 webserver.cc:460] Webserver started at http://127.10.9.61:42357/ using document root <none> and password file <none>
I20260925 09:56:54.627701 10355 fs_manager.cc:359] Metadata directory not provided
I20260925 09:56:54.627943 10355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:56:54.628415 10355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:56:54.633980 10355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-1/data/instance:
uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3"
format_stamp: "Formatted at 2026-09-25 09:56:54 on dist-test-slave-4xvl"
I20260925 09:56:54.635144 10355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-1/wal/instance:
uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3"
format_stamp: "Formatted at 2026-09-25 09:56:54 on dist-test-slave-4xvl"
I20260925 09:56:54.643273 10355 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.005s
I20260925 09:56:54.649874 10376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:56:54.650959 10355 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.003s
I20260925 09:56:54.651316 10355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-1/data,/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-1/wal
uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3"
format_stamp: "Formatted at 2026-09-25 09:56:54 on dist-test-slave-4xvl"
I20260925 09:56:54.651749 10355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:56:54.691042 10355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:56:54.692523 10355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:56:54.692991 10355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:56:54.797456 10355 rpc_server.cc:307] RPC server started. Bound to: 127.10.9.61:33047
I20260925 09:56:54.797538 10427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.9.61:33047 every 8 connection(s)
I20260925 09:56:54.801473 10355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-1/data/info.pb
I20260925 09:56:54.804448 10276 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu as pid 10355
I20260925 09:56:54.804996 10276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-1/wal/instance
I20260925 09:56:54.808198 10276 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu
/tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.9.60:36985
--webserver_interface=127.10.9.60
--webserver_port=0
--builtin_ntp_servers=127.10.9.20:38279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.9.62:34455,127.10.9.61:33047,127.10.9.60:36985
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260925 09:56:54.809235 10428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:56:54.829483 10428 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } has no permanent_uuid. Determining permanent_uuid...
I20260925 09:56:54.862198 10428 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } has no permanent_uuid. Determining permanent_uuid...
I20260925 09:56:54.879093 10428 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } has no permanent_uuid. Determining permanent_uuid...
W20260925 09:56:54.881224 10379 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.9.60:36985: connect: Connection refused (error 111)
W20260925 09:56:54.885063 10428 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.60:36985: Network error: Client connection negotiation failed: client connection to 127.10.9.60:36985: connect: Connection refused (error 111)
I20260925 09:56:54.934542 10428 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } attempt: 1
W20260925 09:56:54.938530 10428 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.60:36985: Network error: Client connection negotiation failed: client connection to 127.10.9.60:36985: connect: Connection refused (error 111)
I20260925 09:56:55.006939 10428 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } attempt: 2
W20260925 09:56:55.010684 10428 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.60:36985: Network error: Client connection negotiation failed: client connection to 127.10.9.60:36985: connect: Connection refused (error 111)
I20260925 09:56:55.102154 10428 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } attempt: 3
W20260925 09:56:55.106314 10428 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.60:36985: Network error: Client connection negotiation failed: client connection to 127.10.9.60:36985: connect: Connection refused (error 111)
W20260925 09:56:55.175077 10430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:56:55.175696 10430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:56:55.176115 10430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:56:55.206617 10430 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:56:55.206918 10430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:56:55.207115 10430 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:56:55.207300 10430 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:56:55.242197 10430 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.9.20:38279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-2/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.10.9.62:34455,127.10.9.61:33047,127.10.9.60:36985
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.9.60:36985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.9.60
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:06:08 UTC on ccycloud.quasar-lntfnh.root.comops.site
build id 4672078
TSAN enabled
I20260925 09:56:55.243355 10430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:56:55.244881 10430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 09:56:55.249782 10428 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } attempt: 4
W20260925 09:56:55.255257 10428 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.60:36985: Network error: Client connection negotiation failed: client connection to 127.10.9.60:36985: connect: Connection refused (error 111)
W20260925 09:56:55.269344 10440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:56:55.309024 10353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } attempt: 7
I20260925 09:56:55.323033 10353 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } has no permanent_uuid. Determining permanent_uuid...
W20260925 09:56:55.329162 10353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.60:36985: Network error: Client connection negotiation failed: client connection to 127.10.9.60:36985: connect: Connection refused (error 111)
I20260925 09:56:55.387596 10353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } attempt: 1
W20260925 09:56:55.392699 10353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.60:36985: Network error: Client connection negotiation failed: client connection to 127.10.9.60:36985: connect: Connection refused (error 111)
I20260925 09:56:55.474071 10353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } attempt: 2
W20260925 09:56:55.478716 10353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.60:36985: Network error: Client connection negotiation failed: client connection to 127.10.9.60:36985: connect: Connection refused (error 111)
I20260925 09:56:55.554821 10428 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } attempt: 5
W20260925 09:56:55.560297 10428 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.60:36985: Network error: Client connection negotiation failed: client connection to 127.10.9.60:36985: connect: Connection refused (error 111)
I20260925 09:56:55.564033 10353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } attempt: 3
W20260925 09:56:55.568491 10353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.60:36985: Network error: Client connection negotiation failed: client connection to 127.10.9.60:36985: connect: Connection refused (error 111)
I20260925 09:56:55.708847 10353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } attempt: 4
W20260925 09:56:55.713193 10353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.60:36985: Network error: Client connection negotiation failed: client connection to 127.10.9.60:36985: connect: Connection refused (error 111)
I20260925 09:56:55.996619 10353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } attempt: 5
W20260925 09:56:56.002705 10353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.60:36985: Network error: Client connection negotiation failed: client connection to 127.10.9.60:36985: connect: Connection refused (error 111)
I20260925 09:56:56.107913 10428 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } attempt: 6
W20260925 09:56:56.118414 10428 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.60:36985: Network error: Client connection negotiation failed: client connection to 127.10.9.60:36985: connect: Connection refused (error 111)
W20260925 09:56:55.269339 10441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:56:55.270450 10443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:56:55.272846 10430 server_base.cc:1034] running on GCE node
I20260925 09:56:56.494620 10430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:56:56.497264 10430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:56:56.498642 10430 hybrid_clock.cc:648] HybridClock initialized: now 1790330216498603 us; error 50 us; skew 500 ppm
I20260925 09:56:56.499542 10430 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:56:56.505857 10430 webserver.cc:460] Webserver started at http://127.10.9.60:41643/ using document root <none> and password file <none>
I20260925 09:56:56.506762 10430 fs_manager.cc:359] Metadata directory not provided
I20260925 09:56:56.506986 10430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:56:56.507476 10430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:56:56.512172 10430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-2/data/instance:
uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2"
format_stamp: "Formatted at 2026-09-25 09:56:56 on dist-test-slave-4xvl"
I20260925 09:56:56.513603 10430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-2/wal/instance:
uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2"
format_stamp: "Formatted at 2026-09-25 09:56:56 on dist-test-slave-4xvl"
I20260925 09:56:56.520483 10430 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260925 09:56:56.526002 10454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:56:56.527119 10430 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260925 09:56:56.527529 10430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-2/data,/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-2/wal
uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2"
format_stamp: "Formatted at 2026-09-25 09:56:56 on dist-test-slave-4xvl"
I20260925 09:56:56.527961 10430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:56:56.555124 10353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } attempt: 6
W20260925 09:56:56.561954 10353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.60:36985: Network error: Client connection negotiation failed: client connection to 127.10.9.60:36985: connect: Connection refused (error 111)
I20260925 09:56:56.588641 10430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:56:56.590052 10430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:56:56.590461 10430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:56:56.690147 10430 rpc_server.cc:307] RPC server started. Bound to: 127.10.9.60:36985
I20260925 09:56:56.690241 10508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.9.60:36985 every 8 connection(s)
I20260925 09:56:56.692953 10430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-2/data/info.pb
I20260925 09:56:56.695595 10276 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu as pid 10430
I20260925 09:56:56.696280 10276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-2/wal/instance
I20260925 09:56:56.699083 10509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:56:56.719696 10509 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } has no permanent_uuid. Determining permanent_uuid...
I20260925 09:56:56.744872 10509 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } has no permanent_uuid. Determining permanent_uuid...
I20260925 09:56:56.758126 10509 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } has no permanent_uuid. Determining permanent_uuid...
I20260925 09:56:56.780658 10509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2: Bootstrap starting.
I20260925 09:56:56.788669 10509 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2: Neither blocks nor log segments found. Creating new log.
I20260925 09:56:56.790289 10509 log.cc:824] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2: Log is configured to *not* fsync() on all Append() calls
I20260925 09:56:56.796836 10509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2: No bootstrap required, opened a new log
I20260925 09:56:56.826529 10509 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } }
I20260925 09:56:56.827160 10509 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:56:56.827349 10509 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8be057e0b6d40a3a4b90f2b1f4a0fa2, State: Initialized, Role: FOLLOWER
I20260925 09:56:56.828009 10509 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } }
I20260925 09:56:56.831952 10518 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } } }
I20260925 09:56:56.833216 10518 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:56:56.834564 10509 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [sys.catalog]: configured and running, proceeding with master startup.
W20260925 09:56:56.846455 10462 tablet.cc:2369] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 09:56:56.869220 10534 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:56:56.869555 10534 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 09:56:56.865516 10430 master_runner.cc:186] Error getting master registration for 127.10.9.62:34455: OK, instance_id { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" instance_seqno: 1790330212938774 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 09:56:56.893605 10430 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 09:56:57.055585 10518 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:56:57.056037 10518 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } }
I20260925 09:56:57.057983 10518 leader_election.cc:288] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 54c8485a7dcc47a1a75d08ef57647ece (127.10.9.62:34455), 8af3f111e8da4d2dad4e4a98ef0fd5d3 (127.10.9.61:33047)
I20260925 09:56:57.179092 10428 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } attempt: 7
I20260925 09:56:57.206581 10428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Bootstrap starting.
I20260925 09:56:57.211687 10428 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Neither blocks nor log segments found. Creating new log.
I20260925 09:56:57.213424 10428 log.cc:824] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Log is configured to *not* fsync() on all Append() calls
I20260925 09:56:57.220953 10428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: No bootstrap required, opened a new log
I20260925 09:56:57.245447 10428 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } }
I20260925 09:56:57.246227 10428 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:56:57.246493 10428 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8af3f111e8da4d2dad4e4a98ef0fd5d3, State: Initialized, Role: FOLLOWER
I20260925 09:56:57.247565 10428 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } }
I20260925 09:56:57.250530 10538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } } }
I20260925 09:56:57.251444 10538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:56:57.252388 10428 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:56:57.252770 10403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" is_pre_election: true
I20260925 09:56:57.253772 10403 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8be057e0b6d40a3a4b90f2b1f4a0fa2 in term 0.
I20260925 09:56:57.255452 10456 leader_election.cc:302] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8af3f111e8da4d2dad4e4a98ef0fd5d3, b8be057e0b6d40a3a4b90f2b1f4a0fa2; no voters:
I20260925 09:56:57.257264 10518 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:56:57.257632 10518 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:56:57.257964 10518 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:56:57.266096 10518 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } }
I20260925 09:56:57.270706 10518 leader_election.cc:288] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [CANDIDATE]: Term 1 election: Requested vote from peers 54c8485a7dcc47a1a75d08ef57647ece (127.10.9.62:34455), 8af3f111e8da4d2dad4e4a98ef0fd5d3 (127.10.9.61:33047)
I20260925 09:56:57.271463 10403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3"
I20260925 09:56:57.272352 10403 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:56:57.279323 10403 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8be057e0b6d40a3a4b90f2b1f4a0fa2 in term 1.
I20260925 09:56:57.280692 10456 leader_election.cc:302] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8af3f111e8da4d2dad4e4a98ef0fd5d3, b8be057e0b6d40a3a4b90f2b1f4a0fa2; no voters:
I20260925 09:56:57.281780 10518 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:56:57.292070 10518 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 1 LEADER]: Becoming Leader. State: Replica: b8be057e0b6d40a3a4b90f2b1f4a0fa2, State: Running, Role: LEADER
I20260925 09:56:57.293154 10518 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } }
I20260925 09:56:57.305167 10542 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8be057e0b6d40a3a4b90f2b1f4a0fa2. Latest consensus state: current_term: 1 leader_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } } }
I20260925 09:56:57.306211 10542 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [sys.catalog]: This master's current role is: LEADER
I20260925 09:56:57.314587 10547 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:56:57.323822 10547 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:56:57.349736 10403 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [term 1 FOLLOWER]: Refusing update from remote peer b8be057e0b6d40a3a4b90f2b1f4a0fa2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:56:57.352779 10542 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260925 09:56:57.369891 10552 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:56:57.370231 10552 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 09:56:57.339224 10355 master_runner.cc:186] Error getting master registration for 127.10.9.62:34455: OK, instance_id { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" instance_seqno: 1790330212938774 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 09:56:57.407178 10355 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 09:56:57.423529 10553 mvcc.cc:205] Tried to move back new op lower bound from 7333192570236477440 to 7333192570044624896. Current Snapshot: MvccSnapshot[applied={T|T < 7333192570236477440}]
I20260925 09:56:57.424911 10542 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } } }
I20260925 09:56:57.425690 10542 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [sys.catalog]: This master's current role is: LEADER
I20260925 09:56:57.429657 10547 catalog_manager.cc:1347] Generated new cluster ID: 189c454748024b14abd819435a918f04
I20260925 09:56:57.429996 10547 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:56:57.432112 10538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8be057e0b6d40a3a4b90f2b1f4a0fa2. Latest consensus state: current_term: 1 leader_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } } }
I20260925 09:56:57.432942 10538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:56:57.452889 10538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } } }
I20260925 09:56:57.454190 10538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:56:57.493003 10547 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:56:57.495572 10547 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:56:57.526429 10547 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2: Generated new TSK 0
I20260925 09:56:57.527689 10547 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:56:57.546166 10276 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu
/tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.9.1:0
--local_ip_for_outbound_sockets=127.10.9.1
--webserver_interface=127.10.9.1
--webserver_port=0
--tserver_master_addrs=127.10.9.62:34455,127.10.9.61:33047,127.10.9.60:36985
--builtin_ntp_servers=127.10.9.20:38279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260925 09:56:57.595345 10353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } attempt: 7
I20260925 09:56:57.646298 10353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece: Bootstrap starting.
I20260925 09:56:57.658631 10353 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece: Neither blocks nor log segments found. Creating new log.
I20260925 09:56:57.662065 10353 log.cc:824] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece: Log is configured to *not* fsync() on all Append() calls
I20260925 09:56:57.680430 10353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece: No bootstrap required, opened a new log
W20260925 09:56:57.795888 10457 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.10.9.62:34455 timed out after 0.736s (SENT)
I20260925 09:56:57.815826 10353 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } }
I20260925 09:56:57.831537 10353 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:56:57.831990 10353 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54c8485a7dcc47a1a75d08ef57647ece, State: Initialized, Role: FOLLOWER
I20260925 09:56:57.833019 10353 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } }
I20260925 09:56:57.851461 10325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54c8485a7dcc47a1a75d08ef57647ece" is_pre_election: true
I20260925 09:56:57.852996 10325 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8be057e0b6d40a3a4b90f2b1f4a0fa2 in term 0.
I20260925 09:56:57.855127 10328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54c8485a7dcc47a1a75d08ef57647ece" is_pre_election: true
I20260925 09:56:57.855752 10328 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8be057e0b6d40a3a4b90f2b1f4a0fa2 in term 0.
W20260925 09:56:57.856209 10328 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52200 (request call id 2) took 797 ms (client timeout 736 ms). Trace:
W20260925 09:56:57.856514 10328 rpcz_store.cc:269] 0925 09:56:57.058659 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:56:57.058819 (+ 160us) service_pool.cc:227] Handling call
0925 09:56:57.856135 (+797316us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260925 09:56:57.858071 10326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:56:57.862708 10327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54c8485a7dcc47a1a75d08ef57647ece"
W20260925 09:56:57.863775 10327 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52200 (request call id 3) took 595 ms (client timeout 596 ms). Trace:
W20260925 09:56:57.864117 10327 rpcz_store.cc:269] 0925 09:56:57.268905 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:56:57.269055 (+ 150us) service_pool.cc:227] Handling call
0925 09:56:57.863727 (+594672us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260925 09:56:57.867686 10326 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 1 FOLLOWER]: Refusing update from remote peer b8be057e0b6d40a3a4b90f2b1f4a0fa2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:56:57.847162 10565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } } }
I20260925 09:56:57.877350 10565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:56:57.875953 10353 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:56:57.881301 10518 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260925 09:56:57.891619 10307 tablet.cc:2369] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:56:57.967365 10565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8be057e0b6d40a3a4b90f2b1f4a0fa2. Latest consensus state: current_term: 1 leader_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } } }
I20260925 09:56:57.970871 10565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:56:57.969462 10571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } } }
I20260925 09:56:57.972502 10571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:56:58.051604 10581 catalog_manager.cc:1259] Loaded cluster ID: 189c454748024b14abd819435a918f04
I20260925 09:56:58.051883 10581 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece: loading cluster ID for follower catalog manager: success
I20260925 09:56:58.059993 10581 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece: acquiring CA information for follower catalog manager: success
I20260925 09:56:58.065615 10581 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260925 09:56:58.314217 10561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:56:58.315055 10561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:56:58.316118 10561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:56:58.374076 10561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:56:58.375782 10561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.9.1
I20260925 09:56:58.376991 10552 catalog_manager.cc:1259] Loaded cluster ID: 189c454748024b14abd819435a918f04
I20260925 09:56:58.377393 10552 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: loading cluster ID for follower catalog manager: success
I20260925 09:56:58.386499 10571 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b8be057e0b6d40a3a4b90f2b1f4a0fa2)
I20260925 09:56:58.387041 10571 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } }
I20260925 09:56:58.391122 10403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "54c8485a7dcc47a1a75d08ef57647ece" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" is_pre_election: true
I20260925 09:56:58.392306 10552 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: acquiring CA information for follower catalog manager: success
I20260925 09:56:58.395183 10483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "54c8485a7dcc47a1a75d08ef57647ece" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" is_pre_election: true
I20260925 09:56:58.397188 10304 leader_election.cc:302] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54c8485a7dcc47a1a75d08ef57647ece; no voters: 8af3f111e8da4d2dad4e4a98ef0fd5d3, b8be057e0b6d40a3a4b90f2b1f4a0fa2
I20260925 09:56:58.398594 10565 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 09:56:58.399362 10571 leader_election.cc:288] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8af3f111e8da4d2dad4e4a98ef0fd5d3 (127.10.9.61:33047), b8be057e0b6d40a3a4b90f2b1f4a0fa2 (127.10.9.60:36985)
I20260925 09:56:58.408850 10552 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 09:56:58.441170 10561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.9.20:38279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.9.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.9.1
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.10.9.62:34455,127.10.9.61:33047,127.10.9.60:36985
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.9.1
--log_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:06:08 UTC on ccycloud.quasar-lntfnh.root.comops.site
build id 4672078
TSAN enabled
I20260925 09:56:58.442845 10561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:56:58.445071 10561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 09:56:58.471007 10587 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b8be057e0b6d40a3a4b90f2b1f4a0fa2)
I20260925 09:56:58.471941 10587 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } }
I20260925 09:56:58.476729 10326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54c8485a7dcc47a1a75d08ef57647ece" is_pre_election: true
I20260925 09:56:58.477298 10587 leader_election.cc:288] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 54c8485a7dcc47a1a75d08ef57647ece (127.10.9.62:34455), b8be057e0b6d40a3a4b90f2b1f4a0fa2 (127.10.9.60:36985)
I20260925 09:56:58.477628 10326 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8af3f111e8da4d2dad4e4a98ef0fd5d3 in term 1.
I20260925 09:56:58.481820 10379 leader_election.cc:302] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54c8485a7dcc47a1a75d08ef57647ece, 8af3f111e8da4d2dad4e4a98ef0fd5d3; no voters:
I20260925 09:56:58.482826 10587 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 09:56:58.483174 10587 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b8be057e0b6d40a3a4b90f2b1f4a0fa2)
I20260925 09:56:58.483600 10587 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:56:58.490262 10587 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } }
I20260925 09:56:58.493196 10326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54c8485a7dcc47a1a75d08ef57647ece"
I20260925 09:56:58.493808 10326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:56:58.491513 10518 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } } }
I20260925 09:56:58.494385 10518 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [sys.catalog]: This master's current role is: LEADER
I20260925 09:56:58.494545 10483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" is_pre_election: true
I20260925 09:56:58.501853 10483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2"
I20260925 09:56:58.502269 10326 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8af3f111e8da4d2dad4e4a98ef0fd5d3 in term 2.
I20260925 09:56:58.502892 10325 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 2 FOLLOWER]: Rejecting Update request from peer b8be057e0b6d40a3a4b90f2b1f4a0fa2 for earlier term 1. Current term is 2. Ops: []
I20260925 09:56:58.503698 10587 leader_election.cc:288] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [CANDIDATE]: Term 2 election: Requested vote from peers 54c8485a7dcc47a1a75d08ef57647ece (127.10.9.62:34455), b8be057e0b6d40a3a4b90f2b1f4a0fa2 (127.10.9.60:36985)
I20260925 09:56:58.508737 10379 leader_election.cc:302] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54c8485a7dcc47a1a75d08ef57647ece, 8af3f111e8da4d2dad4e4a98ef0fd5d3; no voters: b8be057e0b6d40a3a4b90f2b1f4a0fa2
I20260925 09:56:58.510140 10587 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 09:56:58.514842 10591 consensus_queue.cc:1046] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260925 09:56:58.516539 10589 raft_consensus.cc:3032] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 1 LEADER]: Stepping down as leader of term 1
I20260925 09:56:58.517673 10589 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b8be057e0b6d40a3a4b90f2b1f4a0fa2, State: Running, Role: LEADER
I20260925 09:56:58.519187 10589 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } }
I20260925 09:56:58.525815 10587 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [term 2 LEADER]: Becoming Leader. State: Replica: 8af3f111e8da4d2dad4e4a98ef0fd5d3, State: Running, Role: LEADER
I20260925 09:56:58.526937 10589 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:56:58.526759 10587 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } }
W20260925 09:56:58.533291 10590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:56:58.541394 10594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8af3f111e8da4d2dad4e4a98ef0fd5d3. Latest consensus state: current_term: 2 leader_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } } }
I20260925 09:56:58.542160 10594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [sys.catalog]: This master's current role is: LEADER
I20260925 09:56:58.543923 10599 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260925 09:56:58.557929 10593 catalog_manager.cc:1409] Failed waiting for node to catch up after master election: Illegal state: Node b8be057e0b6d40a3a4b90f2b1f4a0fa2 not leader. Raft Consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } } }
I20260925 09:56:58.560196 10599 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:56:58.565333 10599 catalog_manager.cc:1259] Loaded cluster ID: 189c454748024b14abd819435a918f04
I20260925 09:56:58.565594 10599 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:56:58.571066 10599 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:56:58.575632 10599 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Loaded TSK: 0
I20260925 09:56:58.577119 10599 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:56:58.878402 10534 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 09:56:58.984426 10326 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 2 FOLLOWER]: Refusing update from remote peer 8af3f111e8da4d2dad4e4a98ef0fd5d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260925 09:56:58.986678 10594 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260925 09:56:59.018355 10601 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8af3f111e8da4d2dad4e4a98ef0fd5d3. Latest consensus state: current_term: 2 leader_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } } }
I20260925 09:56:59.019210 10601 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:56:59.038457 10483 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 2 FOLLOWER]: Refusing update from remote peer 8af3f111e8da4d2dad4e4a98ef0fd5d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260925 09:56:59.040223 10594 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260925 09:56:59.040601 10601 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } } }
I20260925 09:56:59.041302 10601 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:56:59.045388 10594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } } }
I20260925 09:56:59.046347 10594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [sys.catalog]: This master's current role is: LEADER
I20260925 09:56:59.073694 10589 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8af3f111e8da4d2dad4e4a98ef0fd5d3. Latest consensus state: current_term: 2 leader_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } } }
I20260925 09:56:59.074443 10589 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:56:59.077471 10607 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } } }
I20260925 09:56:59.078281 10607 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:56:59.442296 10587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } } }
I20260925 09:56:59.443116 10587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [sys.catalog]: This master's current role is: LEADER
I20260925 09:56:59.911284 10608 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8af3f111e8da4d2dad4e4a98ef0fd5d3)
I20260925 09:56:59.911900 10608 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } }
I20260925 09:56:59.914645 10403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "54c8485a7dcc47a1a75d08ef57647ece" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" is_pre_election: true
I20260925 09:56:59.917089 10608 leader_election.cc:288] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8af3f111e8da4d2dad4e4a98ef0fd5d3 (127.10.9.61:33047), b8be057e0b6d40a3a4b90f2b1f4a0fa2 (127.10.9.60:36985)
I20260925 09:56:59.917727 10483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "54c8485a7dcc47a1a75d08ef57647ece" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" is_pre_election: true
I20260925 09:56:59.919173 10304 leader_election.cc:302] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54c8485a7dcc47a1a75d08ef57647ece; no voters: 8af3f111e8da4d2dad4e4a98ef0fd5d3, b8be057e0b6d40a3a4b90f2b1f4a0fa2
I20260925 09:56:59.920215 10608 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260925 09:56:59.956063 10609 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8af3f111e8da4d2dad4e4a98ef0fd5d3)
I20260925 09:56:59.956519 10609 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } }
I20260925 09:56:59.959299 10325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54c8485a7dcc47a1a75d08ef57647ece" is_pre_election: true
I20260925 09:56:59.959961 10325 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8be057e0b6d40a3a4b90f2b1f4a0fa2 in term 2.
I20260925 09:56:59.961524 10457 leader_election.cc:302] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54c8485a7dcc47a1a75d08ef57647ece, b8be057e0b6d40a3a4b90f2b1f4a0fa2; no voters:
I20260925 09:56:59.962510 10609 leader_election.cc:288] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 54c8485a7dcc47a1a75d08ef57647ece (127.10.9.62:34455), 8af3f111e8da4d2dad4e4a98ef0fd5d3 (127.10.9.61:33047)
I20260925 09:56:59.962986 10609 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260925 09:56:59.963287 10609 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8af3f111e8da4d2dad4e4a98ef0fd5d3)
I20260925 09:56:59.963655 10609 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 2 FOLLOWER]: Advancing to term 3
I20260925 09:56:59.976598 10403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" is_pre_election: true
I20260925 09:56:59.970738 10609 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } }
I20260925 09:56:59.982604 10483 raft_consensus.cc:1226] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 3 FOLLOWER]: Rejecting Update request from peer 8af3f111e8da4d2dad4e4a98ef0fd5d3 for earlier term 2. Current term is 3. Ops: []
I20260925 09:56:59.985931 10610 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260925 09:56:59.987138 10609 leader_election.cc:288] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [CANDIDATE]: Term 3 election: Requested vote from peers 54c8485a7dcc47a1a75d08ef57647ece (127.10.9.62:34455), 8af3f111e8da4d2dad4e4a98ef0fd5d3 (127.10.9.61:33047)
I20260925 09:56:59.987587 10325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54c8485a7dcc47a1a75d08ef57647ece"
I20260925 09:56:59.988148 10325 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 2 FOLLOWER]: Advancing to term 3
I20260925 09:56:59.988078 10403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3"
I20260925 09:56:59.995230 10325 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8be057e0b6d40a3a4b90f2b1f4a0fa2 in term 3.
I20260925 09:56:59.999084 10612 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [term 2 LEADER]: Stepping down as leader of term 2
I20260925 09:56:59.999469 10612 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8af3f111e8da4d2dad4e4a98ef0fd5d3, State: Running, Role: LEADER
I20260925 09:57:00.000250 10457 leader_election.cc:302] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54c8485a7dcc47a1a75d08ef57647ece, b8be057e0b6d40a3a4b90f2b1f4a0fa2; no voters: 8af3f111e8da4d2dad4e4a98ef0fd5d3
I20260925 09:57:00.000336 10612 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } }
I20260925 09:57:00.001209 10609 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 3 FOLLOWER]: Leader election won for term 3
I20260925 09:57:00.001643 10609 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 3 LEADER]: Becoming Leader. State: Replica: b8be057e0b6d40a3a4b90f2b1f4a0fa2, State: Running, Role: LEADER
I20260925 09:57:00.001786 10612 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [term 2 FOLLOWER]: Advancing to term 3
I20260925 09:57:00.002431 10609 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } }
I20260925 09:57:00.006853 10611 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8be057e0b6d40a3a4b90f2b1f4a0fa2. Latest consensus state: current_term: 3 leader_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } } }
I20260925 09:57:00.007598 10611 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [sys.catalog]: This master's current role is: LEADER
I20260925 09:57:00.029487 10613 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:57:00.044052 10613 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:57:00.047861 10613 catalog_manager.cc:1259] Loaded cluster ID: 189c454748024b14abd819435a918f04
I20260925 09:57:00.048115 10613 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:57:00.053107 10613 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:57:00.057292 10613 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2: Loaded TSK: 0
I20260925 09:57:00.058806 10613 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 09:56:58.551970 10597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:56:58.552067 10561 server_base.cc:1034] running on GCE node
W20260925 09:56:58.539513 10592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:57:00.209599 10561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:57:00.212421 10561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:57:00.213866 10561 hybrid_clock.cc:648] HybridClock initialized: now 1790330220213819 us; error 60 us; skew 500 ppm
I20260925 09:57:00.214901 10561 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:57:00.228422 10561 webserver.cc:460] Webserver started at http://127.10.9.1:35921/ using document root <none> and password file <none>
I20260925 09:57:00.229683 10561 fs_manager.cc:359] Metadata directory not provided
I20260925 09:57:00.229944 10561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:57:00.230490 10561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:57:00.235972 10561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/data/instance:
uuid: "b8850818e2ee42a082802d0095c08d21"
format_stamp: "Formatted at 2026-09-25 09:57:00 on dist-test-slave-4xvl"
I20260925 09:57:00.237354 10561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/instance:
uuid: "b8850818e2ee42a082802d0095c08d21"
format_stamp: "Formatted at 2026-09-25 09:57:00 on dist-test-slave-4xvl"
I20260925 09:57:00.245494 10561 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.004s
I20260925 09:57:00.256742 10618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:57:00.258066 10561 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.000s sys 0.009s
I20260925 09:57:00.258486 10561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal
uuid: "b8850818e2ee42a082802d0095c08d21"
format_stamp: "Formatted at 2026-09-25 09:57:00 on dist-test-slave-4xvl"
I20260925 09:57:00.258849 10561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:57:00.304237 10561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:57:00.305701 10561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:57:00.306182 10561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:57:00.310557 10561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:57:00.315356 10561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:57:00.315629 10561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260925 09:57:00.315922 10561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:57:00.316093 10561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:57:00.414733 10612 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:00.415346 10612 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } }
I20260925 09:57:00.418334 10612 leader_election.cc:288] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 54c8485a7dcc47a1a75d08ef57647ece (127.10.9.62:34455), b8be057e0b6d40a3a4b90f2b1f4a0fa2 (127.10.9.60:36985)
I20260925 09:57:00.421736 10325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54c8485a7dcc47a1a75d08ef57647ece" is_pre_election: true
I20260925 09:57:00.422576 10325 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8af3f111e8da4d2dad4e4a98ef0fd5d3 in term 3.
I20260925 09:57:00.423791 10483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" is_pre_election: true
I20260925 09:57:00.426635 10379 leader_election.cc:302] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54c8485a7dcc47a1a75d08ef57647ece, 8af3f111e8da4d2dad4e4a98ef0fd5d3; no voters:
I20260925 09:57:00.428392 10612 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260925 09:57:00.428866 10612 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:00.429337 10612 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [term 3 FOLLOWER]: Advancing to term 4
I20260925 09:57:00.439291 10612 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } }
I20260925 09:57:00.443147 10325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54c8485a7dcc47a1a75d08ef57647ece"
I20260925 09:57:00.443899 10325 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 3 FOLLOWER]: Advancing to term 4
I20260925 09:57:00.453812 10325 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8af3f111e8da4d2dad4e4a98ef0fd5d3 in term 4.
I20260925 09:57:00.456708 10483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2"
I20260925 09:57:00.458875 10612 leader_election.cc:288] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [CANDIDATE]: Term 4 election: Requested vote from peers 54c8485a7dcc47a1a75d08ef57647ece (127.10.9.62:34455), b8be057e0b6d40a3a4b90f2b1f4a0fa2 (127.10.9.60:36985)
I20260925 09:57:00.462208 10379 leader_election.cc:302] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54c8485a7dcc47a1a75d08ef57647ece, 8af3f111e8da4d2dad4e4a98ef0fd5d3; no voters: b8be057e0b6d40a3a4b90f2b1f4a0fa2
I20260925 09:57:00.463905 10612 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [term 4 FOLLOWER]: Leader election won for term 4
I20260925 09:57:00.464803 10612 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [term 4 LEADER]: Becoming Leader. State: Replica: 8af3f111e8da4d2dad4e4a98ef0fd5d3, State: Running, Role: LEADER
I20260925 09:57:00.465893 10612 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } }
I20260925 09:57:00.475263 10325 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 4 FOLLOWER]: Rejecting Update request from peer b8be057e0b6d40a3a4b90f2b1f4a0fa2 for earlier term 3. Current term is 4. Ops: []
I20260925 09:57:00.476775 10611 consensus_queue.cc:1046] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260925 09:57:00.477526 10611 raft_consensus.cc:3032] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 3 LEADER]: Stepping down as leader of term 3
I20260925 09:57:00.477787 10611 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b8be057e0b6d40a3a4b90f2b1f4a0fa2, State: Running, Role: LEADER
I20260925 09:57:00.478461 10611 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } }
I20260925 09:57:00.479764 10611 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 3 FOLLOWER]: Advancing to term 4
I20260925 09:57:00.480849 10610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8af3f111e8da4d2dad4e4a98ef0fd5d3. Latest consensus state: current_term: 4 leader_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } } }
I20260925 09:57:00.481858 10610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [sys.catalog]: This master's current role is: LEADER
I20260925 09:57:00.492574 10662 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:57:00.504419 10662 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:57:00.508803 10662 catalog_manager.cc:1259] Loaded cluster ID: 189c454748024b14abd819435a918f04
I20260925 09:57:00.509126 10662 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:57:00.516017 10662 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:57:00.522074 10662 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Loaded TSK: 0
I20260925 09:57:00.523368 10662 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:57:00.686828 10561 rpc_server.cc:307] RPC server started. Bound to: 127.10.9.1:33197
I20260925 09:57:00.686942 10733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.9.1:33197 every 8 connection(s)
I20260925 09:57:00.690188 10561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/data/info.pb
I20260925 09:57:00.696005 10276 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu as pid 10561
I20260925 09:57:00.696586 10276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/instance
I20260925 09:57:00.711716 10276 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu
/tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.9.2:0
--local_ip_for_outbound_sockets=127.10.9.2
--webserver_interface=127.10.9.2
--webserver_port=0
--tserver_master_addrs=127.10.9.62:34455,127.10.9.61:33047,127.10.9.60:36985
--builtin_ntp_servers=127.10.9.20:38279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260925 09:57:00.808331 10741 heartbeater.cc:347] Connected to a master server at 127.10.9.62:34455
I20260925 09:57:00.809000 10741 heartbeater.cc:464] Registering TS with master...
I20260925 09:57:00.810559 10741 heartbeater.cc:511] Master 127.10.9.62:34455 requested a full tablet report, sending...
I20260925 09:57:00.814642 10318 ts_manager.cc:194] Registered new tserver with Master: b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:00.825318 10736 heartbeater.cc:347] Connected to a master server at 127.10.9.61:33047
I20260925 09:57:00.825758 10736 heartbeater.cc:464] Registering TS with master...
I20260925 09:57:00.826730 10736 heartbeater.cc:511] Master 127.10.9.61:33047 requested a full tablet report, sending...
I20260925 09:57:00.829311 10734 heartbeater.cc:347] Connected to a master server at 127.10.9.60:36985
I20260925 09:57:00.829830 10734 heartbeater.cc:464] Registering TS with master...
I20260925 09:57:00.830147 10393 ts_manager.cc:194] Registered new tserver with Master: b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:00.831602 10734 heartbeater.cc:511] Master 127.10.9.60:36985 requested a full tablet report, sending...
I20260925 09:57:00.834028 10393 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.9.1:56563
I20260925 09:57:00.835317 10473 ts_manager.cc:194] Registered new tserver with Master: b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:00.949414 10325 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 4 FOLLOWER]: Refusing update from remote peer 8af3f111e8da4d2dad4e4a98ef0fd5d3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260925 09:57:00.950820 10610 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260925 09:57:00.958333 10745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8af3f111e8da4d2dad4e4a98ef0fd5d3. Latest consensus state: current_term: 4 leader_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } } }
I20260925 09:57:00.963762 10745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:57:00.985715 10745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } } }
I20260925 09:57:00.986435 10745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:57:00.990779 10483 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 4 FOLLOWER]: Refusing update from remote peer 8af3f111e8da4d2dad4e4a98ef0fd5d3: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260925 09:57:00.991115 10483 pending_rounds.cc:76] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2: Aborting all ops after (but not including) 5
I20260925 09:57:00.991345 10483 pending_rounds.cc:97] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260925 09:57:00.991621 10483 raft_consensus.cc:2866] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260925 09:57:00.993137 10612 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260925 09:57:00.999972 10612 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } } }
I20260925 09:57:01.000883 10612 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [sys.catalog]: This master's current role is: LEADER
I20260925 09:57:01.025960 10749 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8af3f111e8da4d2dad4e4a98ef0fd5d3. Latest consensus state: current_term: 4 leader_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } } }
I20260925 09:57:01.026711 10749 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:57:01.029404 10752 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } } }
I20260925 09:57:01.030102 10752 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 09:57:01.197629 10735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:57:01.198254 10735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:57:01.198874 10735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:57:01.230659 10735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:57:01.231833 10735 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.9.2
I20260925 09:57:01.269007 10735 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.9.20:38279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.9.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.9.2
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.10.9.62:34455,127.10.9.61:33047,127.10.9.60:36985
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.9.2
--log_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:06:08 UTC on ccycloud.quasar-lntfnh.root.comops.site
build id 4672078
TSAN enabled
I20260925 09:57:01.270277 10735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:57:01.271919 10735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:57:01.294987 10759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:57:01.343061 10736 heartbeater.cc:503] Master 127.10.9.61:33047 was elected leader, sending a full tablet report...
I20260925 09:57:01.488266 10610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } } }
I20260925 09:57:01.489537 10610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [sys.catalog]: This master's current role is: LEADER
W20260925 09:57:01.295928 10760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:57:01.297289 10762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:57:02.581158 10761 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1284 milliseconds
I20260925 09:57:02.581314 10735 server_base.cc:1029] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260925 09:57:02.582501 10735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:57:02.584671 10735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:57:02.586006 10735 hybrid_clock.cc:648] HybridClock initialized: now 1790330222585957 us; error 65 us; skew 500 ppm
I20260925 09:57:02.586768 10735 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:57:02.592896 10735 webserver.cc:460] Webserver started at http://127.10.9.2:33697/ using document root <none> and password file <none>
I20260925 09:57:02.593824 10735 fs_manager.cc:359] Metadata directory not provided
I20260925 09:57:02.594038 10735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:57:02.594499 10735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:57:02.599346 10735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/data/instance:
uuid: "c43461c4ee854ae1ac52b873ccada318"
format_stamp: "Formatted at 2026-09-25 09:57:02 on dist-test-slave-4xvl"
I20260925 09:57:02.600457 10735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/instance:
uuid: "c43461c4ee854ae1ac52b873ccada318"
format_stamp: "Formatted at 2026-09-25 09:57:02 on dist-test-slave-4xvl"
I20260925 09:57:02.607234 10735 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260925 09:57:02.612561 10771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:57:02.613600 10735 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.005s
I20260925 09:57:02.613936 10735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal
uuid: "c43461c4ee854ae1ac52b873ccada318"
format_stamp: "Formatted at 2026-09-25 09:57:02 on dist-test-slave-4xvl"
I20260925 09:57:02.614283 10735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:57:02.648916 10735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:57:02.650329 10735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:57:02.650731 10735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:57:02.654028 10735 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:57:02.658133 10735 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:57:02.658325 10735 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:57:02.658543 10735 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:57:02.658667 10735 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:57:02.863152 10735 rpc_server.cc:307] RPC server started. Bound to: 127.10.9.2:43257
I20260925 09:57:02.862784 10884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.9.2:43257 every 8 connection(s)
I20260925 09:57:02.866180 10735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/data/info.pb
I20260925 09:57:02.871834 10276 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu as pid 10735
I20260925 09:57:02.872639 10276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/instance
I20260925 09:57:02.905974 10276 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu
/tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.9.3:0
--local_ip_for_outbound_sockets=127.10.9.3
--webserver_interface=127.10.9.3
--webserver_port=0
--tserver_master_addrs=127.10.9.62:34455,127.10.9.61:33047,127.10.9.60:36985
--builtin_ntp_servers=127.10.9.20:38279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260925 09:57:02.934001 10887 heartbeater.cc:347] Connected to a master server at 127.10.9.62:34455
I20260925 09:57:02.934489 10887 heartbeater.cc:464] Registering TS with master...
I20260925 09:57:02.934890 10886 heartbeater.cc:347] Connected to a master server at 127.10.9.61:33047
I20260925 09:57:02.935106 10885 heartbeater.cc:347] Connected to a master server at 127.10.9.60:36985
I20260925 09:57:02.935279 10886 heartbeater.cc:464] Registering TS with master...
I20260925 09:57:02.935464 10885 heartbeater.cc:464] Registering TS with master...
I20260925 09:57:02.935761 10887 heartbeater.cc:511] Master 127.10.9.62:34455 requested a full tablet report, sending...
I20260925 09:57:02.936444 10886 heartbeater.cc:511] Master 127.10.9.61:33047 requested a full tablet report, sending...
I20260925 09:57:02.936484 10885 heartbeater.cc:511] Master 127.10.9.60:36985 requested a full tablet report, sending...
I20260925 09:57:02.938745 10393 ts_manager.cc:194] Registered new tserver with Master: c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:02.938967 10473 ts_manager.cc:194] Registered new tserver with Master: c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:02.939071 10318 ts_manager.cc:194] Registered new tserver with Master: c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:02.940742 10393 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.9.2:40761
W20260925 09:57:03.258841 10895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:57:03.259328 10895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:57:03.259938 10895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260925 09:57:03.271286 10900 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8af3f111e8da4d2dad4e4a98ef0fd5d3)
I20260925 09:57:03.271759 10900 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } }
I20260925 09:57:03.273834 10900 leader_election.cc:288] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 54c8485a7dcc47a1a75d08ef57647ece (127.10.9.62:34455), 8af3f111e8da4d2dad4e4a98ef0fd5d3 (127.10.9.61:33047)
I20260925 09:57:03.274891 10326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54c8485a7dcc47a1a75d08ef57647ece" is_pre_election: true
I20260925 09:57:03.274914 10403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" is_pre_election: true
I20260925 09:57:03.276329 10457 leader_election.cc:302] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8be057e0b6d40a3a4b90f2b1f4a0fa2; no voters: 54c8485a7dcc47a1a75d08ef57647ece, 8af3f111e8da4d2dad4e4a98ef0fd5d3
I20260925 09:57:03.276993 10900 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260925 09:57:03.295693 10895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:57:03.296633 10895 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.9.3
I20260925 09:57:03.331367 10895 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.9.20:38279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.9.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.9.3
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.10.9.62:34455,127.10.9.61:33047,127.10.9.60:36985
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.9.3
--log_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:06:08 UTC on ccycloud.quasar-lntfnh.root.comops.site
build id 4672078
TSAN enabled
I20260925 09:57:03.332683 10895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:57:03.334304 10895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:57:03.353165 10903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:57:03.443903 10886 heartbeater.cc:503] Master 127.10.9.61:33047 was elected leader, sending a full tablet report...
I20260925 09:57:03.709721 10909 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8af3f111e8da4d2dad4e4a98ef0fd5d3)
I20260925 09:57:03.710191 10909 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } }
I20260925 09:57:03.712518 10909 leader_election.cc:288] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8af3f111e8da4d2dad4e4a98ef0fd5d3 (127.10.9.61:33047), b8be057e0b6d40a3a4b90f2b1f4a0fa2 (127.10.9.60:36985)
I20260925 09:57:03.712985 10483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "54c8485a7dcc47a1a75d08ef57647ece" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" is_pre_election: true
I20260925 09:57:03.712971 10403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "54c8485a7dcc47a1a75d08ef57647ece" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" is_pre_election: true
I20260925 09:57:03.715023 10304 leader_election.cc:302] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54c8485a7dcc47a1a75d08ef57647ece; no voters: 8af3f111e8da4d2dad4e4a98ef0fd5d3, b8be057e0b6d40a3a4b90f2b1f4a0fa2
I20260925 09:57:03.715869 10909 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260925 09:57:04.282166 10910 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8af3f111e8da4d2dad4e4a98ef0fd5d3)
I20260925 09:57:04.282617 10910 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } }
I20260925 09:57:04.284370 10910 leader_election.cc:288] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8af3f111e8da4d2dad4e4a98ef0fd5d3 (127.10.9.61:33047), b8be057e0b6d40a3a4b90f2b1f4a0fa2 (127.10.9.60:36985)
I20260925 09:57:04.285490 10403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "54c8485a7dcc47a1a75d08ef57647ece" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" is_pre_election: true
I20260925 09:57:04.285696 10482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "54c8485a7dcc47a1a75d08ef57647ece" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" is_pre_election: true
I20260925 09:57:04.287093 10304 leader_election.cc:302] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54c8485a7dcc47a1a75d08ef57647ece; no voters: 8af3f111e8da4d2dad4e4a98ef0fd5d3, b8be057e0b6d40a3a4b90f2b1f4a0fa2
I20260925 09:57:04.287720 10910 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260925 09:57:03.353192 10904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:57:03.355221 10895 server_base.cc:1034] running on GCE node
W20260925 09:57:03.354243 10906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:57:04.592566 10895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:57:04.595448 10895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:57:04.596894 10895 hybrid_clock.cc:648] HybridClock initialized: now 1790330224596868 us; error 42 us; skew 500 ppm
I20260925 09:57:04.597765 10895 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:57:04.604697 10895 webserver.cc:460] Webserver started at http://127.10.9.3:46769/ using document root <none> and password file <none>
I20260925 09:57:04.605597 10895 fs_manager.cc:359] Metadata directory not provided
I20260925 09:57:04.605783 10895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:57:04.606204 10895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:57:04.611187 10895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/data/instance:
uuid: "be97ab5d75934a648d4ab76341193ec0"
format_stamp: "Formatted at 2026-09-25 09:57:04 on dist-test-slave-4xvl"
I20260925 09:57:04.612334 10895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/instance:
uuid: "be97ab5d75934a648d4ab76341193ec0"
format_stamp: "Formatted at 2026-09-25 09:57:04 on dist-test-slave-4xvl"
I20260925 09:57:04.619488 10895 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260925 09:57:04.625541 10915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:57:04.626653 10895 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260925 09:57:04.627002 10895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal
uuid: "be97ab5d75934a648d4ab76341193ec0"
format_stamp: "Formatted at 2026-09-25 09:57:04 on dist-test-slave-4xvl"
I20260925 09:57:04.627367 10895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:57:04.672199 10895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:57:04.674045 10895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:57:04.674587 10895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:57:04.678287 10895 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:57:04.684201 10895 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:57:04.684530 10895 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:57:04.684792 10895 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:57:04.684969 10895 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:57:04.787696 10965 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8af3f111e8da4d2dad4e4a98ef0fd5d3)
I20260925 09:57:04.788343 10965 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } }
I20260925 09:57:04.791890 10326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54c8485a7dcc47a1a75d08ef57647ece" is_pre_election: true
I20260925 09:57:04.792407 10965 leader_election.cc:288] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 54c8485a7dcc47a1a75d08ef57647ece (127.10.9.62:34455), 8af3f111e8da4d2dad4e4a98ef0fd5d3 (127.10.9.61:33047)
I20260925 09:57:04.792474 10403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" is_pre_election: true
I20260925 09:57:04.793838 10456 leader_election.cc:302] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8be057e0b6d40a3a4b90f2b1f4a0fa2; no voters: 54c8485a7dcc47a1a75d08ef57647ece, 8af3f111e8da4d2dad4e4a98ef0fd5d3
I20260925 09:57:04.795933 10965 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260925 09:57:04.954524 10895 rpc_server.cc:307] RPC server started. Bound to: 127.10.9.3:44553
I20260925 09:57:04.954631 11028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.9.3:44553 every 8 connection(s)
I20260925 09:57:04.958774 10895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/data/info.pb
I20260925 09:57:04.968825 10276 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu as pid 10895
I20260925 09:57:04.969427 10276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/instance
I20260925 09:57:05.034659 11029 heartbeater.cc:347] Connected to a master server at 127.10.9.60:36985
I20260925 09:57:05.035192 11029 heartbeater.cc:464] Registering TS with master...
I20260925 09:57:05.036419 11029 heartbeater.cc:511] Master 127.10.9.60:36985 requested a full tablet report, sending...
I20260925 09:57:05.040836 10473 ts_manager.cc:194] Registered new tserver with Master: be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:05.043718 11030 heartbeater.cc:347] Connected to a master server at 127.10.9.61:33047
I20260925 09:57:05.044047 11030 heartbeater.cc:464] Registering TS with master...
I20260925 09:57:05.044927 11030 heartbeater.cc:511] Master 127.10.9.61:33047 requested a full tablet report, sending...
I20260925 09:57:05.047735 10393 ts_manager.cc:194] Registered new tserver with Master: be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:05.049594 11032 heartbeater.cc:347] Connected to a master server at 127.10.9.62:34455
I20260925 09:57:05.049906 11032 heartbeater.cc:464] Registering TS with master...
I20260925 09:57:05.050241 10393 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.9.3:41469
I20260925 09:57:05.050593 11032 heartbeater.cc:511] Master 127.10.9.62:34455 requested a full tablet report, sending...
I20260925 09:57:05.053339 10318 ts_manager.cc:194] Registered new tserver with Master: be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:05.058815 10276 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 09:57:05.129967 10393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_4ba453ba891a48a48914167144b73c36"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:05.130023 10392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_0edbec4bbe134faca56b366a2ca9406a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:05.135140 10391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_297baa725d584db88d59cfd8abd00a09"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:05.137272 10390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_aa633fdad4a4453c81cb4cd34ee7a3f9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:05.138434 10392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0edbec4bbe134faca56b366a2ca9406a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 09:57:05.139808 10390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aa633fdad4a4453c81cb4cd34ee7a3f9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 09:57:05.141659 10393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4ba453ba891a48a48914167144b73c36 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 09:57:05.143039 10391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_297baa725d584db88d59cfd8abd00a09 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:05.182808 10276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu with pid 10430
I20260925 09:57:05.208971 10391 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_297baa725d584db88d59cfd8abd00a09" } new_table_name: "default.Table_ec7ca1fdd1ee40fb8ad4a27ea2c87216" modify_external_catalogs: true
I20260925 09:57:05.219678 10393 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_4ba453ba891a48a48914167144b73c36" } new_table_name: "default.Table_f8d0c571bc9a439e97daca9202484287" modify_external_catalogs: true
W20260925 09:57:05.223266 10379 connection.cc:535] client connection to 127.10.9.60:36985 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 09:57:05.223752 10379 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107) [suppressed 6 similar messages]
I20260925 09:57:05.234220 10392 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_0edbec4bbe134faca56b366a2ca9406a" } modify_external_catalogs: true
I20260925 09:57:05.234885 10392 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_0edbec4bbe134faca56b366a2ca9406a" } modify_external_catalogs: true
I20260925 09:57:05.235314 10390 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_aa633fdad4a4453c81cb4cd34ee7a3f9" } modify_external_catalogs: true
I20260925 09:57:05.235857 10390 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_aa633fdad4a4453c81cb4cd34ee7a3f9" } modify_external_catalogs: true
W20260925 09:57:05.242803 10379 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 -> Peer b8be057e0b6d40a3a4b90f2b1f4a0fa2 (127.10.9.60:36985): Couldn't send request to peer b8be057e0b6d40a3a4b90f2b1f4a0fa2. Status: Network error: Client connection negotiation failed: client connection to 127.10.9.60:36985: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260925 09:57:05.254421 10393 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 840faf042d8749bd958ce1de0207a734 has no consensus state
W20260925 09:57:05.255096 10391 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2d162bcd507943e18f32dc593200eb4c consensus state has no leader
W20260925 09:57:05.280269 10380 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 840faf042d8749bd958ce1de0207a734 (table default.Table_f8d0c571bc9a439e97daca9202484287 [id=800eba46046443a7ad196808dec4cdc0], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 840faf042d8749bd958ce1de0207a734 has no consensus state
I20260925 09:57:05.290341 10668 tablet_service.cc:1467] Processing CreateTablet for tablet 2d162bcd507943e18f32dc593200eb4c (DEFAULT_TABLE table=default.Table_ec7ca1fdd1ee40fb8ad4a27ea2c87216 [id=984ffc82af7b4737bae52327ee2a1755]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:05.293295 10668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d162bcd507943e18f32dc593200eb4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:05.314595 10390 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet a46067495fe746c3ba5bf249ac16001b
I20260925 09:57:05.315580 10392 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet f0ab8cbff26f4975a944271982049370
I20260925 09:57:05.341362 10819 tablet_service.cc:1467] Processing CreateTablet for tablet 2d162bcd507943e18f32dc593200eb4c (DEFAULT_TABLE table=default.Table_297baa725d584db88d59cfd8abd00a09 [id=984ffc82af7b4737bae52327ee2a1755]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:05.342798 10819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d162bcd507943e18f32dc593200eb4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:05.351657 10963 tablet_service.cc:1467] Processing CreateTablet for tablet 2d162bcd507943e18f32dc593200eb4c (DEFAULT_TABLE table=default.Table_ec7ca1fdd1ee40fb8ad4a27ea2c87216 [id=984ffc82af7b4737bae52327ee2a1755]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:05.354022 10963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d162bcd507943e18f32dc593200eb4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:05.368690 10667 tablet_service.cc:1467] Processing CreateTablet for tablet 840faf042d8749bd958ce1de0207a734 (DEFAULT_TABLE table=default.Table_f8d0c571bc9a439e97daca9202484287 [id=800eba46046443a7ad196808dec4cdc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:05.369194 10962 tablet_service.cc:1467] Processing CreateTablet for tablet 840faf042d8749bd958ce1de0207a734 (DEFAULT_TABLE table=default.Table_f8d0c571bc9a439e97daca9202484287 [id=800eba46046443a7ad196808dec4cdc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:05.370208 10667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 840faf042d8749bd958ce1de0207a734. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:05.370926 10962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 840faf042d8749bd958ce1de0207a734. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:05.372777 10818 tablet_service.cc:1467] Processing CreateTablet for tablet 840faf042d8749bd958ce1de0207a734 (DEFAULT_TABLE table=default.Table_f8d0c571bc9a439e97daca9202484287 [id=800eba46046443a7ad196808dec4cdc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:05.374238 10818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 840faf042d8749bd958ce1de0207a734. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:05.407368 11084 tablet_bootstrap.cc:492] T 840faf042d8749bd958ce1de0207a734 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
W20260925 09:57:05.409768 10734 heartbeater.cc:644] Failed to heartbeat to 127.10.9.60:36985 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.9.60:36985: connect: Connection refused (error 111)
I20260925 09:57:05.410348 10392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_869036da470d4b9e9f5afaa74644914a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:05.411528 11082 tablet_bootstrap.cc:492] T 2d162bcd507943e18f32dc593200eb4c P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
W20260925 09:57:05.412065 10392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_869036da470d4b9e9f5afaa74644914a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:05.416625 11084 tablet_bootstrap.cc:654] T 840faf042d8749bd958ce1de0207a734 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:05.422122 10390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_82582ea3080a431c9e13b00349420e93"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:05.423792 10390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_82582ea3080a431c9e13b00349420e93 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:05.432303 11084 log.cc:824] T 840faf042d8749bd958ce1de0207a734 P be97ab5d75934a648d4ab76341193ec0: Log is configured to *not* fsync() on all Append() calls
I20260925 09:57:05.440104 10391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_a400e3e492de40e988e9cb29284a2003"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:05.441783 10391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a400e3e492de40e988e9cb29284a2003 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:05.447150 11084 tablet_bootstrap.cc:492] T 840faf042d8749bd958ce1de0207a734 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:05.447896 11084 ts_tablet_manager.cc:1397] T 840faf042d8749bd958ce1de0207a734 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.041s user 0.014s sys 0.006s
I20260925 09:57:05.447638 10393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_cae21509a44f4fd9aac22d5fea74694b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:05.449276 10393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cae21509a44f4fd9aac22d5fea74694b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:05.452452 11085 tablet_bootstrap.cc:492] T 840faf042d8749bd958ce1de0207a734 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:05.461933 10389 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_ec7ca1fdd1ee40fb8ad4a27ea2c87216" } new_table_name: "default.Table_fda46197d74a46bc8635833ae25d2b7e" modify_external_catalogs: true
I20260925 09:57:05.463057 10392 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_f8d0c571bc9a439e97daca9202484287" } new_table_name: "default.Table_faa21794a80c407f800038e583c08f16" modify_external_catalogs: true
I20260925 09:57:05.464411 11085 tablet_bootstrap.cc:654] T 840faf042d8749bd958ce1de0207a734 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:05.466840 11085 log.cc:824] T 840faf042d8749bd958ce1de0207a734 P c43461c4ee854ae1ac52b873ccada318: Log is configured to *not* fsync() on all Append() calls
I20260925 09:57:05.490520 11085 tablet_bootstrap.cc:492] T 840faf042d8749bd958ce1de0207a734 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:05.491181 11085 ts_tablet_manager.cc:1397] T 840faf042d8749bd958ce1de0207a734 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.039s user 0.018s sys 0.000s
I20260925 09:57:05.505870 10668 tablet_service.cc:1467] Processing CreateTablet for tablet b3ca43e90141411bba9fe651845f8882 (DEFAULT_TABLE table=default.Table_869036da470d4b9e9f5afaa74644914a [id=91e443968e4c4e52b477199bfe2a77a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:05.507025 10668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3ca43e90141411bba9fe651845f8882. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:05.509909 11084 raft_consensus.cc:348] T 840faf042d8749bd958ce1de0207a734 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:05.510950 11084 raft_consensus.cc:374] T 840faf042d8749bd958ce1de0207a734 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:05.511314 11084 raft_consensus.cc:729] T 840faf042d8749bd958ce1de0207a734 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:05.512553 10962 tablet_service.cc:1467] Processing CreateTablet for tablet b3ca43e90141411bba9fe651845f8882 (DEFAULT_TABLE table=default.Table_869036da470d4b9e9f5afaa74644914a [id=91e443968e4c4e52b477199bfe2a77a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:05.515285 10962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3ca43e90141411bba9fe651845f8882. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:05.522395 10393 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_82582ea3080a431c9e13b00349420e93" } modify_external_catalogs: true
I20260925 09:57:05.522706 10391 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_869036da470d4b9e9f5afaa74644914a" } modify_external_catalogs: true
I20260925 09:57:05.522953 10393 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_82582ea3080a431c9e13b00349420e93" } modify_external_catalogs: true
I20260925 09:57:05.523203 10391 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_869036da470d4b9e9f5afaa74644914a" } modify_external_catalogs: true
W20260925 09:57:05.524427 10392 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 840faf042d8749bd958ce1de0207a734 consensus state has no leader
I20260925 09:57:05.526259 11085 raft_consensus.cc:348] T 840faf042d8749bd958ce1de0207a734 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:05.527285 11085 raft_consensus.cc:374] T 840faf042d8749bd958ce1de0207a734 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:05.512511 11084 consensus_queue.cc:260] T 840faf042d8749bd958ce1de0207a734 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:05.527863 10389 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2d162bcd507943e18f32dc593200eb4c consensus state has no leader
I20260925 09:57:05.529870 11085 raft_consensus.cc:729] T 840faf042d8749bd958ce1de0207a734 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:05.531421 11085 consensus_queue.cc:260] T 840faf042d8749bd958ce1de0207a734 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:05.536935 11084 ts_tablet_manager.cc:1428] T 840faf042d8749bd958ce1de0207a734 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.088s user 0.040s sys 0.001s
I20260925 09:57:05.538166 11084 tablet_bootstrap.cc:492] T 2d162bcd507943e18f32dc593200eb4c P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:05.538363 11030 heartbeater.cc:503] Master 127.10.9.61:33047 was elected leader, sending a full tablet report...
W20260925 09:57:05.539347 11029 heartbeater.cc:644] Failed to heartbeat to 127.10.9.60:36985 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.9.60:36985: connect: Connection refused (error 111)
W20260925 09:57:05.543277 10885 heartbeater.cc:644] Failed to heartbeat to 127.10.9.60:36985 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.9.60:36985: connect: Connection refused (error 111)
I20260925 09:57:05.545430 10819 tablet_service.cc:1467] Processing CreateTablet for tablet b3ca43e90141411bba9fe651845f8882 (DEFAULT_TABLE table=default.Table_869036da470d4b9e9f5afaa74644914a [id=91e443968e4c4e52b477199bfe2a77a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:05.541821 11085 ts_tablet_manager.cc:1428] T 840faf042d8749bd958ce1de0207a734 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.050s user 0.040s sys 0.000s
I20260925 09:57:05.546573 10819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3ca43e90141411bba9fe651845f8882. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:05.550973 11084 tablet_bootstrap.cc:654] T 2d162bcd507943e18f32dc593200eb4c P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:05.556239 11085 tablet_bootstrap.cc:492] T 2d162bcd507943e18f32dc593200eb4c P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:05.572738 11084 tablet_bootstrap.cc:492] T 2d162bcd507943e18f32dc593200eb4c P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:05.573297 11084 ts_tablet_manager.cc:1397] T 2d162bcd507943e18f32dc593200eb4c P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.035s user 0.008s sys 0.019s
I20260925 09:57:05.576216 11084 raft_consensus.cc:348] T 2d162bcd507943e18f32dc593200eb4c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:05.577018 11084 raft_consensus.cc:374] T 2d162bcd507943e18f32dc593200eb4c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:05.577392 11084 raft_consensus.cc:729] T 2d162bcd507943e18f32dc593200eb4c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:05.577926 10962 tablet_service.cc:1467] Processing CreateTablet for tablet d803e8490c564b3d8d5435374f93dc2b (DEFAULT_TABLE table=default.Table_a400e3e492de40e988e9cb29284a2003 [id=9230b1811a8a4faba58b239cf40992dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:05.578285 11082 tablet_bootstrap.cc:654] T 2d162bcd507943e18f32dc593200eb4c P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:05.578342 10392 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 840faf042d8749bd958ce1de0207a734 (table default.Table_faa21794a80c407f800038e583c08f16 [id=800eba46046443a7ad196808dec4cdc0]). Expected version 2 got 0
I20260925 09:57:05.578070 11084 consensus_queue.cc:260] T 2d162bcd507943e18f32dc593200eb4c P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:05.578965 10962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d803e8490c564b3d8d5435374f93dc2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:05.580569 11084 ts_tablet_manager.cc:1428] T 2d162bcd507943e18f32dc593200eb4c P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.007s user 0.001s sys 0.005s
I20260925 09:57:05.581420 11084 tablet_bootstrap.cc:492] T b3ca43e90141411bba9fe651845f8882 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:05.582569 10667 tablet_service.cc:1467] Processing CreateTablet for tablet 2b210d181fb0488187c193768f6b748e (DEFAULT_TABLE table=default.Table_82582ea3080a431c9e13b00349420e93 [id=7a12ae6dbd154c9eb1a25add3a9aee9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:05.584189 10667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b210d181fb0488187c193768f6b748e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:05.584908 10668 tablet_service.cc:1467] Processing CreateTablet for tablet d803e8490c564b3d8d5435374f93dc2b (DEFAULT_TABLE table=default.Table_a400e3e492de40e988e9cb29284a2003 [id=9230b1811a8a4faba58b239cf40992dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:05.585876 10668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d803e8490c564b3d8d5435374f93dc2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:05.587299 11084 tablet_bootstrap.cc:654] T b3ca43e90141411bba9fe651845f8882 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:05.589489 11082 log.cc:824] T 2d162bcd507943e18f32dc593200eb4c P b8850818e2ee42a082802d0095c08d21: Log is configured to *not* fsync() on all Append() calls
I20260925 09:57:05.591773 11085 tablet_bootstrap.cc:654] T 2d162bcd507943e18f32dc593200eb4c P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:05.592747 10963 tablet_service.cc:1467] Processing CreateTablet for tablet 2b210d181fb0488187c193768f6b748e (DEFAULT_TABLE table=default.Table_82582ea3080a431c9e13b00349420e93 [id=7a12ae6dbd154c9eb1a25add3a9aee9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:05.593963 10963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b210d181fb0488187c193768f6b748e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:05.593741 10818 tablet_service.cc:1467] Processing CreateTablet for tablet d803e8490c564b3d8d5435374f93dc2b (DEFAULT_TABLE table=default.Table_a400e3e492de40e988e9cb29284a2003 [id=9230b1811a8a4faba58b239cf40992dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:05.594954 10818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d803e8490c564b3d8d5435374f93dc2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:05.600401 10817 tablet_service.cc:1467] Processing CreateTablet for tablet 2b210d181fb0488187c193768f6b748e (DEFAULT_TABLE table=default.Table_82582ea3080a431c9e13b00349420e93 [id=7a12ae6dbd154c9eb1a25add3a9aee9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:05.601652 10817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b210d181fb0488187c193768f6b748e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:05.608946 11085 tablet_bootstrap.cc:492] T 2d162bcd507943e18f32dc593200eb4c P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:05.609525 11085 ts_tablet_manager.cc:1397] T 2d162bcd507943e18f32dc593200eb4c P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.053s user 0.012s sys 0.000s
I20260925 09:57:05.612977 11082 tablet_bootstrap.cc:492] T 2d162bcd507943e18f32dc593200eb4c P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:05.613596 11082 ts_tablet_manager.cc:1397] T 2d162bcd507943e18f32dc593200eb4c P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.203s user 0.010s sys 0.008s
I20260925 09:57:05.615306 10393 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 2b210d181fb0488187c193768f6b748e
I20260925 09:57:05.618134 10666 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b210d181fb0488187c193768f6b748e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:05 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:05.629271 11085 raft_consensus.cc:348] T 2d162bcd507943e18f32dc593200eb4c P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:05.630568 10961 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b210d181fb0488187c193768f6b748e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:05 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:05.636035 11085 raft_consensus.cc:374] T 2d162bcd507943e18f32dc593200eb4c P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260925 09:57:05.636888 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 2b210d181fb0488187c193768f6b748e because the tablet was not found. No further retry: Not found: Tablet not found: 2b210d181fb0488187c193768f6b748e
I20260925 09:57:05.638139 11084 tablet_bootstrap.cc:492] T b3ca43e90141411bba9fe651845f8882 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:05.638898 11084 ts_tablet_manager.cc:1397] T b3ca43e90141411bba9fe651845f8882 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.058s user 0.007s sys 0.005s
I20260925 09:57:05.640691 10389 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 840faf042d8749bd958ce1de0207a734 (table default.Table_faa21794a80c407f800038e583c08f16 [id=800eba46046443a7ad196808dec4cdc0]). Expected version 2 got 0
I20260925 09:57:05.641212 11085 raft_consensus.cc:729] T 2d162bcd507943e18f32dc593200eb4c P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:05.641330 10389 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 2d162bcd507943e18f32dc593200eb4c (table default.Table_fda46197d74a46bc8635833ae25d2b7e [id=984ffc82af7b4737bae52327ee2a1755]). Expected version 2 got 0
I20260925 09:57:05.641572 11084 raft_consensus.cc:348] T b3ca43e90141411bba9fe651845f8882 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:05.642464 11084 raft_consensus.cc:374] T b3ca43e90141411bba9fe651845f8882 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:05.642750 11084 raft_consensus.cc:729] T b3ca43e90141411bba9fe651845f8882 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:05.642778 11085 consensus_queue.cc:260] T 2d162bcd507943e18f32dc593200eb4c P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:05.643725 10391 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet b3ca43e90141411bba9fe651845f8882
I20260925 09:57:05.643527 11084 consensus_queue.cc:260] T b3ca43e90141411bba9fe651845f8882 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:05.649397 11085 ts_tablet_manager.cc:1428] T 2d162bcd507943e18f32dc593200eb4c P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.035s user 0.006s sys 0.000s
I20260925 09:57:05.650539 10392 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 840faf042d8749bd958ce1de0207a734 (table default.Table_faa21794a80c407f800038e583c08f16 [id=800eba46046443a7ad196808dec4cdc0]). Expected version 2 got 0
I20260925 09:57:05.655190 10961 tablet_service.cc:1514] Processing DeleteTablet for tablet b3ca43e90141411bba9fe651845f8882 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:05 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:05.657737 11084 ts_tablet_manager.cc:1428] T b3ca43e90141411bba9fe651845f8882 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.019s user 0.004s sys 0.004s
I20260925 09:57:05.658674 11096 tablet_replica.cc:330] T b3ca43e90141411bba9fe651845f8882 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:05.659739 11096 raft_consensus.cc:2229] T b3ca43e90141411bba9fe651845f8882 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:05.659900 10389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_bcf18c926c6e4a62b31f3bffe9b0037e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:05.659826 11080 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } } }
I20260925 09:57:05.660321 11096 raft_consensus.cc:2258] T b3ca43e90141411bba9fe651845f8882 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:05.660645 11080 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [sys.catalog]: This master's current role is: LEADER
W20260925 09:57:05.661594 10389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bcf18c926c6e4a62b31f3bffe9b0037e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:05.657018 10392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_a04cfff92f844c2db530629705ab39c2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:05.663306 11085 tablet_bootstrap.cc:492] T b3ca43e90141411bba9fe651845f8882 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
W20260925 09:57:05.663663 10392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a04cfff92f844c2db530629705ab39c2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:05.664731 11096 ts_tablet_manager.cc:1905] T b3ca43e90141411bba9fe651845f8882 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:05.667476 10818 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b210d181fb0488187c193768f6b748e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:05 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:05.670058 10817 tablet_service.cc:1514] Processing DeleteTablet for tablet b3ca43e90141411bba9fe651845f8882 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:05 UTC) from {username='slave'} at 127.0.0.1:33344
W20260925 09:57:05.679606 10892 tablet.cc:2369] T 2d162bcd507943e18f32dc593200eb4c P c43461c4ee854ae1ac52b873ccada318: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:57:05.681353 11085 tablet_bootstrap.cc:654] T b3ca43e90141411bba9fe651845f8882 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:05.686431 10390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_000f52fd1e214fa1931ef3743ce76250"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:05.688655 10390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_000f52fd1e214fa1931ef3743ce76250 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:05.691810 10665 tablet_service.cc:1514] Processing DeleteTablet for tablet b3ca43e90141411bba9fe651845f8882 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:05 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:05.707624 11084 tablet_bootstrap.cc:492] T d803e8490c564b3d8d5435374f93dc2b P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:05.713649 11084 tablet_bootstrap.cc:654] T d803e8490c564b3d8d5435374f93dc2b P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:05.725312 10392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_9124bbcbac0548f997ed669cd39d50c4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:05.727104 10392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9124bbcbac0548f997ed669cd39d50c4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:05.727738 10388 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_fda46197d74a46bc8635833ae25d2b7e" } new_table_name: "default.Table_1298469ed8ca4d74b485794a4a7a7dca" modify_external_catalogs: true
W20260925 09:57:05.730192 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 2b210d181fb0488187c193768f6b748e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2b210d181fb0488187c193768f6b748e already in progress: creating tablet
W20260925 09:57:05.731298 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 2b210d181fb0488187c193768f6b748e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2b210d181fb0488187c193768f6b748e already in progress: creating tablet
I20260925 09:57:05.731900 10387 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_a04cfff92f844c2db530629705ab39c2" } new_table_name: "default.Table_924d66e4d68b4a79b062e6fd7a16878f" modify_external_catalogs: true
W20260925 09:57:05.732137 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet b3ca43e90141411bba9fe651845f8882 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b3ca43e90141411bba9fe651845f8882 already in progress: creating tablet
W20260925 09:57:05.741185 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet b3ca43e90141411bba9fe651845f8882 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b3ca43e90141411bba9fe651845f8882 already in progress: creating tablet
I20260925 09:57:05.757130 11085 tablet_bootstrap.cc:492] T b3ca43e90141411bba9fe651845f8882 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:05.757647 11085 ts_tablet_manager.cc:1397] T b3ca43e90141411bba9fe651845f8882 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.095s user 0.026s sys 0.026s
I20260925 09:57:05.760370 11085 raft_consensus.cc:348] T b3ca43e90141411bba9fe651845f8882 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:05.761232 11085 raft_consensus.cc:374] T b3ca43e90141411bba9fe651845f8882 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:05.762760 11085 raft_consensus.cc:729] T b3ca43e90141411bba9fe651845f8882 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:05.763800 11085 consensus_queue.cc:260] T b3ca43e90141411bba9fe651845f8882 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:05.766418 11096 ts_tablet_manager.cc:1918] T b3ca43e90141411bba9fe651845f8882 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:05.766911 11096 log.cc:1196] T b3ca43e90141411bba9fe651845f8882 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/b3ca43e90141411bba9fe651845f8882
I20260925 09:57:05.755967 11084 tablet_bootstrap.cc:492] T d803e8490c564b3d8d5435374f93dc2b P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:05.769985 11084 ts_tablet_manager.cc:1397] T d803e8490c564b3d8d5435374f93dc2b P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.063s user 0.010s sys 0.008s
I20260925 09:57:05.772866 11085 ts_tablet_manager.cc:1428] T b3ca43e90141411bba9fe651845f8882 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.015s user 0.005s sys 0.000s
I20260925 09:57:05.772753 11084 raft_consensus.cc:348] T d803e8490c564b3d8d5435374f93dc2b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:05.774442 11084 raft_consensus.cc:374] T d803e8490c564b3d8d5435374f93dc2b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:05.774801 11084 raft_consensus.cc:729] T d803e8490c564b3d8d5435374f93dc2b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:05.774139 11096 ts_tablet_manager.cc:1939] T b3ca43e90141411bba9fe651845f8882 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:05.778255 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet b3ca43e90141411bba9fe651845f8882 (table default.Table_869036da470d4b9e9f5afaa74644914a [id=91e443968e4c4e52b477199bfe2a77a3]) successfully deleted
I20260925 09:57:05.779743 10393 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 2d162bcd507943e18f32dc593200eb4c (table default.Table_fda46197d74a46bc8635833ae25d2b7e [id=984ffc82af7b4737bae52327ee2a1755]). Expected version 2 got 0
W20260925 09:57:05.780501 11036 tablet.cc:2369] T 2d162bcd507943e18f32dc593200eb4c P be97ab5d75934a648d4ab76341193ec0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:57:05.780582 10963 tablet_service.cc:1467] Processing CreateTablet for tablet 6a5121b78e2d4643b99bd8717c6e93ca (DEFAULT_TABLE table=default.Table_cae21509a44f4fd9aac22d5fea74694b [id=aa7e7f5af2724af797170e1f7429d76a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:05.781745 10963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a5121b78e2d4643b99bd8717c6e93ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:05.782218 10391 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 2d162bcd507943e18f32dc593200eb4c (table default.Table_fda46197d74a46bc8635833ae25d2b7e [id=984ffc82af7b4737bae52327ee2a1755]). Expected version 2 got 0
I20260925 09:57:05.783030 11085 tablet_bootstrap.cc:492] T 2b210d181fb0488187c193768f6b748e P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
W20260925 09:57:05.783677 10388 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2d162bcd507943e18f32dc593200eb4c consensus state has no leader
I20260925 09:57:05.788409 10668 tablet_service.cc:1467] Processing CreateTablet for tablet 6a5121b78e2d4643b99bd8717c6e93ca (DEFAULT_TABLE table=default.Table_cae21509a44f4fd9aac22d5fea74694b [id=aa7e7f5af2724af797170e1f7429d76a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:05.788784 10962 tablet_service.cc:1514] Processing DeleteTablet for tablet b3ca43e90141411bba9fe651845f8882 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:05 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:05.789216 11085 tablet_bootstrap.cc:654] T 2b210d181fb0488187c193768f6b748e P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:05.789718 10668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a5121b78e2d4643b99bd8717c6e93ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:05.792513 10818 tablet_service.cc:1467] Processing CreateTablet for tablet 6a5121b78e2d4643b99bd8717c6e93ca (DEFAULT_TABLE table=default.Table_cae21509a44f4fd9aac22d5fea74694b [id=aa7e7f5af2724af797170e1f7429d76a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:05.793639 10818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a5121b78e2d4643b99bd8717c6e93ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:05.795747 11084 consensus_queue.cc:260] T d803e8490c564b3d8d5435374f93dc2b P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:05.803367 11085 tablet_bootstrap.cc:492] T 2b210d181fb0488187c193768f6b748e P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:05.803931 11085 ts_tablet_manager.cc:1397] T 2b210d181fb0488187c193768f6b748e P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.021s user 0.008s sys 0.008s
I20260925 09:57:05.806349 11085 raft_consensus.cc:348] T 2b210d181fb0488187c193768f6b748e P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:05.807183 11085 raft_consensus.cc:374] T 2b210d181fb0488187c193768f6b748e P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:05.807518 11085 raft_consensus.cc:729] T 2b210d181fb0488187c193768f6b748e P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:05.808284 11085 consensus_queue.cc:260] T 2b210d181fb0488187c193768f6b748e P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:05.811378 11085 ts_tablet_manager.cc:1428] T 2b210d181fb0488187c193768f6b748e P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260925 09:57:05.812299 11085 tablet_bootstrap.cc:492] T d803e8490c564b3d8d5435374f93dc2b P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:05.816731 11084 ts_tablet_manager.cc:1428] T d803e8490c564b3d8d5435374f93dc2b P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.046s user 0.006s sys 0.000s
W20260925 09:57:05.816928 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet b3ca43e90141411bba9fe651845f8882 because the tablet was not found. No further retry: Not found: Tablet not found: b3ca43e90141411bba9fe651845f8882
I20260925 09:57:05.817981 11085 tablet_bootstrap.cc:654] T d803e8490c564b3d8d5435374f93dc2b P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:05.822923 11084 tablet_bootstrap.cc:492] T 2b210d181fb0488187c193768f6b748e P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:05.829882 11085 tablet_bootstrap.cc:492] T d803e8490c564b3d8d5435374f93dc2b P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:05.830334 11085 ts_tablet_manager.cc:1397] T d803e8490c564b3d8d5435374f93dc2b P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.018s user 0.016s sys 0.000s
I20260925 09:57:05.830519 10817 tablet_service.cc:1514] Processing DeleteTablet for tablet b3ca43e90141411bba9fe651845f8882 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:05 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:05.831790 11098 tablet_replica.cc:330] T b3ca43e90141411bba9fe651845f8882 P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:05.832762 11098 raft_consensus.cc:2229] T b3ca43e90141411bba9fe651845f8882 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:05.833314 11098 raft_consensus.cc:2258] T b3ca43e90141411bba9fe651845f8882 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:05.833534 11085 raft_consensus.cc:348] T d803e8490c564b3d8d5435374f93dc2b P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:05.834262 11085 raft_consensus.cc:374] T d803e8490c564b3d8d5435374f93dc2b P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:05.834581 11085 raft_consensus.cc:729] T d803e8490c564b3d8d5435374f93dc2b P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:05.835260 11085 consensus_queue.cc:260] T d803e8490c564b3d8d5435374f93dc2b P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:05.838958 11085 ts_tablet_manager.cc:1428] T d803e8490c564b3d8d5435374f93dc2b P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260925 09:57:05.843887 11090 raft_consensus.cc:482] T 2b210d181fb0488187c193768f6b748e P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:05.844329 11090 raft_consensus.cc:504] T 2b210d181fb0488187c193768f6b748e P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:05.846896 11090 leader_election.cc:288] T 2b210d181fb0488187c193768f6b748e P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:05.856184 11084 tablet_bootstrap.cc:654] T 2b210d181fb0488187c193768f6b748e P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:05.857190 10391 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_a400e3e492de40e988e9cb29284a2003" } modify_external_catalogs: true
I20260925 09:57:05.857771 10391 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_a400e3e492de40e988e9cb29284a2003" } modify_external_catalogs: true
I20260925 09:57:05.859987 11082 raft_consensus.cc:348] T 2d162bcd507943e18f32dc593200eb4c P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:05.860937 11082 raft_consensus.cc:374] T 2d162bcd507943e18f32dc593200eb4c P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:05.861232 11082 raft_consensus.cc:729] T 2d162bcd507943e18f32dc593200eb4c P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:05.868115 11098 ts_tablet_manager.cc:1905] T b3ca43e90141411bba9fe651845f8882 P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:05.869369 11082 consensus_queue.cc:260] T 2d162bcd507943e18f32dc593200eb4c P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:05.875111 11084 tablet_bootstrap.cc:492] T 2b210d181fb0488187c193768f6b748e P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:05.877280 11084 ts_tablet_manager.cc:1397] T 2b210d181fb0488187c193768f6b748e P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.055s user 0.008s sys 0.004s
I20260925 09:57:05.882781 11084 raft_consensus.cc:348] T 2b210d181fb0488187c193768f6b748e P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:05.887635 11084 raft_consensus.cc:374] T 2b210d181fb0488187c193768f6b748e P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:05.889541 11082 ts_tablet_manager.cc:1428] T 2d162bcd507943e18f32dc593200eb4c P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.276s user 0.045s sys 0.001s
I20260925 09:57:05.890194 11084 raft_consensus.cc:729] T 2b210d181fb0488187c193768f6b748e P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:05.891489 11084 consensus_queue.cc:260] T 2b210d181fb0488187c193768f6b748e P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
W20260925 09:57:05.893563 10387 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bbf85a35b22a4c5cbda092877818713f has no consensus state
I20260925 09:57:05.894078 11082 tablet_bootstrap.cc:492] T 840faf042d8749bd958ce1de0207a734 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:05.895422 11090 raft_consensus.cc:482] T 2d162bcd507943e18f32dc593200eb4c P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:05.895851 11090 raft_consensus.cc:504] T 2d162bcd507943e18f32dc593200eb4c P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:05.897570 11090 leader_election.cc:288] T 2d162bcd507943e18f32dc593200eb4c P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:05.899178 11084 ts_tablet_manager.cc:1428] T 2b210d181fb0488187c193768f6b748e P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.021s user 0.003s sys 0.002s
I20260925 09:57:05.900842 11084 tablet_bootstrap.cc:492] T 6a5121b78e2d4643b99bd8717c6e93ca P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:05.906615 10387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_236d6704c5ef4177bf8a34d756e1c74c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:05.907972 11084 tablet_bootstrap.cc:654] T 6a5121b78e2d4643b99bd8717c6e93ca P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:05.908303 10387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_236d6704c5ef4177bf8a34d756e1c74c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:05.909271 11082 tablet_bootstrap.cc:654] T 840faf042d8749bd958ce1de0207a734 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:05.909085 10393 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_1298469ed8ca4d74b485794a4a7a7dca" } modify_external_catalogs: true
I20260925 09:57:05.909610 10393 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_1298469ed8ca4d74b485794a4a7a7dca" } modify_external_catalogs: true
I20260925 09:57:05.930143 10817 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b210d181fb0488187c193768f6b748e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:05 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:05.941473 11109 raft_consensus.cc:482] T 2d162bcd507943e18f32dc593200eb4c P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:05.942054 11109 raft_consensus.cc:504] T 2d162bcd507943e18f32dc593200eb4c P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:05.952718 10386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_ebff7801e9fa43ffafe5b8db4327e6f7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:05.954701 10386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ebff7801e9fa43ffafe5b8db4327e6f7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:05.956970 11109 leader_election.cc:288] T 2d162bcd507943e18f32dc593200eb4c P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:05.981447 11089 raft_consensus.cc:482] T 840faf042d8749bd958ce1de0207a734 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:05.982182 11089 raft_consensus.cc:504] T 840faf042d8749bd958ce1de0207a734 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:05.990110 11089 leader_election.cc:288] T 840faf042d8749bd958ce1de0207a734 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:05.991380 10962 tablet_service.cc:1467] Processing CreateTablet for tablet a0976e0c01234b38a7ab934dc1b289c4 (DEFAULT_TABLE table=default.Table_bcf18c926c6e4a62b31f3bffe9b0037e [id=53df4f0ffd794afab1aa17fcfb4d56bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:05.991467 10963 tablet_service.cc:1467] Processing CreateTablet for tablet bbf85a35b22a4c5cbda092877818713f (DEFAULT_TABLE table=default.Table_924d66e4d68b4a79b062e6fd7a16878f [id=f2b16afb131f4ef5a1e62a27dcce0374]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:05.992777 10962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0976e0c01234b38a7ab934dc1b289c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:05.997766 10392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_f2634f079dff40ac91f6c47ba66482e6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:05.988677 10388 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 2d162bcd507943e18f32dc593200eb4c (table default.Table_1298469ed8ca4d74b485794a4a7a7dca [id=984ffc82af7b4737bae52327ee2a1755]). Expected version 3 got 0
I20260925 09:57:05.999497 11090 raft_consensus.cc:482] T d803e8490c564b3d8d5435374f93dc2b P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:06.000344 11090 raft_consensus.cc:504] T d803e8490c564b3d8d5435374f93dc2b P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:06.000823 10963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbf85a35b22a4c5cbda092877818713f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:06.002159 11090 leader_election.cc:288] T d803e8490c564b3d8d5435374f93dc2b P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:05.997766 10387 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_9124bbcbac0548f997ed669cd39d50c4" } new_table_name: "default.Table_aefaa3aacf0e43e1b32aa184267892d4" modify_external_catalogs: true
W20260925 09:57:06.006469 10392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f2634f079dff40ac91f6c47ba66482e6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 09:57:06.020664 10379 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.62:34455, user_credentials={real_user=slave}} blocked reactor thread for 47622.1us
I20260925 09:57:06.026439 11085 tablet_bootstrap.cc:492] T 6a5121b78e2d4643b99bd8717c6e93ca P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:06.027479 11084 tablet_bootstrap.cc:492] T 6a5121b78e2d4643b99bd8717c6e93ca P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:06.027940 11084 ts_tablet_manager.cc:1397] T 6a5121b78e2d4643b99bd8717c6e93ca P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.127s user 0.024s sys 0.032s
I20260925 09:57:06.030679 11084 raft_consensus.cc:348] T 6a5121b78e2d4643b99bd8717c6e93ca P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:06.032351 11085 tablet_bootstrap.cc:654] T 6a5121b78e2d4643b99bd8717c6e93ca P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:06.037950 10688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b210d181fb0488187c193768f6b748e" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:06.042706 10688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d162bcd507943e18f32dc593200eb4c" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:06.042858 11084 raft_consensus.cc:374] T 6a5121b78e2d4643b99bd8717c6e93ca P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:06.043198 11084 raft_consensus.cc:729] T 6a5121b78e2d4643b99bd8717c6e93ca P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:06.043336 10688 raft_consensus.cc:2454] T 2d162bcd507943e18f32dc593200eb4c P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
I20260925 09:57:06.044467 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d803e8490c564b3d8d5435374f93dc2b" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:06.044443 11084 consensus_queue.cc:260] T 6a5121b78e2d4643b99bd8717c6e93ca P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:06.048482 10775 leader_election.cc:341] T 2b210d181fb0488187c193768f6b748e P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:06.056563 10775 leader_election.cc:302] T 2d162bcd507943e18f32dc593200eb4c P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:06.057868 11084 ts_tablet_manager.cc:1428] T 6a5121b78e2d4643b99bd8717c6e93ca P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.030s user 0.007s sys 0.000s
I20260925 09:57:06.060861 11082 tablet_bootstrap.cc:492] T 840faf042d8749bd958ce1de0207a734 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:06.061321 11082 ts_tablet_manager.cc:1397] T 840faf042d8749bd958ce1de0207a734 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.167s user 0.026s sys 0.034s
I20260925 09:57:06.064409 11082 raft_consensus.cc:348] T 840faf042d8749bd958ce1de0207a734 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:06.065267 11082 raft_consensus.cc:374] T 840faf042d8749bd958ce1de0207a734 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:06.065610 11082 raft_consensus.cc:729] T 840faf042d8749bd958ce1de0207a734 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:06.066341 11082 consensus_queue.cc:260] T 840faf042d8749bd958ce1de0207a734 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:06.073055 11082 ts_tablet_manager.cc:1428] T 840faf042d8749bd958ce1de0207a734 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20260925 09:57:06.074028 11082 tablet_bootstrap.cc:492] T b3ca43e90141411bba9fe651845f8882 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:06.074944 10668 tablet_service.cc:1467] Processing CreateTablet for tablet bbf85a35b22a4c5cbda092877818713f (DEFAULT_TABLE table=default.Table_924d66e4d68b4a79b062e6fd7a16878f [id=f2b16afb131f4ef5a1e62a27dcce0374]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:06.077438 10668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbf85a35b22a4c5cbda092877818713f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:06.080466 11082 tablet_bootstrap.cc:654] T b3ca43e90141411bba9fe651845f8882 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:06.082026 10742 tablet.cc:2369] T 2d162bcd507943e18f32dc593200eb4c P b8850818e2ee42a082802d0095c08d21: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:57:06.089011 10666 tablet_service.cc:1467] Processing CreateTablet for tablet a0976e0c01234b38a7ab934dc1b289c4 (DEFAULT_TABLE table=default.Table_bcf18c926c6e4a62b31f3bffe9b0037e [id=53df4f0ffd794afab1aa17fcfb4d56bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:06.090427 10391 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet d803e8490c564b3d8d5435374f93dc2b
I20260925 09:57:06.090690 10666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0976e0c01234b38a7ab934dc1b289c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:06.092957 11082 tablet_bootstrap.cc:492] T b3ca43e90141411bba9fe651845f8882 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:06.093495 11082 ts_tablet_manager.cc:1397] T b3ca43e90141411bba9fe651845f8882 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.020s user 0.013s sys 0.004s
I20260925 09:57:06.097819 11082 raft_consensus.cc:348] T b3ca43e90141411bba9fe651845f8882 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:06.098712 11082 raft_consensus.cc:374] T b3ca43e90141411bba9fe651845f8882 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:06.098455 10390 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_236d6704c5ef4177bf8a34d756e1c74c" } new_table_name: "default.Table_a122f131534b429ab5465d14f64c25a3" modify_external_catalogs: true
I20260925 09:57:06.099030 11082 raft_consensus.cc:729] T b3ca43e90141411bba9fe651845f8882 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:06.100394 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "840faf042d8749bd958ce1de0207a734" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:06.100690 10667 tablet_service.cc:1514] Processing DeleteTablet for tablet d803e8490c564b3d8d5435374f93dc2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:05 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:06.100991 10687 raft_consensus.cc:2454] T 840faf042d8749bd958ce1de0207a734 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be97ab5d75934a648d4ab76341193ec0 in term 0.
W20260925 09:57:06.100987 10775 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 44604.7us
I20260925 09:57:06.069474 10817 tablet_service.cc:1467] Processing CreateTablet for tablet a0976e0c01234b38a7ab934dc1b289c4 (DEFAULT_TABLE table=default.Table_bcf18c926c6e4a62b31f3bffe9b0037e [id=53df4f0ffd794afab1aa17fcfb4d56bc]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 09:57:06.101699 10775 leader_election.cc:341] T d803e8490c564b3d8d5435374f93dc2b P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:06.102371 10817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0976e0c01234b38a7ab934dc1b289c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:06.106202 11090 raft_consensus.cc:2781] T 2d162bcd507943e18f32dc593200eb4c P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:06.106562 11090 raft_consensus.cc:482] T 2d162bcd507943e18f32dc593200eb4c P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:06.106928 11090 raft_consensus.cc:3037] T 2d162bcd507943e18f32dc593200eb4c P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:06.107100 10819 tablet_service.cc:1514] Processing DeleteTablet for tablet d803e8490c564b3d8d5435374f93dc2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:05 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:06.069814 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d162bcd507943e18f32dc593200eb4c" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:06.072394 10818 tablet_service.cc:1467] Processing CreateTablet for tablet bbf85a35b22a4c5cbda092877818713f (DEFAULT_TABLE table=default.Table_924d66e4d68b4a79b062e6fd7a16878f [id=f2b16afb131f4ef5a1e62a27dcce0374]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:06.112815 10818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbf85a35b22a4c5cbda092877818713f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:06.115901 10819 tablet_service.cc:1514] Processing DeleteTablet for tablet d803e8490c564b3d8d5435374f93dc2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:05 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:06.119220 11098 ts_tablet_manager.cc:1918] T b3ca43e90141411bba9fe651845f8882 P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:06.127655 11098 log.cc:1196] T b3ca43e90141411bba9fe651845f8882 P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/b3ca43e90141411bba9fe651845f8882
I20260925 09:57:06.128881 11098 ts_tablet_manager.cc:1939] T b3ca43e90141411bba9fe651845f8882 P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
W20260925 09:57:06.133862 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet d803e8490c564b3d8d5435374f93dc2b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d803e8490c564b3d8d5435374f93dc2b already in progress: creating tablet
I20260925 09:57:06.133033 11090 raft_consensus.cc:504] T 2d162bcd507943e18f32dc593200eb4c P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:06.136868 10839 raft_consensus.cc:2379] T 2d162bcd507943e18f32dc593200eb4c P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b8850818e2ee42a082802d0095c08d21 in current term 1: Already voted for candidate c43461c4ee854ae1ac52b873ccada318 in this term.
I20260925 09:57:06.141758 11098 tablet_replica.cc:330] T 2b210d181fb0488187c193768f6b748e P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:06.073314 11085 tablet_bootstrap.cc:492] T 6a5121b78e2d4643b99bd8717c6e93ca P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
W20260925 09:57:06.143143 10387 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bd89e13e0c534822b3ce0391fff9e3c9 has no consensus state
I20260925 09:57:06.143750 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet b3ca43e90141411bba9fe651845f8882 (table default.Table_869036da470d4b9e9f5afaa74644914a [id=91e443968e4c4e52b477199bfe2a77a3]) successfully deleted
I20260925 09:57:06.144613 11085 ts_tablet_manager.cc:1397] T 6a5121b78e2d4643b99bd8717c6e93ca P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.118s user 0.007s sys 0.006s
I20260925 09:57:06.099946 11082 consensus_queue.cc:260] T b3ca43e90141411bba9fe651845f8882 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:06.148911 10393 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 2d162bcd507943e18f32dc593200eb4c
I20260925 09:57:06.142509 11098 raft_consensus.cc:2229] T 2b210d181fb0488187c193768f6b748e P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:06.149539 11098 raft_consensus.cc:2258] T 2b210d181fb0488187c193768f6b748e P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:06.155854 11098 ts_tablet_manager.cc:1905] T 2b210d181fb0488187c193768f6b748e P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:06.155908 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d162bcd507943e18f32dc593200eb4c" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:06.163607 10687 raft_consensus.cc:3037] T 2d162bcd507943e18f32dc593200eb4c P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:06.172144 10819 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d162bcd507943e18f32dc593200eb4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:05 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:06.174331 10687 raft_consensus.cc:2454] T 2d162bcd507943e18f32dc593200eb4c P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:06.175626 10775 leader_election.cc:302] T 2d162bcd507943e18f32dc593200eb4c P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:06.177182 11082 ts_tablet_manager.cc:1428] T b3ca43e90141411bba9fe651845f8882 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.083s user 0.005s sys 0.001s
I20260925 09:57:06.151906 11090 leader_election.cc:288] T 2d162bcd507943e18f32dc593200eb4c P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:06.181170 10388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_3cf34f59a5ba48d091a65aae5b4a4586"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:06.183173 10388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3cf34f59a5ba48d091a65aae5b4a4586 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:06.191215 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b210d181fb0488187c193768f6b748e" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:06.192183 11090 raft_consensus.cc:2781] T 2d162bcd507943e18f32dc593200eb4c P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:06.192435 10984 raft_consensus.cc:2454] T 2b210d181fb0488187c193768f6b748e P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
I20260925 09:57:06.193092 11090 raft_consensus.cc:686] T 2d162bcd507943e18f32dc593200eb4c P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Becoming Leader. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Running, Role: LEADER
I20260925 09:57:06.193920 10919 leader_election.cc:302] T 840faf042d8749bd958ce1de0207a734 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:06.194832 11089 raft_consensus.cc:2781] T 840faf042d8749bd958ce1de0207a734 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:06.192777 10666 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d162bcd507943e18f32dc593200eb4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:05 UTC) from {username='slave'} at 127.0.0.1:37876
W20260925 09:57:06.194752 10919 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 57012.6us
I20260925 09:57:06.196375 11095 tablet_replica.cc:330] T 2d162bcd507943e18f32dc593200eb4c P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:06.197456 11095 raft_consensus.cc:2229] T 2d162bcd507943e18f32dc593200eb4c P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:06.198060 11095 raft_consensus.cc:2258] T 2d162bcd507943e18f32dc593200eb4c P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:06.198927 11082 tablet_bootstrap.cc:492] T 2b210d181fb0488187c193768f6b748e P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:06.195196 11089 raft_consensus.cc:482] T 840faf042d8749bd958ce1de0207a734 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:06.199023 10392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_5905da5527364d3690f9e41ec25ce890"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:06.201031 10392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5905da5527364d3690f9e41ec25ce890 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:06.201471 11085 raft_consensus.cc:348] T 6a5121b78e2d4643b99bd8717c6e93ca P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:06.206434 11095 ts_tablet_manager.cc:1905] T 2d162bcd507943e18f32dc593200eb4c P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:06.211964 11082 tablet_bootstrap.cc:654] T 2b210d181fb0488187c193768f6b748e P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:06.212344 11085 raft_consensus.cc:374] T 6a5121b78e2d4643b99bd8717c6e93ca P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:06.212723 11085 raft_consensus.cc:729] T 6a5121b78e2d4643b99bd8717c6e93ca P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:06.213487 11085 consensus_queue.cc:260] T 6a5121b78e2d4643b99bd8717c6e93ca P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:06.217919 11085 ts_tablet_manager.cc:1428] T 6a5121b78e2d4643b99bd8717c6e93ca P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.071s user 0.002s sys 0.004s
I20260925 09:57:06.196151 11090 consensus_queue.cc:237] T 2d162bcd507943e18f32dc593200eb4c P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:06.221522 10390 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1969ac471b6f471ebe8e048a96c22837 has no consensus state
I20260925 09:57:06.225062 11085 tablet_bootstrap.cc:492] T a0976e0c01234b38a7ab934dc1b289c4 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:06.200162 11089 raft_consensus.cc:3037] T 840faf042d8749bd958ce1de0207a734 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:06.230597 11085 tablet_bootstrap.cc:654] T a0976e0c01234b38a7ab934dc1b289c4 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:06.238358 10982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d162bcd507943e18f32dc593200eb4c" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:06.239274 10982 raft_consensus.cc:3037] T 2d162bcd507943e18f32dc593200eb4c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:06.235744 10772 leader_election.cc:302] T 2b210d181fb0488187c193768f6b748e P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21
W20260925 09:57:06.244083 10772 raft_consensus.cc:2670] T 2b210d181fb0488187c193768f6b748e P c43461c4ee854ae1ac52b873ccada318: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260925 09:57:06.246114 10983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d803e8490c564b3d8d5435374f93dc2b" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:06.247140 10983 raft_consensus.cc:2454] T d803e8490c564b3d8d5435374f93dc2b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
I20260925 09:57:06.249799 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d162bcd507943e18f32dc593200eb4c" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:06.258090 10961 tablet_service.cc:1514] Processing DeleteTablet for tablet d803e8490c564b3d8d5435374f93dc2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:05 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:06.258965 10772 leader_election.cc:302] T d803e8490c564b3d8d5435374f93dc2b P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21
I20260925 09:57:06.259720 11098 ts_tablet_manager.cc:1918] T 2b210d181fb0488187c193768f6b748e P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:06.260116 11098 log.cc:1196] T 2b210d181fb0488187c193768f6b748e P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/2b210d181fb0488187c193768f6b748e
I20260925 09:57:06.260757 11098 ts_tablet_manager.cc:1939] T 2b210d181fb0488187c193768f6b748e P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:06.261197 11122 raft_consensus.cc:2781] T d803e8490c564b3d8d5435374f93dc2b P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:06.261528 11122 raft_consensus.cc:482] T d803e8490c564b3d8d5435374f93dc2b P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:06.261893 11122 raft_consensus.cc:3037] T d803e8490c564b3d8d5435374f93dc2b P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:06.264223 11082 tablet_bootstrap.cc:492] T 2b210d181fb0488187c193768f6b748e P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:06.264683 11082 ts_tablet_manager.cc:1397] T 2b210d181fb0488187c193768f6b748e P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.066s user 0.007s sys 0.005s
I20260925 09:57:06.267771 10960 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d162bcd507943e18f32dc593200eb4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:05 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:06.268658 11096 tablet_replica.cc:330] T d803e8490c564b3d8d5435374f93dc2b P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:06.269388 11096 raft_consensus.cc:2229] T d803e8490c564b3d8d5435374f93dc2b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:06.269851 11096 raft_consensus.cc:2258] T d803e8490c564b3d8d5435374f93dc2b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:06.274366 11089 raft_consensus.cc:504] T 840faf042d8749bd958ce1de0207a734 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:06.276096 11082 raft_consensus.cc:348] T 2b210d181fb0488187c193768f6b748e P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:06.276955 11082 raft_consensus.cc:374] T 2b210d181fb0488187c193768f6b748e P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:06.277277 11082 raft_consensus.cc:729] T 2b210d181fb0488187c193768f6b748e P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:06.277967 11082 consensus_queue.cc:260] T 2b210d181fb0488187c193768f6b748e P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:06.282074 11082 ts_tablet_manager.cc:1428] T 2b210d181fb0488187c193768f6b748e P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.017s user 0.002s sys 0.005s
I20260925 09:57:06.283617 11082 tablet_bootstrap.cc:492] T d803e8490c564b3d8d5435374f93dc2b P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:06.291628 11082 tablet_bootstrap.cc:654] T d803e8490c564b3d8d5435374f93dc2b P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:06.294979 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d162bcd507943e18f32dc593200eb4c" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:06.298535 11096 ts_tablet_manager.cc:1905] T d803e8490c564b3d8d5435374f93dc2b P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:06.298026 10390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_e53095ec47d14f3ebc0f2f8fcc583719"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:06.300067 10390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e53095ec47d14f3ebc0f2f8fcc583719 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:06.300769 11095 ts_tablet_manager.cc:1918] T 2d162bcd507943e18f32dc593200eb4c P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:06.301249 11095 log.cc:1196] T 2d162bcd507943e18f32dc593200eb4c P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/2d162bcd507943e18f32dc593200eb4c
I20260925 09:57:06.302170 11095 ts_tablet_manager.cc:1939] T 2d162bcd507943e18f32dc593200eb4c P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
W20260925 09:57:06.303977 10378 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 1969ac471b6f471ebe8e048a96c22837 (table default.Table_a122f131534b429ab5465d14f64c25a3 [id=f17a11d223204599afb24e2e9d1d9d9d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 1969ac471b6f471ebe8e048a96c22837 has no consensus state
I20260925 09:57:06.305092 10619 leader_election.cc:302] T 2d162bcd507943e18f32dc593200eb4c P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8850818e2ee42a082802d0095c08d21; no voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318
W20260925 09:57:06.305995 10619 raft_consensus.cc:2670] T 2d162bcd507943e18f32dc593200eb4c P b8850818e2ee42a082802d0095c08d21: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260925 09:57:06.310508 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "840faf042d8749bd958ce1de0207a734" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:06.311434 10687 raft_consensus.cc:3037] T 840faf042d8749bd958ce1de0207a734 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:06.311563 10387 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_ebff7801e9fa43ffafe5b8db4327e6f7" } modify_external_catalogs: true
I20260925 09:57:06.312350 10387 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_ebff7801e9fa43ffafe5b8db4327e6f7" } modify_external_catalogs: true
I20260925 09:57:06.316119 11089 leader_election.cc:288] T 840faf042d8749bd958ce1de0207a734 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:06.336282 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 2d162bcd507943e18f32dc593200eb4c (table default.Table_1298469ed8ca4d74b485794a4a7a7dca [id=984ffc82af7b4737bae52327ee2a1755]) successfully deleted
I20260925 09:57:06.337811 10982 raft_consensus.cc:2454] T 2d162bcd507943e18f32dc593200eb4c P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:06.338768 11109 raft_consensus.cc:482] T b3ca43e90141411bba9fe651845f8882 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:06.339329 11109 raft_consensus.cc:504] T b3ca43e90141411bba9fe651845f8882 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:06.342111 11109 leader_election.cc:288] T b3ca43e90141411bba9fe651845f8882 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:06.347050 10388 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_a122f131534b429ab5465d14f64c25a3" } new_table_name: "default.Table_a9dd56ac2283472da4a217ea2d04fa2f" modify_external_catalogs: true
I20260925 09:57:06.351825 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "840faf042d8749bd958ce1de0207a734" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:06.351969 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "840faf042d8749bd958ce1de0207a734" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
W20260925 09:57:06.352058 10621 leader_election.cc:341] T b3ca43e90141411bba9fe651845f8882 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Not found: Tablet not found: b3ca43e90141411bba9fe651845f8882
I20260925 09:57:06.352494 10839 raft_consensus.cc:2454] T 840faf042d8749bd958ce1de0207a734 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be97ab5d75934a648d4ab76341193ec0 in term 0.
I20260925 09:57:06.357208 11098 tablet_replica.cc:330] T d803e8490c564b3d8d5435374f93dc2b P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
W20260925 09:57:06.358027 10619 leader_election.cc:341] T b3ca43e90141411bba9fe651845f8882 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: b3ca43e90141411bba9fe651845f8882
I20260925 09:57:06.358726 10619 leader_election.cc:302] T b3ca43e90141411bba9fe651845f8882 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8850818e2ee42a082802d0095c08d21; no voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:06.360525 11109 raft_consensus.cc:2726] T b3ca43e90141411bba9fe651845f8882 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:06.362164 10392 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_3cf34f59a5ba48d091a65aae5b4a4586" } modify_external_catalogs: true
I20260925 09:57:06.362862 10392 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_3cf34f59a5ba48d091a65aae5b4a4586" } modify_external_catalogs: true
I20260925 09:57:06.364470 11122 raft_consensus.cc:504] T d803e8490c564b3d8d5435374f93dc2b P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
W20260925 09:57:06.364987 10379 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.60:36985, user_credentials={real_user=slave}} blocked reactor thread for 45442.6us
I20260925 09:57:06.365576 11098 raft_consensus.cc:2229] T d803e8490c564b3d8d5435374f93dc2b P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:06.367997 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 2b210d181fb0488187c193768f6b748e (table default.Table_82582ea3080a431c9e13b00349420e93 [id=7a12ae6dbd154c9eb1a25add3a9aee9c]) successfully deleted
I20260925 09:57:06.368428 11098 raft_consensus.cc:2258] T d803e8490c564b3d8d5435374f93dc2b P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:06.371500 11084 tablet_bootstrap.cc:492] T bbf85a35b22a4c5cbda092877818713f P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:06.379210 11084 tablet_bootstrap.cc:654] T bbf85a35b22a4c5cbda092877818713f P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:06.379479 11090 raft_consensus.cc:482] T 840faf042d8749bd958ce1de0207a734 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:06.387708 10393 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 840faf042d8749bd958ce1de0207a734 (table default.Table_faa21794a80c407f800038e583c08f16 [id=800eba46046443a7ad196808dec4cdc0]). Expected version 2 got 0
I20260925 09:57:06.387998 10963 tablet_service.cc:1467] Processing CreateTablet for tablet d8d6ac842c8040659c2589049eb21c5f (DEFAULT_TABLE table=default.Table_000f52fd1e214fa1931ef3743ce76250 [id=f67a73ffb5574153bc3b52c5b32ecabd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:06.389995 10960 tablet_service.cc:1467] Processing CreateTablet for tablet bd89e13e0c534822b3ce0391fff9e3c9 (DEFAULT_TABLE table=default.Table_aefaa3aacf0e43e1b32aa184267892d4 [id=e6ae2bf4301141f790fefe7bc560029a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:06.391120 10960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd89e13e0c534822b3ce0391fff9e3c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:06.397034 10668 tablet_service.cc:1467] Processing CreateTablet for tablet bd89e13e0c534822b3ce0391fff9e3c9 (DEFAULT_TABLE table=default.Table_aefaa3aacf0e43e1b32aa184267892d4 [id=e6ae2bf4301141f790fefe7bc560029a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:06.397553 10818 tablet_service.cc:1467] Processing CreateTablet for tablet d8d6ac842c8040659c2589049eb21c5f (DEFAULT_TABLE table=default.Table_000f52fd1e214fa1931ef3743ce76250 [id=f67a73ffb5574153bc3b52c5b32ecabd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:06.398262 10668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd89e13e0c534822b3ce0391fff9e3c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:06.399405 10818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8d6ac842c8040659c2589049eb21c5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:06.399350 10667 tablet_service.cc:1467] Processing CreateTablet for tablet 1969ac471b6f471ebe8e048a96c22837 (DEFAULT_TABLE table=default.Table_a122f131534b429ab5465d14f64c25a3 [id=f17a11d223204599afb24e2e9d1d9d9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:06.401386 10667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1969ac471b6f471ebe8e048a96c22837. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:06.397986 10666 tablet_service.cc:1467] Processing CreateTablet for tablet d8d6ac842c8040659c2589049eb21c5f (DEFAULT_TABLE table=default.Table_000f52fd1e214fa1931ef3743ce76250 [id=f67a73ffb5574153bc3b52c5b32ecabd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:06.403692 10665 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d162bcd507943e18f32dc593200eb4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:05 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:06.404146 10666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8d6ac842c8040659c2589049eb21c5f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:06.410665 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 2d162bcd507943e18f32dc593200eb4c because the tablet was not found. No further retry: Not found: Tablet not found: 2d162bcd507943e18f32dc593200eb4c
I20260925 09:57:06.411906 10961 tablet_service.cc:1467] Processing CreateTablet for tablet 1969ac471b6f471ebe8e048a96c22837 (DEFAULT_TABLE table=default.Table_a122f131534b429ab5465d14f64c25a3 [id=f17a11d223204599afb24e2e9d1d9d9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:06.413439 10961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1969ac471b6f471ebe8e048a96c22837. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:06.416009 10817 tablet_service.cc:1467] Processing CreateTablet for tablet bd89e13e0c534822b3ce0391fff9e3c9 (DEFAULT_TABLE table=default.Table_aefaa3aacf0e43e1b32aa184267892d4 [id=e6ae2bf4301141f790fefe7bc560029a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:06.417232 10817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd89e13e0c534822b3ce0391fff9e3c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:06.417757 10389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_56925b2babcc4f4790bafb815b0e1380"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:06.421030 10389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_56925b2babcc4f4790bafb815b0e1380 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:06.423686 10959 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b210d181fb0488187c193768f6b748e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:05 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:06.424289 10963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8d6ac842c8040659c2589049eb21c5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:06.428865 10819 tablet_service.cc:1467] Processing CreateTablet for tablet 1969ac471b6f471ebe8e048a96c22837 (DEFAULT_TABLE table=default.Table_a122f131534b429ab5465d14f64c25a3 [id=f17a11d223204599afb24e2e9d1d9d9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:06.430061 10819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1969ac471b6f471ebe8e048a96c22837. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:06.379938 11090 raft_consensus.cc:504] T 840faf042d8749bd958ce1de0207a734 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:06.444501 10982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "840faf042d8749bd958ce1de0207a734" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:06.444905 11085 tablet_bootstrap.cc:492] T a0976e0c01234b38a7ab934dc1b289c4 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:06.445299 10982 raft_consensus.cc:2379] T 840faf042d8749bd958ce1de0207a734 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c43461c4ee854ae1ac52b873ccada318 in current term 1: Already voted for candidate be97ab5d75934a648d4ab76341193ec0 in this term.
I20260925 09:57:06.445343 11085 ts_tablet_manager.cc:1397] T a0976e0c01234b38a7ab934dc1b289c4 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.220s user 0.010s sys 0.028s
I20260925 09:57:06.447930 10387 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 6936fd592dd94e48bf32b425292b64ce
I20260925 09:57:06.451342 11084 tablet_bootstrap.cc:492] T bbf85a35b22a4c5cbda092877818713f P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:06.451776 11084 ts_tablet_manager.cc:1397] T bbf85a35b22a4c5cbda092877818713f P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.080s user 0.010s sys 0.004s
I20260925 09:57:06.457795 10392 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 496ece82617d4d91993fff58b0ca62e3
I20260925 09:57:06.461607 11090 leader_election.cc:288] T 840faf042d8749bd958ce1de0207a734 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:06.462877 11122 leader_election.cc:288] T d803e8490c564b3d8d5435374f93dc2b P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:06.465920 11098 ts_tablet_manager.cc:1905] T d803e8490c564b3d8d5435374f93dc2b P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:06.343271 10687 raft_consensus.cc:2454] T 840faf042d8749bd958ce1de0207a734 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be97ab5d75934a648d4ab76341193ec0 in term 1.
I20260925 09:57:06.471247 10667 tablet_service.cc:1514] Processing DeleteTablet for tablet b3ca43e90141411bba9fe651845f8882 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:05 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:06.478832 11095 tablet_replica.cc:330] T b3ca43e90141411bba9fe651845f8882 P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:06.478778 11085 raft_consensus.cc:348] T a0976e0c01234b38a7ab934dc1b289c4 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:06.479916 11095 raft_consensus.cc:2229] T b3ca43e90141411bba9fe651845f8882 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:06.480085 11085 raft_consensus.cc:374] T a0976e0c01234b38a7ab934dc1b289c4 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:06.480540 11095 raft_consensus.cc:2258] T b3ca43e90141411bba9fe651845f8882 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:06.480343 11085 raft_consensus.cc:729] T a0976e0c01234b38a7ab934dc1b289c4 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:06.489331 11096 ts_tablet_manager.cc:1918] T d803e8490c564b3d8d5435374f93dc2b P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:06.489988 11096 log.cc:1196] T d803e8490c564b3d8d5435374f93dc2b P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/d803e8490c564b3d8d5435374f93dc2b
I20260925 09:57:06.490626 11096 ts_tablet_manager.cc:1939] T d803e8490c564b3d8d5435374f93dc2b P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:06.490321 10919 leader_election.cc:302] T 840faf042d8749bd958ce1de0207a734 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:06.491700 11089 raft_consensus.cc:2781] T 840faf042d8749bd958ce1de0207a734 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:06.484262 10387 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_5905da5527364d3690f9e41ec25ce890" } new_table_name: "default.Table_1e7379199917430eb9e027fc7bb2a4f9" modify_external_catalogs: true
I20260925 09:57:06.494791 11095 ts_tablet_manager.cc:1905] T b3ca43e90141411bba9fe651845f8882 P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:06.497262 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet d803e8490c564b3d8d5435374f93dc2b (table default.Table_a400e3e492de40e988e9cb29284a2003 [id=9230b1811a8a4faba58b239cf40992dd]) successfully deleted
I20260925 09:57:06.497571 11096 tablet_replica.cc:330] T 2d162bcd507943e18f32dc593200eb4c P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:06.505800 10667 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b210d181fb0488187c193768f6b748e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:05 UTC) from {username='slave'} at 127.0.0.1:37876
W20260925 09:57:06.515147 10388 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1969ac471b6f471ebe8e048a96c22837 consensus state has no leader
I20260925 09:57:06.516253 11084 raft_consensus.cc:348] T bbf85a35b22a4c5cbda092877818713f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:06.516949 11084 raft_consensus.cc:374] T bbf85a35b22a4c5cbda092877818713f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:06.517206 11084 raft_consensus.cc:729] T bbf85a35b22a4c5cbda092877818713f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:06.517845 11084 consensus_queue.cc:260] T bbf85a35b22a4c5cbda092877818713f P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
W20260925 09:57:06.526036 10772 leader_election.cc:341] T d803e8490c564b3d8d5435374f93dc2b P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: d803e8490c564b3d8d5435374f93dc2b
I20260925 09:57:06.531414 11082 tablet_bootstrap.cc:492] T d803e8490c564b3d8d5435374f93dc2b P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:06.531817 11082 ts_tablet_manager.cc:1397] T d803e8490c564b3d8d5435374f93dc2b P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.249s user 0.014s sys 0.006s
I20260925 09:57:06.534540 11082 raft_consensus.cc:348] T d803e8490c564b3d8d5435374f93dc2b P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:06.522176 11096 raft_consensus.cc:2229] T 2d162bcd507943e18f32dc593200eb4c P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:06.536938 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "840faf042d8749bd958ce1de0207a734" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:06.537783 10687 raft_consensus.cc:2379] T 840faf042d8749bd958ce1de0207a734 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c43461c4ee854ae1ac52b873ccada318 in current term 1: Already voted for candidate be97ab5d75934a648d4ab76341193ec0 in this term.
I20260925 09:57:06.538062 11084 ts_tablet_manager.cc:1428] T bbf85a35b22a4c5cbda092877818713f P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.086s user 0.007s sys 0.004s
I20260925 09:57:06.538244 10688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d803e8490c564b3d8d5435374f93dc2b" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:06.535308 11085 consensus_queue.cc:260] T a0976e0c01234b38a7ab934dc1b289c4 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:06.541294 11085 ts_tablet_manager.cc:1428] T a0976e0c01234b38a7ab934dc1b289c4 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.096s user 0.007s sys 0.000s
I20260925 09:57:06.546983 10390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_c4ae6258da494589b65be6ede95f4a1d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:06.549448 10390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c4ae6258da494589b65be6ede95f4a1d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:06.558280 11084 tablet_bootstrap.cc:492] T a0976e0c01234b38a7ab934dc1b289c4 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:06.559501 11096 raft_consensus.cc:2258] T 2d162bcd507943e18f32dc593200eb4c P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:06.566586 11084 tablet_bootstrap.cc:654] T a0976e0c01234b38a7ab934dc1b289c4 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:06.569056 11109 raft_consensus.cc:482] T 2b210d181fb0488187c193768f6b748e P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:06.569553 11109 raft_consensus.cc:504] T 2b210d181fb0488187c193768f6b748e P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:06.573043 11109 leader_election.cc:288] T 2b210d181fb0488187c193768f6b748e P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:06.576562 10982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b210d181fb0488187c193768f6b748e" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:06.577157 10982 raft_consensus.cc:2454] T 2b210d181fb0488187c193768f6b748e P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
W20260925 09:57:06.578250 10621 leader_election.cc:341] T 2b210d181fb0488187c193768f6b748e P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Not found: Tablet not found: 2b210d181fb0488187c193768f6b748e
I20260925 09:57:06.578460 11096 ts_tablet_manager.cc:1905] T 2d162bcd507943e18f32dc593200eb4c P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:06.579488 10619 leader_election.cc:302] T 2b210d181fb0488187c193768f6b748e P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:06.580560 11109 raft_consensus.cc:2781] T 2b210d181fb0488187c193768f6b748e P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:06.580890 11109 raft_consensus.cc:482] T 2b210d181fb0488187c193768f6b748e P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:06.581178 11109 raft_consensus.cc:3037] T 2b210d181fb0488187c193768f6b748e P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:06.586481 10388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_b13e058d7a264358a7b057cb23f04d18"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:06.573525 11089 raft_consensus.cc:686] T 840faf042d8749bd958ce1de0207a734 P be97ab5d75934a648d4ab76341193ec0 [term 1 LEADER]: Becoming Leader. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Running, Role: LEADER
I20260925 09:57:06.588397 11095 ts_tablet_manager.cc:1918] T b3ca43e90141411bba9fe651845f8882 P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260925 09:57:06.588511 10388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b13e058d7a264358a7b057cb23f04d18 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:06.588745 11095 log.cc:1196] T b3ca43e90141411bba9fe651845f8882 P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/b3ca43e90141411bba9fe651845f8882
I20260925 09:57:06.588294 11089 consensus_queue.cc:237] T 840faf042d8749bd958ce1de0207a734 P be97ab5d75934a648d4ab76341193ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:06.589377 11095 ts_tablet_manager.cc:1939] T b3ca43e90141411bba9fe651845f8882 P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:06.592630 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet b3ca43e90141411bba9fe651845f8882 (table default.Table_869036da470d4b9e9f5afaa74644914a [id=91e443968e4c4e52b477199bfe2a77a3]) successfully deleted
I20260925 09:57:06.595882 11109 raft_consensus.cc:504] T 2b210d181fb0488187c193768f6b748e P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:06.599303 10982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b210d181fb0488187c193768f6b748e" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:06.600358 10982 raft_consensus.cc:3037] T 2b210d181fb0488187c193768f6b748e P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:06.552139 11122 raft_consensus.cc:482] T 6a5121b78e2d4643b99bd8717c6e93ca P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:06.601297 11122 raft_consensus.cc:504] T 6a5121b78e2d4643b99bd8717c6e93ca P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:06.535209 11082 raft_consensus.cc:374] T d803e8490c564b3d8d5435374f93dc2b P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:06.602742 11082 raft_consensus.cc:729] T d803e8490c564b3d8d5435374f93dc2b P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:06.606966 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a5121b78e2d4643b99bd8717c6e93ca" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:06.607913 10984 raft_consensus.cc:2454] T 6a5121b78e2d4643b99bd8717c6e93ca P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
I20260925 09:57:06.602427 11085 tablet_bootstrap.cc:492] T bbf85a35b22a4c5cbda092877818713f P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:06.605583 11122 leader_election.cc:288] T 6a5121b78e2d4643b99bd8717c6e93ca P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:06.614028 11082 consensus_queue.cc:260] T d803e8490c564b3d8d5435374f93dc2b P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:06.620024 10772 leader_election.cc:302] T 6a5121b78e2d4643b99bd8717c6e93ca P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:06.617089 11095 tablet_replica.cc:330] T 2b210d181fb0488187c193768f6b748e P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:06.621829 11122 raft_consensus.cc:2781] T 6a5121b78e2d4643b99bd8717c6e93ca P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:06.621896 11095 raft_consensus.cc:2229] T 2b210d181fb0488187c193768f6b748e P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:06.622180 11122 raft_consensus.cc:482] T 6a5121b78e2d4643b99bd8717c6e93ca P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:06.622499 11122 raft_consensus.cc:3037] T 6a5121b78e2d4643b99bd8717c6e93ca P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:06.622499 11095 raft_consensus.cc:2258] T 2b210d181fb0488187c193768f6b748e P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:06.617452 11109 leader_election.cc:288] T 2b210d181fb0488187c193768f6b748e P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
W20260925 09:57:06.628083 10621 leader_election.cc:341] T 2b210d181fb0488187c193768f6b748e P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Not found: Tablet not found: 2b210d181fb0488187c193768f6b748e
I20260925 09:57:06.629074 11095 ts_tablet_manager.cc:1905] T 2b210d181fb0488187c193768f6b748e P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:06.637697 11085 tablet_bootstrap.cc:654] T bbf85a35b22a4c5cbda092877818713f P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:06.555984 10775 leader_election.cc:302] T 840faf042d8749bd958ce1de0207a734 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:06.640163 11128 raft_consensus.cc:3037] T 840faf042d8749bd958ce1de0207a734 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:57:06.640057 10775 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 84170.9us
I20260925 09:57:06.644104 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a5121b78e2d4643b99bd8717c6e93ca" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:06.646497 10387 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4be3f6e81ddf418fabc7b694a2e62f35 has no consensus state
W20260925 09:57:06.647953 10775 leader_election.cc:341] T 6a5121b78e2d4643b99bd8717c6e93ca P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:06.650249 11128 raft_consensus.cc:2726] T 840faf042d8749bd958ce1de0207a734 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:06.652105 11084 tablet_bootstrap.cc:492] T a0976e0c01234b38a7ab934dc1b289c4 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:06.652504 11084 ts_tablet_manager.cc:1397] T a0976e0c01234b38a7ab934dc1b289c4 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.094s user 0.010s sys 0.001s
I20260925 09:57:06.652711 10688 raft_consensus.cc:3037] T d803e8490c564b3d8d5435374f93dc2b P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:06.652418 11122 raft_consensus.cc:504] T 6a5121b78e2d4643b99bd8717c6e93ca P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:06.656028 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a5121b78e2d4643b99bd8717c6e93ca" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:06.656018 11084 raft_consensus.cc:348] T a0976e0c01234b38a7ab934dc1b289c4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:06.656663 11084 raft_consensus.cc:374] T a0976e0c01234b38a7ab934dc1b289c4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:06.656891 11084 raft_consensus.cc:729] T a0976e0c01234b38a7ab934dc1b289c4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
W20260925 09:57:06.657305 10775 leader_election.cc:341] T 6a5121b78e2d4643b99bd8717c6e93ca P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:06.657450 11084 consensus_queue.cc:260] T a0976e0c01234b38a7ab934dc1b289c4 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:06.658862 11122 leader_election.cc:288] T 6a5121b78e2d4643b99bd8717c6e93ca P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:06.660528 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a5121b78e2d4643b99bd8717c6e93ca" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:06.661293 10984 raft_consensus.cc:3037] T 6a5121b78e2d4643b99bd8717c6e93ca P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:06.665418 10688 raft_consensus.cc:2454] T d803e8490c564b3d8d5435374f93dc2b P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:06.659303 11084 ts_tablet_manager.cc:1428] T a0976e0c01234b38a7ab934dc1b289c4 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.007s user 0.002s sys 0.001s
I20260925 09:57:06.666854 11084 tablet_bootstrap.cc:492] T 1969ac471b6f471ebe8e048a96c22837 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:06.677002 11082 ts_tablet_manager.cc:1428] T d803e8490c564b3d8d5435374f93dc2b P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.145s user 0.007s sys 0.000s
I20260925 09:57:06.677371 11084 tablet_bootstrap.cc:654] T 1969ac471b6f471ebe8e048a96c22837 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:06.678680 10387 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_924d66e4d68b4a79b062e6fd7a16878f" } modify_external_catalogs: true
I20260925 09:57:06.679679 10982 raft_consensus.cc:2454] T 2b210d181fb0488187c193768f6b748e P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:06.680850 10619 leader_election.cc:302] T 2b210d181fb0488187c193768f6b748e P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
W20260925 09:57:06.681835 10619 raft_consensus.cc:2670] T 2b210d181fb0488187c193768f6b748e P b8850818e2ee42a082802d0095c08d21: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260925 09:57:06.682646 10984 raft_consensus.cc:2454] T 6a5121b78e2d4643b99bd8717c6e93ca P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:06.684089 10389 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_56925b2babcc4f4790bafb815b0e1380" } modify_external_catalogs: true
I20260925 09:57:06.684684 10389 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_56925b2babcc4f4790bafb815b0e1380" } modify_external_catalogs: true
I20260925 09:57:06.679286 10387 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_924d66e4d68b4a79b062e6fd7a16878f" } modify_external_catalogs: true
I20260925 09:57:06.699618 10667 tablet_service.cc:1514] Processing DeleteTablet for tablet d803e8490c564b3d8d5435374f93dc2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:05 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:06.706290 10772 leader_election.cc:302] T 6a5121b78e2d4643b99bd8717c6e93ca P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21
I20260925 09:57:06.707297 11122 raft_consensus.cc:2781] T 6a5121b78e2d4643b99bd8717c6e93ca P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:06.707625 11095 ts_tablet_manager.cc:1918] T 2b210d181fb0488187c193768f6b748e P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:06.707712 11122 raft_consensus.cc:686] T 6a5121b78e2d4643b99bd8717c6e93ca P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Becoming Leader. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Running, Role: LEADER
I20260925 09:57:06.708009 11095 log.cc:1196] T 2b210d181fb0488187c193768f6b748e P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/2b210d181fb0488187c193768f6b748e
I20260925 09:57:06.708634 11095 ts_tablet_manager.cc:1939] T 2b210d181fb0488187c193768f6b748e P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:06.709527 11082 tablet_bootstrap.cc:492] T 6a5121b78e2d4643b99bd8717c6e93ca P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:06.708421 11122 consensus_queue.cc:237] T 6a5121b78e2d4643b99bd8717c6e93ca P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:06.713526 10775 leader_election.cc:302] T d803e8490c564b3d8d5435374f93dc2b P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters: be97ab5d75934a648d4ab76341193ec0
W20260925 09:57:06.714167 10775 raft_consensus.cc:2670] T d803e8490c564b3d8d5435374f93dc2b P c43461c4ee854ae1ac52b873ccada318: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260925 09:57:06.717984 11084 tablet_bootstrap.cc:492] T 1969ac471b6f471ebe8e048a96c22837 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:06.718393 11084 ts_tablet_manager.cc:1397] T 1969ac471b6f471ebe8e048a96c22837 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.052s user 0.007s sys 0.003s
I20260925 09:57:06.720956 11084 raft_consensus.cc:348] T 1969ac471b6f471ebe8e048a96c22837 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:06.721616 11084 raft_consensus.cc:374] T 1969ac471b6f471ebe8e048a96c22837 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:06.721900 11084 raft_consensus.cc:729] T 1969ac471b6f471ebe8e048a96c22837 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:06.724419 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 2b210d181fb0488187c193768f6b748e (table default.Table_82582ea3080a431c9e13b00349420e93 [id=7a12ae6dbd154c9eb1a25add3a9aee9c]) successfully deleted
I20260925 09:57:06.726037 11096 ts_tablet_manager.cc:1918] T 2d162bcd507943e18f32dc593200eb4c P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:06.726033 11095 tablet_replica.cc:330] T d803e8490c564b3d8d5435374f93dc2b P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:06.726418 11096 log.cc:1196] T 2d162bcd507943e18f32dc593200eb4c P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/2d162bcd507943e18f32dc593200eb4c
I20260925 09:57:06.727003 11096 ts_tablet_manager.cc:1939] T 2d162bcd507943e18f32dc593200eb4c P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:06.713569 11098 ts_tablet_manager.cc:1918] T d803e8490c564b3d8d5435374f93dc2b P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:06.727512 11098 log.cc:1196] T d803e8490c564b3d8d5435374f93dc2b P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/d803e8490c564b3d8d5435374f93dc2b
I20260925 09:57:06.728175 11098 ts_tablet_manager.cc:1939] T d803e8490c564b3d8d5435374f93dc2b P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:06.731078 11095 raft_consensus.cc:2229] T d803e8490c564b3d8d5435374f93dc2b P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:06.731549 11095 raft_consensus.cc:2258] T d803e8490c564b3d8d5435374f93dc2b P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:06.733976 11095 ts_tablet_manager.cc:1905] T d803e8490c564b3d8d5435374f93dc2b P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:06.737493 10393 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_cae21509a44f4fd9aac22d5fea74694b" } new_table_name: "default.Table_5ff0f5bde5954217976f2876e76adaaa" modify_external_catalogs: true
I20260925 09:57:06.738847 11084 consensus_queue.cc:260] T 1969ac471b6f471ebe8e048a96c22837 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:06.740236 11089 raft_consensus.cc:482] T 1969ac471b6f471ebe8e048a96c22837 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:06.740221 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 2d162bcd507943e18f32dc593200eb4c (table default.Table_1298469ed8ca4d74b485794a4a7a7dca [id=984ffc82af7b4737bae52327ee2a1755]) successfully deleted
I20260925 09:57:06.741166 11096 tablet_replica.cc:330] T 2b210d181fb0488187c193768f6b748e P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:06.740629 11089 raft_consensus.cc:504] T 1969ac471b6f471ebe8e048a96c22837 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:06.740639 10390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_b9a40e813ba14ce89495f9edffeb4ee0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:06.742019 11096 raft_consensus.cc:2229] T 2b210d181fb0488187c193768f6b748e P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:06.742455 11085 tablet_bootstrap.cc:492] T bbf85a35b22a4c5cbda092877818713f P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:06.742458 11096 raft_consensus.cc:2258] T 2b210d181fb0488187c193768f6b748e P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:06.742933 11085 ts_tablet_manager.cc:1397] T bbf85a35b22a4c5cbda092877818713f P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.141s user 0.005s sys 0.005s
W20260925 09:57:06.742900 10390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b9a40e813ba14ce89495f9edffeb4ee0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:06.746587 11089 leader_election.cc:288] T 1969ac471b6f471ebe8e048a96c22837 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:06.747211 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1969ac471b6f471ebe8e048a96c22837" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:06.750001 10918 leader_election.cc:341] T 1969ac471b6f471ebe8e048a96c22837 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:06.752386 11096 ts_tablet_manager.cc:1905] T 2b210d181fb0488187c193768f6b748e P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:06.753391 11084 ts_tablet_manager.cc:1428] T 1969ac471b6f471ebe8e048a96c22837 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.035s user 0.004s sys 0.004s
I20260925 09:57:06.758141 11084 tablet_bootstrap.cc:492] T d8d6ac842c8040659c2589049eb21c5f P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:06.759519 10389 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 2939060d6e334bf3861ec6ac10188b30
I20260925 09:57:06.761905 10688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1969ac471b6f471ebe8e048a96c22837" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:06.764678 11085 raft_consensus.cc:348] T bbf85a35b22a4c5cbda092877818713f P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
W20260925 09:57:06.766417 10919 leader_election.cc:341] T 1969ac471b6f471ebe8e048a96c22837 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:06.766947 10919 leader_election.cc:302] T 1969ac471b6f471ebe8e048a96c22837 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:06.767673 11084 tablet_bootstrap.cc:654] T d8d6ac842c8040659c2589049eb21c5f P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:06.768247 11089 raft_consensus.cc:2726] T 1969ac471b6f471ebe8e048a96c22837 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:06.770313 11085 raft_consensus.cc:374] T bbf85a35b22a4c5cbda092877818713f P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:06.780102 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet d803e8490c564b3d8d5435374f93dc2b (table default.Table_a400e3e492de40e988e9cb29284a2003 [id=9230b1811a8a4faba58b239cf40992dd]) successfully deleted
I20260925 09:57:06.788412 10960 tablet_service.cc:1467] Processing CreateTablet for tablet 4be3f6e81ddf418fabc7b694a2e62f35 (DEFAULT_TABLE table=default.Table_1e7379199917430eb9e027fc7bb2a4f9 [id=c2d0336fd84c4dbe95472f458b813fe6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:06.789861 10960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4be3f6e81ddf418fabc7b694a2e62f35. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:06.790022 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet d803e8490c564b3d8d5435374f93dc2b because the tablet was not found. No further retry: Not found: Tablet not found: d803e8490c564b3d8d5435374f93dc2b
I20260925 09:57:06.792532 10668 tablet_service.cc:1467] Processing CreateTablet for tablet 8aae639695f44d8c8a61b3ea42ab6ac8 (DEFAULT_TABLE table=default.Table_f2634f079dff40ac91f6c47ba66482e6 [id=e493befbf9d94366aa09d518014389eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:06.778520 11098 tablet_replica.cc:330] T 2d162bcd507943e18f32dc593200eb4c P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:06.796710 10667 tablet_service.cc:1467] Processing CreateTablet for tablet 4be3f6e81ddf418fabc7b694a2e62f35 (DEFAULT_TABLE table=default.Table_1e7379199917430eb9e027fc7bb2a4f9 [id=c2d0336fd84c4dbe95472f458b813fe6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:06.797012 11098 raft_consensus.cc:2229] T 2d162bcd507943e18f32dc593200eb4c P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:06.797924 11098 pending_rounds.cc:62] T 2d162bcd507943e18f32dc593200eb4c P c43461c4ee854ae1ac52b873ccada318: Trying to abort 1 pending ops.
I20260925 09:57:06.798166 11098 pending_rounds.cc:68] T 2d162bcd507943e18f32dc593200eb4c P c43461c4ee854ae1ac52b873ccada318: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192606597566464 op_type: NO_OP noop_request { }
I20260925 09:57:06.798730 11098 raft_consensus.cc:2866] T 2d162bcd507943e18f32dc593200eb4c P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:57:06.799113 11098 raft_consensus.cc:2258] T 2d162bcd507943e18f32dc593200eb4c P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:06.801491 10961 tablet_service.cc:1467] Processing CreateTablet for tablet 8aae639695f44d8c8a61b3ea42ab6ac8 (DEFAULT_TABLE table=default.Table_f2634f079dff40ac91f6c47ba66482e6 [id=e493befbf9d94366aa09d518014389eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:06.801930 11098 ts_tablet_manager.cc:1905] T 2d162bcd507943e18f32dc593200eb4c P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:06.802834 10961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8aae639695f44d8c8a61b3ea42ab6ac8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:06.794056 10819 tablet_service.cc:1467] Processing CreateTablet for tablet 4be3f6e81ddf418fabc7b694a2e62f35 (DEFAULT_TABLE table=default.Table_1e7379199917430eb9e027fc7bb2a4f9 [id=c2d0336fd84c4dbe95472f458b813fe6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:06.808861 10819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4be3f6e81ddf418fabc7b694a2e62f35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:06.809059 11084 tablet_bootstrap.cc:492] T d8d6ac842c8040659c2589049eb21c5f P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:06.809419 11084 ts_tablet_manager.cc:1397] T d8d6ac842c8040659c2589049eb21c5f P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.051s user 0.009s sys 0.001s
I20260925 09:57:06.810165 11082 tablet_bootstrap.cc:654] T 6a5121b78e2d4643b99bd8717c6e93ca P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:06.811470 11085 raft_consensus.cc:729] T bbf85a35b22a4c5cbda092877818713f P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:06.795125 10818 tablet_service.cc:1467] Processing CreateTablet for tablet 8aae639695f44d8c8a61b3ea42ab6ac8 (DEFAULT_TABLE table=default.Table_f2634f079dff40ac91f6c47ba66482e6 [id=e493befbf9d94366aa09d518014389eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:06.812448 10818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8aae639695f44d8c8a61b3ea42ab6ac8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:06.812925 11084 raft_consensus.cc:348] T d8d6ac842c8040659c2589049eb21c5f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:06.813585 11084 raft_consensus.cc:374] T d8d6ac842c8040659c2589049eb21c5f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:06.813827 11084 raft_consensus.cc:729] T d8d6ac842c8040659c2589049eb21c5f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:06.814674 11096 ts_tablet_manager.cc:1918] T 2b210d181fb0488187c193768f6b748e P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:06.814829 11085 consensus_queue.cc:260] T bbf85a35b22a4c5cbda092877818713f P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:06.815848 10668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8aae639695f44d8c8a61b3ea42ab6ac8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:06.816779 11085 ts_tablet_manager.cc:1428] T bbf85a35b22a4c5cbda092877818713f P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.074s user 0.003s sys 0.002s
I20260925 09:57:06.820676 11085 tablet_bootstrap.cc:492] T bd89e13e0c534822b3ce0391fff9e3c9 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:06.823807 11084 consensus_queue.cc:260] T d8d6ac842c8040659c2589049eb21c5f P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:06.826464 11085 tablet_bootstrap.cc:654] T bd89e13e0c534822b3ce0391fff9e3c9 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:06.802549 10389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_bc69fdbbaf5b4a4e8cdfe195eefb16ab"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:06.832993 10389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bc69fdbbaf5b4a4e8cdfe195eefb16ab in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:06.834591 11084 ts_tablet_manager.cc:1428] T d8d6ac842c8040659c2589049eb21c5f P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.025s user 0.002s sys 0.005s
I20260925 09:57:06.846943 11084 tablet_bootstrap.cc:492] T bd89e13e0c534822b3ce0391fff9e3c9 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:06.850939 10388 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_e53095ec47d14f3ebc0f2f8fcc583719" } new_table_name: "default.Table_534d354d5b4f4f90b8f80a76ddbc8214" modify_external_catalogs: true
I20260925 09:57:06.815127 11096 log.cc:1196] T 2b210d181fb0488187c193768f6b748e P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/2b210d181fb0488187c193768f6b748e
I20260925 09:57:06.853609 11096 ts_tablet_manager.cc:1939] T 2b210d181fb0488187c193768f6b748e P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:06.868523 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 2b210d181fb0488187c193768f6b748e (table default.Table_82582ea3080a431c9e13b00349420e93 [id=7a12ae6dbd154c9eb1a25add3a9aee9c]) successfully deleted
I20260925 09:57:06.873790 11082 tablet_bootstrap.cc:492] T 6a5121b78e2d4643b99bd8717c6e93ca P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:06.874691 11082 ts_tablet_manager.cc:1397] T 6a5121b78e2d4643b99bd8717c6e93ca P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.165s user 0.005s sys 0.016s
I20260925 09:57:06.876262 11084 tablet_bootstrap.cc:654] T bd89e13e0c534822b3ce0391fff9e3c9 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:06.881426 10667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4be3f6e81ddf418fabc7b694a2e62f35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:06.880339 11082 raft_consensus.cc:348] T 6a5121b78e2d4643b99bd8717c6e93ca P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:06.883605 11082 raft_consensus.cc:374] T 6a5121b78e2d4643b99bd8717c6e93ca P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:06.883882 11082 raft_consensus.cc:729] T 6a5121b78e2d4643b99bd8717c6e93ca P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:06.885965 11085 tablet_bootstrap.cc:492] T bd89e13e0c534822b3ce0391fff9e3c9 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:06.886360 11085 ts_tablet_manager.cc:1397] T bd89e13e0c534822b3ce0391fff9e3c9 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.066s user 0.000s sys 0.014s
I20260925 09:57:06.888339 11085 raft_consensus.cc:348] T bd89e13e0c534822b3ce0391fff9e3c9 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:06.888928 11085 raft_consensus.cc:374] T bd89e13e0c534822b3ce0391fff9e3c9 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:06.889142 11085 raft_consensus.cc:729] T bd89e13e0c534822b3ce0391fff9e3c9 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:06.889812 11082 consensus_queue.cc:260] T 6a5121b78e2d4643b99bd8717c6e93ca P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:06.894583 11098 ts_tablet_manager.cc:1918] T 2d162bcd507943e18f32dc593200eb4c P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:06.894980 11098 log.cc:1196] T 2d162bcd507943e18f32dc593200eb4c P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/2d162bcd507943e18f32dc593200eb4c
I20260925 09:57:06.895845 11098 ts_tablet_manager.cc:1939] T 2d162bcd507943e18f32dc593200eb4c P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:06.898872 10385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_a8fb0e19083b495c978e0318d857492f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:06.900698 11082 ts_tablet_manager.cc:1428] T 6a5121b78e2d4643b99bd8717c6e93ca P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.026s user 0.001s sys 0.004s
W20260925 09:57:06.900604 10385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a8fb0e19083b495c978e0318d857492f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:06.903755 11085 consensus_queue.cc:260] T bd89e13e0c534822b3ce0391fff9e3c9 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:06.905151 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 2d162bcd507943e18f32dc593200eb4c (table default.Table_1298469ed8ca4d74b485794a4a7a7dca [id=984ffc82af7b4737bae52327ee2a1755]) successfully deleted
I20260925 09:57:06.908455 11084 tablet_bootstrap.cc:492] T bd89e13e0c534822b3ce0391fff9e3c9 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:06.908888 11084 ts_tablet_manager.cc:1397] T bd89e13e0c534822b3ce0391fff9e3c9 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.062s user 0.008s sys 0.003s
I20260925 09:57:06.909929 10384 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_3e77b19accd1463cb978e456128a53ef"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:06.911934 10384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3e77b19accd1463cb978e456128a53ef in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:06.915611 11082 tablet_bootstrap.cc:492] T bbf85a35b22a4c5cbda092877818713f P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:06.917189 11084 raft_consensus.cc:348] T bd89e13e0c534822b3ce0391fff9e3c9 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:06.917861 11084 raft_consensus.cc:374] T bd89e13e0c534822b3ce0391fff9e3c9 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:06.918143 11084 raft_consensus.cc:729] T bd89e13e0c534822b3ce0391fff9e3c9 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:06.918830 11084 consensus_queue.cc:260] T bd89e13e0c534822b3ce0391fff9e3c9 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:06.921056 11085 ts_tablet_manager.cc:1428] T bd89e13e0c534822b3ce0391fff9e3c9 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.034s user 0.005s sys 0.002s
I20260925 09:57:06.924685 11085 tablet_bootstrap.cc:492] T d8d6ac842c8040659c2589049eb21c5f P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:06.928022 11084 ts_tablet_manager.cc:1428] T bd89e13e0c534822b3ce0391fff9e3c9 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.019s user 0.005s sys 0.000s
I20260925 09:57:06.929450 11085 tablet_bootstrap.cc:654] T d8d6ac842c8040659c2589049eb21c5f P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:06.932814 11084 tablet_bootstrap.cc:492] T 4be3f6e81ddf418fabc7b694a2e62f35 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:06.944397 11084 tablet_bootstrap.cc:654] T 4be3f6e81ddf418fabc7b694a2e62f35 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:06.953008 11082 tablet_bootstrap.cc:654] T bbf85a35b22a4c5cbda092877818713f P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:06.965813 10392 catalog_manager.cc:5495] T 840faf042d8749bd958ce1de0207a734 P be97ab5d75934a648d4ab76341193ec0 reported cstate change: term changed from 0 to 1, leader changed from <none> to be97ab5d75934a648d4ab76341193ec0 (127.10.9.3). New cstate: current_term: 1 leader_uuid: "be97ab5d75934a648d4ab76341193ec0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: HEALTHY } } }
I20260925 09:57:06.968454 10392 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 840faf042d8749bd958ce1de0207a734 (table default.Table_faa21794a80c407f800038e583c08f16 [id=800eba46046443a7ad196808dec4cdc0]). Expected version 2 got 0
I20260925 09:57:06.970312 10387 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet bbf85a35b22a4c5cbda092877818713f
I20260925 09:57:06.973593 10961 tablet_service.cc:1514] Processing DeleteTablet for tablet bbf85a35b22a4c5cbda092877818713f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:06 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:06.974383 11096 tablet_replica.cc:330] T bbf85a35b22a4c5cbda092877818713f P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:06.974325 10817 tablet_service.cc:1514] Processing DeleteTablet for tablet bbf85a35b22a4c5cbda092877818713f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:06 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:06.975046 11098 tablet_replica.cc:330] T bbf85a35b22a4c5cbda092877818713f P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:06.975286 11096 raft_consensus.cc:2229] T bbf85a35b22a4c5cbda092877818713f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:06.975759 11096 raft_consensus.cc:2258] T bbf85a35b22a4c5cbda092877818713f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:06.975818 11098 raft_consensus.cc:2229] T bbf85a35b22a4c5cbda092877818713f P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:06.975903 10666 tablet_service.cc:1514] Processing DeleteTablet for tablet bbf85a35b22a4c5cbda092877818713f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:06 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:06.976246 11098 raft_consensus.cc:2258] T bbf85a35b22a4c5cbda092877818713f P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:06.978893 11096 ts_tablet_manager.cc:1905] T bbf85a35b22a4c5cbda092877818713f P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:06.981249 11098 ts_tablet_manager.cc:1905] T bbf85a35b22a4c5cbda092877818713f P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:06.983459 11084 tablet_bootstrap.cc:492] T 4be3f6e81ddf418fabc7b694a2e62f35 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:06.983836 11084 ts_tablet_manager.cc:1397] T 4be3f6e81ddf418fabc7b694a2e62f35 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.051s user 0.008s sys 0.003s
I20260925 09:57:06.986182 11084 raft_consensus.cc:348] T 4be3f6e81ddf418fabc7b694a2e62f35 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:06.986871 11084 raft_consensus.cc:374] T 4be3f6e81ddf418fabc7b694a2e62f35 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:06.987106 11084 raft_consensus.cc:729] T 4be3f6e81ddf418fabc7b694a2e62f35 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:06.987807 11084 consensus_queue.cc:260] T 4be3f6e81ddf418fabc7b694a2e62f35 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:06.992596 11084 ts_tablet_manager.cc:1428] T 4be3f6e81ddf418fabc7b694a2e62f35 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.009s user 0.002s sys 0.004s
I20260925 09:57:07.002475 10390 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_aefaa3aacf0e43e1b32aa184267892d4" } modify_external_catalogs: true
I20260925 09:57:07.003680 11084 tablet_bootstrap.cc:492] T 8aae639695f44d8c8a61b3ea42ab6ac8 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:07.004315 10390 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_aefaa3aacf0e43e1b32aa184267892d4" } modify_external_catalogs: true
I20260925 09:57:07.009382 11084 tablet_bootstrap.cc:654] T 8aae639695f44d8c8a61b3ea42ab6ac8 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:07.011842 10388 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6b03e5aad59e46a6b0e43b9cb36278d7 has no consensus state
I20260925 09:57:07.021639 11085 tablet_bootstrap.cc:492] T d8d6ac842c8040659c2589049eb21c5f P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:07.022068 11085 ts_tablet_manager.cc:1397] T d8d6ac842c8040659c2589049eb21c5f P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.098s user 0.008s sys 0.015s
I20260925 09:57:07.024217 11085 raft_consensus.cc:348] T d8d6ac842c8040659c2589049eb21c5f P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:07.024843 11085 raft_consensus.cc:374] T d8d6ac842c8040659c2589049eb21c5f P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:07.025094 11085 raft_consensus.cc:729] T d8d6ac842c8040659c2589049eb21c5f P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:07.033545 11082 tablet_bootstrap.cc:492] T bbf85a35b22a4c5cbda092877818713f P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:07.034063 11082 ts_tablet_manager.cc:1397] T bbf85a35b22a4c5cbda092877818713f P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.133s user 0.003s sys 0.023s
I20260925 09:57:07.036989 11085 consensus_queue.cc:260] T d8d6ac842c8040659c2589049eb21c5f P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:07.038059 11082 raft_consensus.cc:348] T bbf85a35b22a4c5cbda092877818713f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:07.038727 11082 raft_consensus.cc:374] T bbf85a35b22a4c5cbda092877818713f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:07.038982 11082 raft_consensus.cc:729] T bbf85a35b22a4c5cbda092877818713f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:07.042574 11084 tablet_bootstrap.cc:492] T 8aae639695f44d8c8a61b3ea42ab6ac8 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:07.043215 11084 ts_tablet_manager.cc:1397] T 8aae639695f44d8c8a61b3ea42ab6ac8 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.040s user 0.010s sys 0.000s
I20260925 09:57:07.045231 11084 raft_consensus.cc:348] T 8aae639695f44d8c8a61b3ea42ab6ac8 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:07.045858 11084 raft_consensus.cc:374] T 8aae639695f44d8c8a61b3ea42ab6ac8 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:07.046082 11084 raft_consensus.cc:729] T 8aae639695f44d8c8a61b3ea42ab6ac8 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:07.048486 11085 ts_tablet_manager.cc:1428] T d8d6ac842c8040659c2589049eb21c5f P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.026s user 0.006s sys 0.001s
I20260925 09:57:07.047873 11082 consensus_queue.cc:260] T bbf85a35b22a4c5cbda092877818713f P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:07.046823 11084 consensus_queue.cc:260] T 8aae639695f44d8c8a61b3ea42ab6ac8 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:07.056728 11085 tablet_bootstrap.cc:492] T 1969ac471b6f471ebe8e048a96c22837 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:07.057325 11089 raft_consensus.cc:482] T bd89e13e0c534822b3ce0391fff9e3c9 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:07.057703 11089 raft_consensus.cc:504] T bd89e13e0c534822b3ce0391fff9e3c9 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:07.059484 11089 leader_election.cc:288] T bd89e13e0c534822b3ce0391fff9e3c9 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:07.061864 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd89e13e0c534822b3ce0391fff9e3c9" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:07.062513 10839 raft_consensus.cc:2454] T bd89e13e0c534822b3ce0391fff9e3c9 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be97ab5d75934a648d4ab76341193ec0 in term 0.
I20260925 09:57:07.062493 10688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd89e13e0c534822b3ce0391fff9e3c9" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:07.063791 10918 leader_election.cc:302] T bd89e13e0c534822b3ce0391fff9e3c9 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters:
W20260925 09:57:07.064330 10919 leader_election.cc:341] T bd89e13e0c534822b3ce0391fff9e3c9 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:07.064517 11084 ts_tablet_manager.cc:1428] T 8aae639695f44d8c8a61b3ea42ab6ac8 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.021s user 0.003s sys 0.001s
I20260925 09:57:07.064627 11089 raft_consensus.cc:2781] T bd89e13e0c534822b3ce0391fff9e3c9 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:07.064946 11089 raft_consensus.cc:482] T bd89e13e0c534822b3ce0391fff9e3c9 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:07.065253 11089 raft_consensus.cc:3037] T bd89e13e0c534822b3ce0391fff9e3c9 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:07.095448 11082 ts_tablet_manager.cc:1428] T bbf85a35b22a4c5cbda092877818713f P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.027s user 0.007s sys 0.000s
I20260925 09:57:07.102543 11082 tablet_bootstrap.cc:492] T a0976e0c01234b38a7ab934dc1b289c4 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:07.109997 10390 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet bd89e13e0c534822b3ce0391fff9e3c9
I20260925 09:57:07.111860 10666 tablet_service.cc:1514] Processing DeleteTablet for tablet bd89e13e0c534822b3ce0391fff9e3c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:07 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:07.113147 10961 tablet_service.cc:1514] Processing DeleteTablet for tablet bd89e13e0c534822b3ce0391fff9e3c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:07 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:07.115209 10817 tablet_service.cc:1514] Processing DeleteTablet for tablet bd89e13e0c534822b3ce0391fff9e3c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:07 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:07.128130 11085 tablet_bootstrap.cc:654] T 1969ac471b6f471ebe8e048a96c22837 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:07.130756 11082 tablet_bootstrap.cc:654] T a0976e0c01234b38a7ab934dc1b289c4 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:07.134507 11096 ts_tablet_manager.cc:1918] T bbf85a35b22a4c5cbda092877818713f P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:07.134903 11096 log.cc:1196] T bbf85a35b22a4c5cbda092877818713f P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/bbf85a35b22a4c5cbda092877818713f
I20260925 09:57:07.135754 11096 ts_tablet_manager.cc:1939] T bbf85a35b22a4c5cbda092877818713f P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:07.136929 11098 ts_tablet_manager.cc:1918] T bbf85a35b22a4c5cbda092877818713f P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:07.137568 11098 log.cc:1196] T bbf85a35b22a4c5cbda092877818713f P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/bbf85a35b22a4c5cbda092877818713f
I20260925 09:57:07.138640 11098 ts_tablet_manager.cc:1939] T bbf85a35b22a4c5cbda092877818713f P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:07.139752 11089 raft_consensus.cc:504] T bd89e13e0c534822b3ce0391fff9e3c9 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:07.147414 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet bbf85a35b22a4c5cbda092877818713f (table default.Table_924d66e4d68b4a79b062e6fd7a16878f [id=f2b16afb131f4ef5a1e62a27dcce0374]) successfully deleted
I20260925 09:57:07.148433 11089 leader_election.cc:288] T bd89e13e0c534822b3ce0391fff9e3c9 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:07.149122 11082 tablet_bootstrap.cc:492] T a0976e0c01234b38a7ab934dc1b289c4 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:07.149256 11096 tablet_replica.cc:330] T bd89e13e0c534822b3ce0391fff9e3c9 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:07.149232 10688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd89e13e0c534822b3ce0391fff9e3c9" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:07.149609 11082 ts_tablet_manager.cc:1397] T a0976e0c01234b38a7ab934dc1b289c4 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.053s user 0.011s sys 0.000s
I20260925 09:57:07.149554 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd89e13e0c534822b3ce0391fff9e3c9" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:07.150084 11096 raft_consensus.cc:2229] T bd89e13e0c534822b3ce0391fff9e3c9 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:07.150321 10839 raft_consensus.cc:3037] T bd89e13e0c534822b3ce0391fff9e3c9 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:07.150693 11096 raft_consensus.cc:2258] T bd89e13e0c534822b3ce0391fff9e3c9 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 09:57:07.150959 10919 leader_election.cc:341] T bd89e13e0c534822b3ce0391fff9e3c9 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:07.153586 11085 tablet_bootstrap.cc:492] T 1969ac471b6f471ebe8e048a96c22837 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:07.153957 11085 ts_tablet_manager.cc:1397] T 1969ac471b6f471ebe8e048a96c22837 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.100s user 0.009s sys 0.002s
W20260925 09:57:07.156590 10393 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6a5121b78e2d4643b99bd8717c6e93ca consensus state has no leader
I20260925 09:57:07.159650 11098 tablet_replica.cc:330] T bd89e13e0c534822b3ce0391fff9e3c9 P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:07.159885 10665 tablet_service.cc:1467] Processing CreateTablet for tablet 6b03e5aad59e46a6b0e43b9cb36278d7 (DEFAULT_TABLE table=default.Table_534d354d5b4f4f90b8f80a76ddbc8214 [id=2c0582d5ad4b4eab8db802302c925de1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:07.161233 10666 tablet_service.cc:1467] Processing CreateTablet for tablet a8be01fe42894b96bbdd5ce2403ff1f4 (DEFAULT_TABLE table=default.Table_b13e058d7a264358a7b057cb23f04d18 [id=5e17848977874de3af045b3202674dbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:07.156075 11085 raft_consensus.cc:348] T 1969ac471b6f471ebe8e048a96c22837 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:07.161972 11085 raft_consensus.cc:374] T 1969ac471b6f471ebe8e048a96c22837 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:07.162115 11096 ts_tablet_manager.cc:1905] T bd89e13e0c534822b3ce0391fff9e3c9 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:07.162343 11085 raft_consensus.cc:729] T 1969ac471b6f471ebe8e048a96c22837 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:07.162959 11085 consensus_queue.cc:260] T 1969ac471b6f471ebe8e048a96c22837 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:07.163760 10817 tablet_service.cc:1467] Processing CreateTablet for tablet a8be01fe42894b96bbdd5ce2403ff1f4 (DEFAULT_TABLE table=default.Table_b13e058d7a264358a7b057cb23f04d18 [id=5e17848977874de3af045b3202674dbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:07.164970 10817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8be01fe42894b96bbdd5ce2403ff1f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:07.168270 11082 raft_consensus.cc:348] T a0976e0c01234b38a7ab934dc1b289c4 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:07.175151 10392 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 1969ac471b6f471ebe8e048a96c22837 (table default.Table_a9dd56ac2283472da4a217ea2d04fa2f [id=f17a11d223204599afb24e2e9d1d9d9d]). Expected version 2 got 0
I20260925 09:57:07.178368 10391 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_534d354d5b4f4f90b8f80a76ddbc8214" } modify_external_catalogs: true
I20260925 09:57:07.178932 10391 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_534d354d5b4f4f90b8f80a76ddbc8214" } modify_external_catalogs: true
I20260925 09:57:07.180554 11085 ts_tablet_manager.cc:1428] T 1969ac471b6f471ebe8e048a96c22837 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.026s user 0.005s sys 0.000s
I20260925 09:57:07.168967 11082 raft_consensus.cc:374] T a0976e0c01234b38a7ab934dc1b289c4 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:07.180876 10961 tablet_service.cc:1467] Processing CreateTablet for tablet a8be01fe42894b96bbdd5ce2403ff1f4 (DEFAULT_TABLE table=default.Table_b13e058d7a264358a7b057cb23f04d18 [id=5e17848977874de3af045b3202674dbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:07.181143 11082 raft_consensus.cc:729] T a0976e0c01234b38a7ab934dc1b289c4 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:07.182026 10961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8be01fe42894b96bbdd5ce2403ff1f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:07.181944 10390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_8f64c62cd63d43ce89d446956c0883e3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:07.183938 10664 tablet_service.cc:1467] Processing CreateTablet for tablet e3b6b985a76b459dbd1de42ea20d3856 (DEFAULT_TABLE table=default.Table_c4ae6258da494589b65be6ede95f4a1d [id=5e5f72cecad44b5e9bfd60320ee42243]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 09:57:07.185111 10390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8f64c62cd63d43ce89d446956c0883e3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:07.176004 10392 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 4be3f6e81ddf418fabc7b694a2e62f35 (table default.Table_1e7379199917430eb9e027fc7bb2a4f9 [id=c2d0336fd84c4dbe95472f458b813fe6]). Expected version 1 got 0
I20260925 09:57:07.185963 10960 tablet_service.cc:1467] Processing CreateTablet for tablet 6b03e5aad59e46a6b0e43b9cb36278d7 (DEFAULT_TABLE table=default.Table_534d354d5b4f4f90b8f80a76ddbc8214 [id=2c0582d5ad4b4eab8db802302c925de1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:07.187371 10963 tablet_service.cc:1467] Processing CreateTablet for tablet e3b6b985a76b459dbd1de42ea20d3856 (DEFAULT_TABLE table=default.Table_c4ae6258da494589b65be6ede95f4a1d [id=5e5f72cecad44b5e9bfd60320ee42243]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:07.188951 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet bbf85a35b22a4c5cbda092877818713f (table default.Table_924d66e4d68b4a79b062e6fd7a16878f [id=f2b16afb131f4ef5a1e62a27dcce0374]) successfully deleted
I20260925 09:57:07.189307 11082 consensus_queue.cc:260] T a0976e0c01234b38a7ab934dc1b289c4 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:07.133941 11095 ts_tablet_manager.cc:1918] T d803e8490c564b3d8d5435374f93dc2b P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:07.190773 11095 log.cc:1196] T d803e8490c564b3d8d5435374f93dc2b P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/d803e8490c564b3d8d5435374f93dc2b
I20260925 09:57:07.190905 10816 tablet_service.cc:1467] Processing CreateTablet for tablet 6b03e5aad59e46a6b0e43b9cb36278d7 (DEFAULT_TABLE table=default.Table_534d354d5b4f4f90b8f80a76ddbc8214 [id=2c0582d5ad4b4eab8db802302c925de1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:07.192001 10816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b03e5aad59e46a6b0e43b9cb36278d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:07.196422 11085 tablet_bootstrap.cc:492] T 8aae639695f44d8c8a61b3ea42ab6ac8 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:07.196506 10815 tablet_service.cc:1467] Processing CreateTablet for tablet e3b6b985a76b459dbd1de42ea20d3856 (DEFAULT_TABLE table=default.Table_c4ae6258da494589b65be6ede95f4a1d [id=5e5f72cecad44b5e9bfd60320ee42243]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:07.197247 11089 raft_consensus.cc:482] T d8d6ac842c8040659c2589049eb21c5f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:07.197666 11089 raft_consensus.cc:504] T d8d6ac842c8040659c2589049eb21c5f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:07.198767 11095 ts_tablet_manager.cc:1939] T d803e8490c564b3d8d5435374f93dc2b P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:07.200536 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8d6ac842c8040659c2589049eb21c5f" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:07.201105 10838 raft_consensus.cc:2454] T d8d6ac842c8040659c2589049eb21c5f P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be97ab5d75934a648d4ab76341193ec0 in term 0.
I20260925 09:57:07.202498 10918 leader_election.cc:302] T d8d6ac842c8040659c2589049eb21c5f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:07.203373 11134 raft_consensus.cc:2781] T d8d6ac842c8040659c2589049eb21c5f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:07.203703 11134 raft_consensus.cc:482] T d8d6ac842c8040659c2589049eb21c5f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:07.203989 11134 raft_consensus.cc:3037] T d8d6ac842c8040659c2589049eb21c5f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:07.204679 11085 tablet_bootstrap.cc:654] T 8aae639695f44d8c8a61b3ea42ab6ac8 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:07.208339 10815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3b6b985a76b459dbd1de42ea20d3856. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:07.210281 11082 ts_tablet_manager.cc:1428] T a0976e0c01234b38a7ab934dc1b289c4 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.060s user 0.007s sys 0.000s
I20260925 09:57:07.212010 11089 leader_election.cc:288] T d8d6ac842c8040659c2589049eb21c5f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:07.215476 10386 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_a8fb0e19083b495c978e0318d857492f" } new_table_name: "default.Table_699fb6630e69435d946304ff8473d1e8" modify_external_catalogs: true
I20260925 09:57:07.224960 10393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_18ecd2dd5b4e472cb12e72ce8bce84f9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:07.226521 10393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_18ecd2dd5b4e472cb12e72ce8bce84f9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:07.227851 10384 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_ce8527f03cfc4ae29d76f64597e0b075"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:07.228410 10839 raft_consensus.cc:2454] T bd89e13e0c534822b3ce0391fff9e3c9 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be97ab5d75934a648d4ab76341193ec0 in term 1.
W20260925 09:57:07.229310 10384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ce8527f03cfc4ae29d76f64597e0b075 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:07.231424 11089 raft_consensus.cc:482] T 4be3f6e81ddf418fabc7b694a2e62f35 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:07.231838 11089 raft_consensus.cc:504] T 4be3f6e81ddf418fabc7b694a2e62f35 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:07.233812 10389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_cc535de81c9845ba80428bcc01e78cfa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:07.235323 10389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cc535de81c9845ba80428bcc01e78cfa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:07.235697 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4be3f6e81ddf418fabc7b694a2e62f35" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:07.237004 10960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b03e5aad59e46a6b0e43b9cb36278d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:07.236981 11085 tablet_bootstrap.cc:492] T 8aae639695f44d8c8a61b3ea42ab6ac8 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:07.237412 11085 ts_tablet_manager.cc:1397] T 8aae639695f44d8c8a61b3ea42ab6ac8 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.041s user 0.008s sys 0.001s
I20260925 09:57:07.238363 10963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3b6b985a76b459dbd1de42ea20d3856. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:07.240175 11085 raft_consensus.cc:348] T 8aae639695f44d8c8a61b3ea42ab6ac8 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:07.241027 11085 raft_consensus.cc:374] T 8aae639695f44d8c8a61b3ea42ab6ac8 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:07.241329 11085 raft_consensus.cc:729] T 8aae639695f44d8c8a61b3ea42ab6ac8 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:07.242107 11085 consensus_queue.cc:260] T 8aae639695f44d8c8a61b3ea42ab6ac8 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:07.242276 10918 leader_election.cc:341] T 4be3f6e81ddf418fabc7b694a2e62f35 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:07.246918 11089 leader_election.cc:288] T 4be3f6e81ddf418fabc7b694a2e62f35 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:07.247996 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4be3f6e81ddf418fabc7b694a2e62f35" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:07.248641 10688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8d6ac842c8040659c2589049eb21c5f" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:07.249419 10665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b03e5aad59e46a6b0e43b9cb36278d7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:07.250098 10919 leader_election.cc:341] T 4be3f6e81ddf418fabc7b694a2e62f35 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:07.250653 10919 leader_election.cc:302] T 4be3f6e81ddf418fabc7b694a2e62f35 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:07.252200 10666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8be01fe42894b96bbdd5ce2403ff1f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:07.259145 11085 ts_tablet_manager.cc:1428] T 8aae639695f44d8c8a61b3ea42ab6ac8 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.021s user 0.006s sys 0.001s
I20260925 09:57:07.259665 10962 tablet_service.cc:1514] Processing DeleteTablet for tablet bbf85a35b22a4c5cbda092877818713f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:06 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:07.260133 10664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3b6b985a76b459dbd1de42ea20d3856. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:07.255436 11089 raft_consensus.cc:2726] T 4be3f6e81ddf418fabc7b694a2e62f35 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:07.265374 11082 tablet_bootstrap.cc:492] T d8d6ac842c8040659c2589049eb21c5f P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:07.265305 10918 leader_election.cc:302] T bd89e13e0c534822b3ce0391fff9e3c9 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21
W20260925 09:57:07.266047 10918 raft_consensus.cc:2670] T bd89e13e0c534822b3ce0391fff9e3c9 P be97ab5d75934a648d4ab76341193ec0: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260925 09:57:07.266887 11098 raft_consensus.cc:2229] T bd89e13e0c534822b3ce0391fff9e3c9 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:07.272589 11098 raft_consensus.cc:2258] T bd89e13e0c534822b3ce0391fff9e3c9 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:07.276924 11098 ts_tablet_manager.cc:1905] T bd89e13e0c534822b3ce0391fff9e3c9 P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:07.278165 11085 tablet_bootstrap.cc:492] T 4be3f6e81ddf418fabc7b694a2e62f35 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:07.278251 11096 ts_tablet_manager.cc:1918] T bd89e13e0c534822b3ce0391fff9e3c9 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:07.278602 11096 log.cc:1196] T bd89e13e0c534822b3ce0391fff9e3c9 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/bd89e13e0c534822b3ce0391fff9e3c9
I20260925 09:57:07.279251 11096 ts_tablet_manager.cc:1939] T bd89e13e0c534822b3ce0391fff9e3c9 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:07.292428 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet d803e8490c564b3d8d5435374f93dc2b (table default.Table_a400e3e492de40e988e9cb29284a2003 [id=9230b1811a8a4faba58b239cf40992dd]) successfully deleted
I20260925 09:57:07.295881 11095 tablet_replica.cc:330] T bbf85a35b22a4c5cbda092877818713f P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:07.297027 11095 raft_consensus.cc:2229] T bbf85a35b22a4c5cbda092877818713f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:07.297658 11095 raft_consensus.cc:2258] T bbf85a35b22a4c5cbda092877818713f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:07.301770 11082 tablet_bootstrap.cc:654] T d8d6ac842c8040659c2589049eb21c5f P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:07.304546 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet bd89e13e0c534822b3ce0391fff9e3c9 (table default.Table_aefaa3aacf0e43e1b32aa184267892d4 [id=e6ae2bf4301141f790fefe7bc560029a]) successfully deleted
W20260925 09:57:07.305277 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet bbf85a35b22a4c5cbda092877818713f because the tablet was not found. No further retry: Not found: Tablet not found: bbf85a35b22a4c5cbda092877818713f
W20260925 09:57:07.306344 10380 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 4be3f6e81ddf418fabc7b694a2e62f35 (table default.Table_1e7379199917430eb9e027fc7bb2a4f9 [id=c2d0336fd84c4dbe95472f458b813fe6], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 4be3f6e81ddf418fabc7b694a2e62f35 consensus state has no leader
I20260925 09:57:07.318975 10392 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_000f52fd1e214fa1931ef3743ce76250" } modify_external_catalogs: true
I20260925 09:57:07.319690 10392 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_000f52fd1e214fa1931ef3743ce76250" } modify_external_catalogs: true
I20260925 09:57:07.315264 11085 tablet_bootstrap.cc:654] T 4be3f6e81ddf418fabc7b694a2e62f35 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:07.329474 11095 ts_tablet_manager.cc:1905] T bbf85a35b22a4c5cbda092877818713f P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:07.308359 11089 raft_consensus.cc:482] T a0976e0c01234b38a7ab934dc1b289c4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:07.338454 11089 raft_consensus.cc:504] T a0976e0c01234b38a7ab934dc1b289c4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:07.343439 11084 tablet_bootstrap.cc:492] T a8be01fe42894b96bbdd5ce2403ff1f4 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:07.345937 11089 leader_election.cc:288] T a0976e0c01234b38a7ab934dc1b289c4 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
W20260925 09:57:07.343699 10919 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 93648.6us
W20260925 09:57:07.348608 10919 leader_election.cc:341] T d8d6ac842c8040659c2589049eb21c5f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:07.283089 11134 raft_consensus.cc:504] T d8d6ac842c8040659c2589049eb21c5f P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:07.352428 10688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0976e0c01234b38a7ab934dc1b289c4" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:07.353047 10688 raft_consensus.cc:2454] T a0976e0c01234b38a7ab934dc1b289c4 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be97ab5d75934a648d4ab76341193ec0 in term 0.
I20260925 09:57:07.354279 11084 tablet_bootstrap.cc:654] T a8be01fe42894b96bbdd5ce2403ff1f4 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:07.355188 10919 leader_election.cc:302] T a0976e0c01234b38a7ab934dc1b289c4 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:07.355748 10688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8d6ac842c8040659c2589049eb21c5f" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
W20260925 09:57:07.359925 10379 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.60:36985, user_credentials={real_user=slave}} blocked reactor thread for 68295.7us
I20260925 09:57:07.360638 10387 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_a9dd56ac2283472da4a217ea2d04fa2f" } new_table_name: "default.Table_4e09a615fb82487baa4783a5f6e2181e" modify_external_catalogs: true
I20260925 09:57:07.367604 11089 raft_consensus.cc:2781] T a0976e0c01234b38a7ab934dc1b289c4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:07.367938 11089 raft_consensus.cc:482] T a0976e0c01234b38a7ab934dc1b289c4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:07.368252 11089 raft_consensus.cc:3037] T a0976e0c01234b38a7ab934dc1b289c4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:07.373224 11122 raft_consensus.cc:482] T a0976e0c01234b38a7ab934dc1b289c4 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:07.373719 11122 raft_consensus.cc:504] T a0976e0c01234b38a7ab934dc1b289c4 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:07.375782 10391 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 6b03e5aad59e46a6b0e43b9cb36278d7
I20260925 09:57:07.380209 11095 ts_tablet_manager.cc:1918] T bbf85a35b22a4c5cbda092877818713f P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:07.380553 11095 log.cc:1196] T bbf85a35b22a4c5cbda092877818713f P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/bbf85a35b22a4c5cbda092877818713f
I20260925 09:57:07.381134 11095 ts_tablet_manager.cc:1939] T bbf85a35b22a4c5cbda092877818713f P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:07.381598 11122 leader_election.cc:288] T a0976e0c01234b38a7ab934dc1b289c4 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:07.382215 11122 raft_consensus.cc:482] T 1969ac471b6f471ebe8e048a96c22837 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:07.384930 10688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0976e0c01234b38a7ab934dc1b289c4" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:07.385612 10688 raft_consensus.cc:2454] T a0976e0c01234b38a7ab934dc1b289c4 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
I20260925 09:57:07.387108 10775 leader_election.cc:302] T a0976e0c01234b38a7ab934dc1b289c4 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:07.382676 11122 raft_consensus.cc:504] T 1969ac471b6f471ebe8e048a96c22837 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:07.389362 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0976e0c01234b38a7ab934dc1b289c4" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:07.387652 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0976e0c01234b38a7ab934dc1b289c4" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:07.390506 10839 raft_consensus.cc:2454] T a0976e0c01234b38a7ab934dc1b289c4 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be97ab5d75934a648d4ab76341193ec0 in term 0.
I20260925 09:57:07.394245 10664 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b03e5aad59e46a6b0e43b9cb36278d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:07 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:07.397533 10982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1969ac471b6f471ebe8e048a96c22837" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:07.398464 10982 raft_consensus.cc:2454] T 1969ac471b6f471ebe8e048a96c22837 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
I20260925 09:57:07.400455 10772 leader_election.cc:302] T 1969ac471b6f471ebe8e048a96c22837 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:07.402098 10688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1969ac471b6f471ebe8e048a96c22837" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:07.406606 11084 tablet_bootstrap.cc:492] T a8be01fe42894b96bbdd5ce2403ff1f4 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:07.407136 11084 ts_tablet_manager.cc:1397] T a8be01fe42894b96bbdd5ce2403ff1f4 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.064s user 0.008s sys 0.000s
I20260925 09:57:07.407806 10961 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b03e5aad59e46a6b0e43b9cb36278d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:07 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:07.408051 11152 raft_consensus.cc:482] T 8aae639695f44d8c8a61b3ea42ab6ac8 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:07.413954 11152 raft_consensus.cc:504] T 8aae639695f44d8c8a61b3ea42ab6ac8 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:07.418088 11152 leader_election.cc:288] T 8aae639695f44d8c8a61b3ea42ab6ac8 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:07.420946 11159 raft_consensus.cc:2781] T a0976e0c01234b38a7ab934dc1b289c4 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260925 09:57:07.421201 10377 catalog_manager.cc:4821] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 6b03e5aad59e46a6b0e43b9cb36278d7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6b03e5aad59e46a6b0e43b9cb36278d7 already in progress: creating tablet
I20260925 09:57:07.421347 11159 raft_consensus.cc:482] T a0976e0c01234b38a7ab934dc1b289c4 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:07.421780 11159 raft_consensus.cc:3037] T a0976e0c01234b38a7ab934dc1b289c4 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:57:07.420321 10775 leader_election.cc:341] T 1969ac471b6f471ebe8e048a96c22837 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:07.428192 10919 leader_election.cc:341] T d8d6ac842c8040659c2589049eb21c5f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:07.429749 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet bbf85a35b22a4c5cbda092877818713f (table default.Table_924d66e4d68b4a79b062e6fd7a16878f [id=f2b16afb131f4ef5a1e62a27dcce0374]) successfully deleted
I20260925 09:57:07.401579 11122 leader_election.cc:288] T 1969ac471b6f471ebe8e048a96c22837 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
W20260925 09:57:07.430488 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet bd89e13e0c534822b3ce0391fff9e3c9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bd89e13e0c534822b3ce0391fff9e3c9 already in progress: creating tablet
I20260925 09:57:07.430476 10688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aae639695f44d8c8a61b3ea42ab6ac8" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:07.430932 11122 raft_consensus.cc:2781] T 1969ac471b6f471ebe8e048a96c22837 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260925 09:57:07.431107 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 6b03e5aad59e46a6b0e43b9cb36278d7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6b03e5aad59e46a6b0e43b9cb36278d7 already in progress: creating tablet
I20260925 09:57:07.431285 11122 raft_consensus.cc:482] T 1969ac471b6f471ebe8e048a96c22837 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:07.431670 11122 raft_consensus.cc:3037] T 1969ac471b6f471ebe8e048a96c22837 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:07.436857 11084 raft_consensus.cc:348] T a8be01fe42894b96bbdd5ce2403ff1f4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:07.438107 11152 consensus_queue.cc:1035] T 840faf042d8749bd958ce1de0207a734 P be97ab5d75934a648d4ab76341193ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:07.442332 11084 raft_consensus.cc:374] T a8be01fe42894b96bbdd5ce2403ff1f4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:07.442602 11084 raft_consensus.cc:729] T a8be01fe42894b96bbdd5ce2403ff1f4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:07.443177 11084 consensus_queue.cc:260] T a8be01fe42894b96bbdd5ce2403ff1f4 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:07.444279 11082 tablet_bootstrap.cc:492] T d8d6ac842c8040659c2589049eb21c5f P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:07.444851 11082 ts_tablet_manager.cc:1397] T d8d6ac842c8040659c2589049eb21c5f P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.180s user 0.026s sys 0.050s
I20260925 09:57:07.447356 11082 raft_consensus.cc:348] T d8d6ac842c8040659c2589049eb21c5f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
W20260925 09:57:07.439193 10919 leader_election.cc:341] T 8aae639695f44d8c8a61b3ea42ab6ac8 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:07.448688 11082 raft_consensus.cc:374] T d8d6ac842c8040659c2589049eb21c5f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:07.448987 11082 raft_consensus.cc:729] T d8d6ac842c8040659c2589049eb21c5f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
W20260925 09:57:07.449812 10772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 49488.3us
I20260925 09:57:07.449622 11082 consensus_queue.cc:260] T d8d6ac842c8040659c2589049eb21c5f P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:07.451232 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aae639695f44d8c8a61b3ea42ab6ac8" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:07.451799 11082 ts_tablet_manager.cc:1428] T d8d6ac842c8040659c2589049eb21c5f P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.007s user 0.001s sys 0.004s
I20260925 09:57:07.452006 10839 raft_consensus.cc:2454] T 8aae639695f44d8c8a61b3ea42ab6ac8 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be97ab5d75934a648d4ab76341193ec0 in term 0.
I20260925 09:57:07.452663 11082 tablet_bootstrap.cc:492] T bd89e13e0c534822b3ce0391fff9e3c9 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:07.454573 10918 leader_election.cc:302] T 8aae639695f44d8c8a61b3ea42ab6ac8 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21
I20260925 09:57:07.458287 11082 tablet_bootstrap.cc:654] T bd89e13e0c534822b3ce0391fff9e3c9 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:07.459640 11134 leader_election.cc:288] T d8d6ac842c8040659c2589049eb21c5f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:07.460965 11134 raft_consensus.cc:2781] T 8aae639695f44d8c8a61b3ea42ab6ac8 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:07.461483 11134 raft_consensus.cc:482] T 8aae639695f44d8c8a61b3ea42ab6ac8 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:07.462023 11134 raft_consensus.cc:3037] T 8aae639695f44d8c8a61b3ea42ab6ac8 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:57:07.470706 10386 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d48d4517c13049389c5b2ba1ab7b14bc has no consensus state
I20260925 09:57:07.479276 10818 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b03e5aad59e46a6b0e43b9cb36278d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:07 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:07.481364 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8d6ac842c8040659c2589049eb21c5f" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:07.481994 10839 raft_consensus.cc:3037] T d8d6ac842c8040659c2589049eb21c5f P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:07.484668 11084 ts_tablet_manager.cc:1428] T a8be01fe42894b96bbdd5ce2403ff1f4 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.072s user 0.006s sys 0.000s
I20260925 09:57:07.488055 11160 consensus_queue.cc:1035] T 840faf042d8749bd958ce1de0207a734 P be97ab5d75934a648d4ab76341193ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:07.498636 11084 tablet_bootstrap.cc:492] T 6b03e5aad59e46a6b0e43b9cb36278d7 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:07.505621 11084 tablet_bootstrap.cc:654] T 6b03e5aad59e46a6b0e43b9cb36278d7 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:07.506175 11085 tablet_bootstrap.cc:492] T 4be3f6e81ddf418fabc7b694a2e62f35 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:07.506558 11085 ts_tablet_manager.cc:1397] T 4be3f6e81ddf418fabc7b694a2e62f35 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.229s user 0.011s sys 0.011s
I20260925 09:57:07.508687 11085 raft_consensus.cc:348] T 4be3f6e81ddf418fabc7b694a2e62f35 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:07.509250 11082 tablet_bootstrap.cc:492] T bd89e13e0c534822b3ce0391fff9e3c9 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:07.509306 11085 raft_consensus.cc:374] T 4be3f6e81ddf418fabc7b694a2e62f35 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:07.509554 11085 raft_consensus.cc:729] T 4be3f6e81ddf418fabc7b694a2e62f35 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:07.509743 11082 ts_tablet_manager.cc:1397] T bd89e13e0c534822b3ce0391fff9e3c9 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.057s user 0.008s sys 0.017s
I20260925 09:57:07.519780 11122 raft_consensus.cc:504] T 1969ac471b6f471ebe8e048a96c22837 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
W20260925 09:57:07.520606 10387 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1969ac471b6f471ebe8e048a96c22837 consensus state has no leader
I20260925 09:57:07.523243 10982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1969ac471b6f471ebe8e048a96c22837" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:07.523952 10982 raft_consensus.cc:3037] T 1969ac471b6f471ebe8e048a96c22837 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:07.510211 11085 consensus_queue.cc:260] T 4be3f6e81ddf418fabc7b694a2e62f35 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:07.523381 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1969ac471b6f471ebe8e048a96c22837" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
W20260925 09:57:07.530480 10775 leader_election.cc:341] T 1969ac471b6f471ebe8e048a96c22837 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:07.533488 10982 raft_consensus.cc:2454] T 1969ac471b6f471ebe8e048a96c22837 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:07.533655 11159 raft_consensus.cc:504] T a0976e0c01234b38a7ab934dc1b289c4 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:07.534577 10772 leader_election.cc:302] T 1969ac471b6f471ebe8e048a96c22837 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21
I20260925 09:57:07.535717 11157 raft_consensus.cc:2781] T 1969ac471b6f471ebe8e048a96c22837 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:07.530771 11082 raft_consensus.cc:348] T bd89e13e0c534822b3ce0391fff9e3c9 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:07.543026 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0976e0c01234b38a7ab934dc1b289c4" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:07.543630 11082 raft_consensus.cc:374] T bd89e13e0c534822b3ce0391fff9e3c9 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:07.543843 10687 raft_consensus.cc:3037] T a0976e0c01234b38a7ab934dc1b289c4 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:07.543987 11082 raft_consensus.cc:729] T bd89e13e0c534822b3ce0391fff9e3c9 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:07.560384 11159 leader_election.cc:288] T a0976e0c01234b38a7ab934dc1b289c4 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:07.560586 11082 consensus_queue.cc:260] T bd89e13e0c534822b3ce0391fff9e3c9 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:07.561702 11109 raft_consensus.cc:482] T bd89e13e0c534822b3ce0391fff9e3c9 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:07.562215 11109 raft_consensus.cc:504] T bd89e13e0c534822b3ce0391fff9e3c9 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:07.562369 10982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0976e0c01234b38a7ab934dc1b289c4" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:07.565754 11098 ts_tablet_manager.cc:1918] T bd89e13e0c534822b3ce0391fff9e3c9 P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:07.568682 11109 leader_election.cc:288] T bd89e13e0c534822b3ce0391fff9e3c9 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:07.570338 11098 log.cc:1196] T bd89e13e0c534822b3ce0391fff9e3c9 P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/bd89e13e0c534822b3ce0391fff9e3c9
I20260925 09:57:07.569234 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd89e13e0c534822b3ce0391fff9e3c9" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:07.574412 11082 ts_tablet_manager.cc:1428] T bd89e13e0c534822b3ce0391fff9e3c9 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.046s user 0.006s sys 0.000s
W20260925 09:57:07.574460 10621 leader_election.cc:341] T bd89e13e0c534822b3ce0391fff9e3c9 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: Tablet not RUNNING: STOPPED
I20260925 09:57:07.567533 11122 leader_election.cc:288] T 1969ac471b6f471ebe8e048a96c22837 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:07.576346 11098 ts_tablet_manager.cc:1939] T bd89e13e0c534822b3ce0391fff9e3c9 P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:07.579005 11082 tablet_bootstrap.cc:492] T 1969ac471b6f471ebe8e048a96c22837 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:07.582297 10839 raft_consensus.cc:2454] T d8d6ac842c8040659c2589049eb21c5f P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be97ab5d75934a648d4ab76341193ec0 in term 1.
I20260925 09:57:07.584858 10918 leader_election.cc:302] T d8d6ac842c8040659c2589049eb21c5f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21
I20260925 09:57:07.586356 11160 raft_consensus.cc:2781] T d8d6ac842c8040659c2589049eb21c5f P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:07.587728 10387 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_699fb6630e69435d946304ff8473d1e8" } modify_external_catalogs: true
I20260925 09:57:07.588356 10387 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_699fb6630e69435d946304ff8473d1e8" } modify_external_catalogs: true
I20260925 09:57:07.590476 10687 raft_consensus.cc:2454] T a0976e0c01234b38a7ab934dc1b289c4 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:07.592253 11089 raft_consensus.cc:504] T a0976e0c01234b38a7ab934dc1b289c4 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:07.593524 10984 raft_consensus.cc:2379] T a0976e0c01234b38a7ab934dc1b289c4 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c43461c4ee854ae1ac52b873ccada318 in current term 1: Already voted for candidate be97ab5d75934a648d4ab76341193ec0 in this term.
I20260925 09:57:07.594102 11085 ts_tablet_manager.cc:1428] T 4be3f6e81ddf418fabc7b694a2e62f35 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.087s user 0.007s sys 0.000s
W20260925 09:57:07.598960 10619 leader_election.cc:341] T bd89e13e0c534822b3ce0391fff9e3c9 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: bd89e13e0c534822b3ce0391fff9e3c9
I20260925 09:57:07.596318 10775 leader_election.cc:302] T a0976e0c01234b38a7ab934dc1b289c4 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters: be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:07.599437 10619 leader_election.cc:302] T bd89e13e0c534822b3ce0391fff9e3c9 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8850818e2ee42a082802d0095c08d21; no voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:07.599758 11084 tablet_bootstrap.cc:492] T 6b03e5aad59e46a6b0e43b9cb36278d7 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:07.600097 11122 raft_consensus.cc:2781] T a0976e0c01234b38a7ab934dc1b289c4 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:07.600251 11084 ts_tablet_manager.cc:1397] T 6b03e5aad59e46a6b0e43b9cb36278d7 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.102s user 0.008s sys 0.005s
I20260925 09:57:07.600419 11109 raft_consensus.cc:2726] T bd89e13e0c534822b3ce0391fff9e3c9 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:07.600555 11122 raft_consensus.cc:686] T a0976e0c01234b38a7ab934dc1b289c4 P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Becoming Leader. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Running, Role: LEADER
I20260925 09:57:07.601476 11122 consensus_queue.cc:237] T a0976e0c01234b38a7ab934dc1b289c4 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:07.602445 11089 leader_election.cc:288] T a0976e0c01234b38a7ab934dc1b289c4 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:07.604750 10393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_2bc55a20487a41b182135205d1a4b65d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:07.605301 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0976e0c01234b38a7ab934dc1b289c4" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
W20260925 09:57:07.606590 10393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2bc55a20487a41b182135205d1a4b65d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:07.610000 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0976e0c01234b38a7ab934dc1b289c4" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:07.610863 10686 raft_consensus.cc:2379] T a0976e0c01234b38a7ab934dc1b289c4 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate be97ab5d75934a648d4ab76341193ec0 in current term 1: Already voted for candidate c43461c4ee854ae1ac52b873ccada318 in this term.
I20260925 09:57:07.613075 10919 leader_election.cc:302] T a0976e0c01234b38a7ab934dc1b289c4 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:07.615142 11089 raft_consensus.cc:2726] T a0976e0c01234b38a7ab934dc1b289c4 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260925 09:57:07.616319 11157 raft_consensus.cc:686] T 1969ac471b6f471ebe8e048a96c22837 P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Becoming Leader. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Running, Role: LEADER
I20260925 09:57:07.617393 11157 consensus_queue.cc:237] T 1969ac471b6f471ebe8e048a96c22837 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:07.625782 11160 raft_consensus.cc:686] T d8d6ac842c8040659c2589049eb21c5f P be97ab5d75934a648d4ab76341193ec0 [term 1 LEADER]: Becoming Leader. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Running, Role: LEADER
I20260925 09:57:07.626891 11160 consensus_queue.cc:237] T d8d6ac842c8040659c2589049eb21c5f P be97ab5d75934a648d4ab76341193ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:07.629503 11134 raft_consensus.cc:504] T 8aae639695f44d8c8a61b3ea42ab6ac8 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:07.636504 11134 leader_election.cc:288] T 8aae639695f44d8c8a61b3ea42ab6ac8 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:07.639573 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aae639695f44d8c8a61b3ea42ab6ac8" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:07.640304 10839 raft_consensus.cc:3037] T 8aae639695f44d8c8a61b3ea42ab6ac8 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:07.641114 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet bd89e13e0c534822b3ce0391fff9e3c9 (table default.Table_aefaa3aacf0e43e1b32aa184267892d4 [id=e6ae2bf4301141f790fefe7bc560029a]) successfully deleted
W20260925 09:57:07.642596 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 6b03e5aad59e46a6b0e43b9cb36278d7 because the tablet was not found. No further retry: Not found: Tablet not found: 6b03e5aad59e46a6b0e43b9cb36278d7
I20260925 09:57:07.643560 11085 tablet_bootstrap.cc:492] T a8be01fe42894b96bbdd5ce2403ff1f4 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:07.657490 11084 raft_consensus.cc:348] T 6b03e5aad59e46a6b0e43b9cb36278d7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:07.662670 10385 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 6a5121b78e2d4643b99bd8717c6e93ca (table default.Table_5ff0f5bde5954217976f2876e76adaaa [id=aa7e7f5af2724af797170e1f7429d76a]). Expected version 1 got 0
I20260925 09:57:07.667153 11084 raft_consensus.cc:374] T 6b03e5aad59e46a6b0e43b9cb36278d7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:07.667446 11084 raft_consensus.cc:729] T 6b03e5aad59e46a6b0e43b9cb36278d7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:07.668074 11084 consensus_queue.cc:260] T 6b03e5aad59e46a6b0e43b9cb36278d7 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:07.623298 11082 tablet_bootstrap.cc:654] T 1969ac471b6f471ebe8e048a96c22837 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:07.665928 10664 tablet_service.cc:1514] Processing DeleteTablet for tablet bbf85a35b22a4c5cbda092877818713f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:06 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:07.650753 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aae639695f44d8c8a61b3ea42ab6ac8" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
W20260925 09:57:07.673658 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet bbf85a35b22a4c5cbda092877818713f because the tablet was not found. No further retry: Not found: Tablet not found: bbf85a35b22a4c5cbda092877818713f
W20260925 09:57:07.674083 10919 leader_election.cc:341] T 8aae639695f44d8c8a61b3ea42ab6ac8 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:07.679672 10386 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_cc535de81c9845ba80428bcc01e78cfa" } new_table_name: "default.Table_ecec3b643d1c4b76b700090f3a3cab06" modify_external_catalogs: true
I20260925 09:57:07.683646 10389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_c6c124e548d3457fab9199a521cfa2c9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:07.685192 10389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c6c124e548d3457fab9199a521cfa2c9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:07.690987 10384 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_62ea48a919ec4159bcc86ebb048e3545"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:07.693220 10384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_62ea48a919ec4159bcc86ebb048e3545 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:07.702224 10390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_fe84e50a7371410d81b423c4324b17f2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:07.704335 10390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fe84e50a7371410d81b423c4324b17f2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:07.710366 10388 catalog_manager.cc:5495] T 6a5121b78e2d4643b99bd8717c6e93ca P c43461c4ee854ae1ac52b873ccada318 reported cstate change: term changed from 0 to 1, leader changed from <none> to c43461c4ee854ae1ac52b873ccada318 (127.10.9.2). New cstate: current_term: 1 leader_uuid: "c43461c4ee854ae1ac52b873ccada318" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:07.712316 10839 raft_consensus.cc:2454] T 8aae639695f44d8c8a61b3ea42ab6ac8 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be97ab5d75934a648d4ab76341193ec0 in term 1.
I20260925 09:57:07.713200 10388 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 6a5121b78e2d4643b99bd8717c6e93ca (table default.Table_5ff0f5bde5954217976f2876e76adaaa [id=aa7e7f5af2724af797170e1f7429d76a]). Expected version 1 got 0
I20260925 09:57:07.713749 10918 leader_election.cc:302] T 8aae639695f44d8c8a61b3ea42ab6ac8 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21
I20260925 09:57:07.714808 11134 raft_consensus.cc:2781] T 8aae639695f44d8c8a61b3ea42ab6ac8 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:07.715420 11134 raft_consensus.cc:686] T 8aae639695f44d8c8a61b3ea42ab6ac8 P be97ab5d75934a648d4ab76341193ec0 [term 1 LEADER]: Becoming Leader. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Running, Role: LEADER
I20260925 09:57:07.716444 11134 consensus_queue.cc:237] T 8aae639695f44d8c8a61b3ea42ab6ac8 P be97ab5d75934a648d4ab76341193ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:07.718416 10664 tablet_service.cc:1467] Processing CreateTablet for tablet 59f0c6489a33433899bb6e62088121b6 (DEFAULT_TABLE table=default.Table_3e77b19accd1463cb978e456128a53ef [id=64032d286bf94c4cbdddde18adfb739e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:07.719857 10664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59f0c6489a33433899bb6e62088121b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:07.724752 10391 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_f2634f079dff40ac91f6c47ba66482e6" } new_table_name: "default.Table_396cf11a26a04a17a2ad247fdf6a6c1b" modify_external_catalogs: true
I20260925 09:57:07.730482 10963 tablet_service.cc:1467] Processing CreateTablet for tablet 59f0c6489a33433899bb6e62088121b6 (DEFAULT_TABLE table=default.Table_3e77b19accd1463cb978e456128a53ef [id=64032d286bf94c4cbdddde18adfb739e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:07.731139 10666 tablet_service.cc:1467] Processing CreateTablet for tablet 04fafb9dcf274806b11b2598e73853e9 (DEFAULT_TABLE table=default.Table_bc69fdbbaf5b4a4e8cdfe195eefb16ab [id=d76cef751d12440190294e86649e2639]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:07.731688 10963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59f0c6489a33433899bb6e62088121b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:07.732393 10666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04fafb9dcf274806b11b2598e73853e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:07.732616 10818 tablet_service.cc:1467] Processing CreateTablet for tablet 04fafb9dcf274806b11b2598e73853e9 (DEFAULT_TABLE table=default.Table_bc69fdbbaf5b4a4e8cdfe195eefb16ab [id=d76cef751d12440190294e86649e2639]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:07.733829 10818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04fafb9dcf274806b11b2598e73853e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:07.738377 10961 tablet_service.cc:1467] Processing CreateTablet for tablet 04fafb9dcf274806b11b2598e73853e9 (DEFAULT_TABLE table=default.Table_bc69fdbbaf5b4a4e8cdfe195eefb16ab [id=d76cef751d12440190294e86649e2639]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:07.739686 10961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04fafb9dcf274806b11b2598e73853e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:07.742182 10960 tablet_service.cc:1467] Processing CreateTablet for tablet d48d4517c13049389c5b2ba1ab7b14bc (DEFAULT_TABLE table=default.Table_699fb6630e69435d946304ff8473d1e8 [id=1b1ce9c0cada4beeae7e02eb3105a564]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:07.744249 10819 tablet_service.cc:1467] Processing CreateTablet for tablet d48d4517c13049389c5b2ba1ab7b14bc (DEFAULT_TABLE table=default.Table_699fb6630e69435d946304ff8473d1e8 [id=1b1ce9c0cada4beeae7e02eb3105a564]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:07.745684 10819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d48d4517c13049389c5b2ba1ab7b14bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:07.750160 10665 tablet_service.cc:1467] Processing CreateTablet for tablet d48d4517c13049389c5b2ba1ab7b14bc (DEFAULT_TABLE table=default.Table_699fb6630e69435d946304ff8473d1e8 [id=1b1ce9c0cada4beeae7e02eb3105a564]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:07.750486 10667 tablet_service.cc:1467] Processing CreateTablet for tablet 931769e78f32467b8f27bacb286a7a60 (DEFAULT_TABLE table=default.Table_b9a40e813ba14ce89495f9edffeb4ee0 [id=68697340a8b046d79b795c15a68e1c6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:07.751461 10665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d48d4517c13049389c5b2ba1ab7b14bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:07.752095 10667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 931769e78f32467b8f27bacb286a7a60. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:07.758064 10379 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 -> Peer b8be057e0b6d40a3a4b90f2b1f4a0fa2 (127.10.9.60:36985): Couldn't send request to peer b8be057e0b6d40a3a4b90f2b1f4a0fa2. Status: Network error: Client connection negotiation failed: client connection to 127.10.9.60:36985: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260925 09:57:07.743357 10960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d48d4517c13049389c5b2ba1ab7b14bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:07.767989 11084 ts_tablet_manager.cc:1428] T 6b03e5aad59e46a6b0e43b9cb36278d7 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.167s user 0.006s sys 0.001s
I20260925 09:57:07.769567 10688 raft_consensus.cc:3037] T 6a5121b78e2d4643b99bd8717c6e93ca P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:07.777215 11159 consensus_queue.cc:1035] T 6a5121b78e2d4643b99bd8717c6e93ca P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:07.791935 11084 tablet_bootstrap.cc:492] T e3b6b985a76b459dbd1de42ea20d3856 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:07.801647 11082 tablet_bootstrap.cc:492] T 1969ac471b6f471ebe8e048a96c22837 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:07.802090 11082 ts_tablet_manager.cc:1397] T 1969ac471b6f471ebe8e048a96c22837 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.227s user 0.012s sys 0.000s
I20260925 09:57:07.803115 10387 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet d48d4517c13049389c5b2ba1ab7b14bc
I20260925 09:57:07.805274 10817 tablet_service.cc:1514] Processing DeleteTablet for tablet d48d4517c13049389c5b2ba1ab7b14bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:07 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:07.806535 11085 tablet_bootstrap.cc:654] T a8be01fe42894b96bbdd5ce2403ff1f4 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:07.806700 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet d48d4517c13049389c5b2ba1ab7b14bc because the tablet was not found. No further retry: Not found: Tablet not found: d48d4517c13049389c5b2ba1ab7b14bc
I20260925 09:57:07.813473 11084 tablet_bootstrap.cc:654] T e3b6b985a76b459dbd1de42ea20d3856 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:07.815673 10962 tablet_service.cc:1514] Processing DeleteTablet for tablet d48d4517c13049389c5b2ba1ab7b14bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:07 UTC) from {username='slave'} at 127.0.0.1:53560
W20260925 09:57:07.817772 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet d48d4517c13049389c5b2ba1ab7b14bc because the tablet was not found. No further retry: Not found: Tablet not found: d48d4517c13049389c5b2ba1ab7b14bc
I20260925 09:57:07.818311 10665 tablet_service.cc:1514] Processing DeleteTablet for tablet d48d4517c13049389c5b2ba1ab7b14bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:07 UTC) from {username='slave'} at 127.0.0.1:37876
W20260925 09:57:07.819958 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet d48d4517c13049389c5b2ba1ab7b14bc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d48d4517c13049389c5b2ba1ab7b14bc already in progress: creating tablet
I20260925 09:57:07.826758 11082 raft_consensus.cc:348] T 1969ac471b6f471ebe8e048a96c22837 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:07.827591 10818 tablet_service.cc:1467] Processing CreateTablet for tablet 931769e78f32467b8f27bacb286a7a60 (DEFAULT_TABLE table=default.Table_b9a40e813ba14ce89495f9edffeb4ee0 [id=68697340a8b046d79b795c15a68e1c6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:07.828683 10818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 931769e78f32467b8f27bacb286a7a60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:07.829257 10962 tablet_service.cc:1467] Processing CreateTablet for tablet 931769e78f32467b8f27bacb286a7a60 (DEFAULT_TABLE table=default.Table_b9a40e813ba14ce89495f9edffeb4ee0 [id=68697340a8b046d79b795c15a68e1c6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:07.831008 10962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 931769e78f32467b8f27bacb286a7a60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:07.843315 11109 raft_consensus.cc:482] T 6a5121b78e2d4643b99bd8717c6e93ca P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:07.843513 11152 raft_consensus.cc:482] T a8be01fe42894b96bbdd5ce2403ff1f4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:07.843799 11109 raft_consensus.cc:504] T 6a5121b78e2d4643b99bd8717c6e93ca P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:07.843920 11152 raft_consensus.cc:504] T a8be01fe42894b96bbdd5ce2403ff1f4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:07.845999 11109 leader_election.cc:288] T 6a5121b78e2d4643b99bd8717c6e93ca P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:07.846778 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a5121b78e2d4643b99bd8717c6e93ca" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:07.847087 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a5121b78e2d4643b99bd8717c6e93ca" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:07.848420 10619 leader_election.cc:302] T 6a5121b78e2d4643b99bd8717c6e93ca P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8850818e2ee42a082802d0095c08d21; no voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:07.848409 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8be01fe42894b96bbdd5ce2403ff1f4" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:07.849689 11109 raft_consensus.cc:2726] T 6a5121b78e2d4643b99bd8717c6e93ca P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 09:57:07.849753 10919 leader_election.cc:341] T a8be01fe42894b96bbdd5ce2403ff1f4 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:07.851117 11167 consensus_queue.cc:1035] T 6a5121b78e2d4643b99bd8717c6e93ca P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260925 09:57:07.855088 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8be01fe42894b96bbdd5ce2403ff1f4" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:07.856781 10918 leader_election.cc:341] T a8be01fe42894b96bbdd5ce2403ff1f4 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:07.857290 10918 leader_election.cc:302] T a8be01fe42894b96bbdd5ce2403ff1f4 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:07.858630 11169 raft_consensus.cc:2726] T a8be01fe42894b96bbdd5ce2403ff1f4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:07.859143 11084 tablet_bootstrap.cc:492] T e3b6b985a76b459dbd1de42ea20d3856 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:07.859622 11084 ts_tablet_manager.cc:1397] T e3b6b985a76b459dbd1de42ea20d3856 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.068s user 0.005s sys 0.006s
I20260925 09:57:07.859560 11152 leader_election.cc:288] T a8be01fe42894b96bbdd5ce2403ff1f4 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:07.860960 10815 tablet_service.cc:1467] Processing CreateTablet for tablet 59f0c6489a33433899bb6e62088121b6 (DEFAULT_TABLE table=default.Table_3e77b19accd1463cb978e456128a53ef [id=64032d286bf94c4cbdddde18adfb739e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:07.827481 11082 raft_consensus.cc:374] T 1969ac471b6f471ebe8e048a96c22837 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:07.876025 10393 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 840faf042d8749bd958ce1de0207a734 (table default.Table_faa21794a80c407f800038e583c08f16 [id=800eba46046443a7ad196808dec4cdc0]). Expected version 2 got 0
I20260925 09:57:07.876178 11082 raft_consensus.cc:729] T 1969ac471b6f471ebe8e048a96c22837 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:07.876742 10392 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet d8d6ac842c8040659c2589049eb21c5f
I20260925 09:57:07.879277 10817 tablet_service.cc:1514] Processing DeleteTablet for tablet d8d6ac842c8040659c2589049eb21c5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:07 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:07.879606 10963 tablet_service.cc:1514] Processing DeleteTablet for tablet d8d6ac842c8040659c2589049eb21c5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:07 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:07.880036 11098 tablet_replica.cc:330] T d8d6ac842c8040659c2589049eb21c5f P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:07.880654 11096 tablet_replica.cc:330] T d8d6ac842c8040659c2589049eb21c5f P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:07.880810 11098 raft_consensus.cc:2229] T d8d6ac842c8040659c2589049eb21c5f P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:07.881289 11098 raft_consensus.cc:2258] T d8d6ac842c8040659c2589049eb21c5f P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:07.881464 11096 raft_consensus.cc:2229] T d8d6ac842c8040659c2589049eb21c5f P be97ab5d75934a648d4ab76341193ec0 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:07.882252 11096 pending_rounds.cc:62] T d8d6ac842c8040659c2589049eb21c5f P be97ab5d75934a648d4ab76341193ec0: Trying to abort 1 pending ops.
I20260925 09:57:07.882484 11096 pending_rounds.cc:68] T d8d6ac842c8040659c2589049eb21c5f P be97ab5d75934a648d4ab76341193ec0: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192612486025216 op_type: NO_OP noop_request { }
I20260925 09:57:07.882969 10815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59f0c6489a33433899bb6e62088121b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:07.882920 11096 raft_consensus.cc:2866] T d8d6ac842c8040659c2589049eb21c5f P be97ab5d75934a648d4ab76341193ec0 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:57:07.883245 11096 raft_consensus.cc:2258] T d8d6ac842c8040659c2589049eb21c5f P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:07.885710 11096 ts_tablet_manager.cc:1905] T d8d6ac842c8040659c2589049eb21c5f P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:07.883172 11098 ts_tablet_manager.cc:1905] T d8d6ac842c8040659c2589049eb21c5f P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:07.887882 10665 tablet_service.cc:1514] Processing DeleteTablet for tablet d8d6ac842c8040659c2589049eb21c5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:07 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:07.888703 11095 tablet_replica.cc:330] T d8d6ac842c8040659c2589049eb21c5f P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:07.889598 11095 raft_consensus.cc:2229] T d8d6ac842c8040659c2589049eb21c5f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:07.890041 11095 raft_consensus.cc:2258] T d8d6ac842c8040659c2589049eb21c5f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:07.892179 11095 ts_tablet_manager.cc:1905] T d8d6ac842c8040659c2589049eb21c5f P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:07.894416 10817 tablet_service.cc:1514] Processing DeleteTablet for tablet bd89e13e0c534822b3ce0391fff9e3c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:07 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:07.900031 11082 consensus_queue.cc:260] T 1969ac471b6f471ebe8e048a96c22837 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:07.901413 10665 tablet_service.cc:1514] Processing DeleteTablet for tablet bd89e13e0c534822b3ce0391fff9e3c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:07 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:07.862013 11084 raft_consensus.cc:348] T e3b6b985a76b459dbd1de42ea20d3856 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:07.904074 11084 raft_consensus.cc:374] T e3b6b985a76b459dbd1de42ea20d3856 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:07.904577 11084 raft_consensus.cc:729] T e3b6b985a76b459dbd1de42ea20d3856 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:07.907307 11084 consensus_queue.cc:260] T e3b6b985a76b459dbd1de42ea20d3856 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:07.909654 11085 tablet_bootstrap.cc:492] T a8be01fe42894b96bbdd5ce2403ff1f4 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:07.910048 11085 ts_tablet_manager.cc:1397] T a8be01fe42894b96bbdd5ce2403ff1f4 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.267s user 0.005s sys 0.004s
I20260925 09:57:07.911468 11084 ts_tablet_manager.cc:1428] T e3b6b985a76b459dbd1de42ea20d3856 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.052s user 0.007s sys 0.000s
I20260925 09:57:07.911644 11082 ts_tablet_manager.cc:1428] T 1969ac471b6f471ebe8e048a96c22837 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.109s user 0.008s sys 0.000s
I20260925 09:57:07.912297 11084 tablet_bootstrap.cc:492] T 59f0c6489a33433899bb6e62088121b6 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:07.918612 11084 tablet_bootstrap.cc:654] T 59f0c6489a33433899bb6e62088121b6 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:07.939167 11082 tablet_bootstrap.cc:492] T 8aae639695f44d8c8a61b3ea42ab6ac8 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:07.974943 11082 tablet_bootstrap.cc:654] T 8aae639695f44d8c8a61b3ea42ab6ac8 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:07.976827 11180 tablet.cc:1722] T 840faf042d8749bd958ce1de0207a734 P be97ab5d75934a648d4ab76341193ec0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 09:57:08.002452 11186 tablet.cc:1722] T 840faf042d8749bd958ce1de0207a734 P b8850818e2ee42a082802d0095c08d21: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 09:57:08.018221 11082 tablet_bootstrap.cc:492] T 8aae639695f44d8c8a61b3ea42ab6ac8 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:08.018635 11082 ts_tablet_manager.cc:1397] T 8aae639695f44d8c8a61b3ea42ab6ac8 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.082s user 0.010s sys 0.005s
I20260925 09:57:08.031520 10388 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_2bc55a20487a41b182135205d1a4b65d" } modify_external_catalogs: true
I20260925 09:57:08.032673 10388 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_2bc55a20487a41b182135205d1a4b65d" } modify_external_catalogs: true
I20260925 09:57:08.029985 11082 raft_consensus.cc:348] T 8aae639695f44d8c8a61b3ea42ab6ac8 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.033447 11082 raft_consensus.cc:374] T 8aae639695f44d8c8a61b3ea42ab6ac8 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:08.033742 11082 raft_consensus.cc:729] T 8aae639695f44d8c8a61b3ea42ab6ac8 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:08.034011 10385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_968581c68bef42468bbdfed05e8dd9ca"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:08.034581 11082 consensus_queue.cc:260] T 8aae639695f44d8c8a61b3ea42ab6ac8 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.036892 11082 ts_tablet_manager.cc:1428] T 8aae639695f44d8c8a61b3ea42ab6ac8 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.018s user 0.006s sys 0.001s
I20260925 09:57:08.037611 11082 tablet_bootstrap.cc:492] T 4be3f6e81ddf418fabc7b694a2e62f35 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
W20260925 09:57:08.041208 10385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_968581c68bef42468bbdfed05e8dd9ca in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:08.049299 11096 ts_tablet_manager.cc:1918] T d8d6ac842c8040659c2589049eb21c5f P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:08.049710 11096 log.cc:1196] T d8d6ac842c8040659c2589049eb21c5f P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/d8d6ac842c8040659c2589049eb21c5f
I20260925 09:57:08.039141 11188 tablet.cc:1722] T 840faf042d8749bd958ce1de0207a734 P c43461c4ee854ae1ac52b873ccada318: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 09:57:08.050426 11096 ts_tablet_manager.cc:1939] T d8d6ac842c8040659c2589049eb21c5f P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:08.051424 11095 ts_tablet_manager.cc:1918] T d8d6ac842c8040659c2589049eb21c5f P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:08.051821 11095 log.cc:1196] T d8d6ac842c8040659c2589049eb21c5f P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/d8d6ac842c8040659c2589049eb21c5f
I20260925 09:57:08.052510 11095 ts_tablet_manager.cc:1939] T d8d6ac842c8040659c2589049eb21c5f P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:08.053182 11084 tablet_bootstrap.cc:492] T 59f0c6489a33433899bb6e62088121b6 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:08.053354 11098 ts_tablet_manager.cc:1918] T d8d6ac842c8040659c2589049eb21c5f P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:08.053718 11084 ts_tablet_manager.cc:1397] T 59f0c6489a33433899bb6e62088121b6 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.142s user 0.009s sys 0.018s
I20260925 09:57:08.053782 11098 log.cc:1196] T d8d6ac842c8040659c2589049eb21c5f P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/d8d6ac842c8040659c2589049eb21c5f
I20260925 09:57:08.054389 11098 ts_tablet_manager.cc:1939] T d8d6ac842c8040659c2589049eb21c5f P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:08.056573 10393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_c6db4ef50ad442fcb28e2646fd6b7571"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:08.057231 11084 raft_consensus.cc:348] T 59f0c6489a33433899bb6e62088121b6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.057821 11084 raft_consensus.cc:374] T 59f0c6489a33433899bb6e62088121b6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:08.058066 11084 raft_consensus.cc:729] T 59f0c6489a33433899bb6e62088121b6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
W20260925 09:57:08.058239 10393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c6db4ef50ad442fcb28e2646fd6b7571 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:08.058619 11084 consensus_queue.cc:260] T 59f0c6489a33433899bb6e62088121b6 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.059921 11109 raft_consensus.cc:482] T 1969ac471b6f471ebe8e048a96c22837 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:08.060518 11109 raft_consensus.cc:504] T 1969ac471b6f471ebe8e048a96c22837 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } }
I20260925 09:57:08.062434 11109 leader_election.cc:288] T 1969ac471b6f471ebe8e048a96c22837 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:08.063953 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1969ac471b6f471ebe8e048a96c22837" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:08.064654 10984 raft_consensus.cc:2379] T 1969ac471b6f471ebe8e048a96c22837 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b8850818e2ee42a082802d0095c08d21 in current term 1: Already voted for candidate c43461c4ee854ae1ac52b873ccada318 in this term.
I20260925 09:57:08.064806 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1969ac471b6f471ebe8e048a96c22837" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:08.066140 10621 leader_election.cc:302] T 1969ac471b6f471ebe8e048a96c22837 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8850818e2ee42a082802d0095c08d21; no voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:08.067075 11109 raft_consensus.cc:3037] T 1969ac471b6f471ebe8e048a96c22837 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:08.072018 11084 ts_tablet_manager.cc:1428] T 59f0c6489a33433899bb6e62088121b6 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.018s user 0.006s sys 0.000s
I20260925 09:57:08.074019 11084 tablet_bootstrap.cc:492] T 931769e78f32467b8f27bacb286a7a60 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:07.919198 11085 raft_consensus.cc:348] T a8be01fe42894b96bbdd5ce2403ff1f4 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.077971 11085 raft_consensus.cc:374] T a8be01fe42894b96bbdd5ce2403ff1f4 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:08.078277 11085 raft_consensus.cc:729] T a8be01fe42894b96bbdd5ce2403ff1f4 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:08.079406 11085 consensus_queue.cc:260] T a8be01fe42894b96bbdd5ce2403ff1f4 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.087668 10390 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_fe84e50a7371410d81b423c4324b17f2" } modify_external_catalogs: true
I20260925 09:57:08.088260 10390 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_fe84e50a7371410d81b423c4324b17f2" } modify_external_catalogs: true
I20260925 09:57:08.097406 11084 tablet_bootstrap.cc:654] T 931769e78f32467b8f27bacb286a7a60 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:08.097853 10386 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8c13a2f288b14d03a3ca4fdd4889cbad has no consensus state
I20260925 09:57:08.101755 11082 tablet_bootstrap.cc:654] T 4be3f6e81ddf418fabc7b694a2e62f35 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:08.135550 11169 raft_consensus.cc:482] T 59f0c6489a33433899bb6e62088121b6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:08.136222 11169 raft_consensus.cc:504] T 59f0c6489a33433899bb6e62088121b6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.140669 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59f0c6489a33433899bb6e62088121b6" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:08.142517 10919 leader_election.cc:341] T 59f0c6489a33433899bb6e62088121b6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:08.147069 11169 leader_election.cc:288] T 59f0c6489a33433899bb6e62088121b6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:08.150818 11085 ts_tablet_manager.cc:1428] T a8be01fe42894b96bbdd5ce2403ff1f4 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.240s user 0.008s sys 0.000s
I20260925 09:57:08.154737 11109 raft_consensus.cc:2726] T 1969ac471b6f471ebe8e048a96c22837 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:08.155452 10386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_0fe949ec62d547c9a9acbf96b374edcb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:08.172032 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet d8d6ac842c8040659c2589049eb21c5f (table default.Table_000f52fd1e214fa1931ef3743ce76250 [id=f67a73ffb5574153bc3b52c5b32ecabd]) successfully deleted
I20260925 09:57:08.175336 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59f0c6489a33433899bb6e62088121b6" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:08.176924 10918 leader_election.cc:341] T 59f0c6489a33433899bb6e62088121b6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:08.177212 11095 tablet_replica.cc:330] T bd89e13e0c534822b3ce0391fff9e3c9 P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:08.177333 10918 leader_election.cc:302] T 59f0c6489a33433899bb6e62088121b6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:08.178031 11095 raft_consensus.cc:2229] T bd89e13e0c534822b3ce0391fff9e3c9 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:08.178623 11095 raft_consensus.cc:2258] T bd89e13e0c534822b3ce0391fff9e3c9 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:08.178642 11169 raft_consensus.cc:2726] T 59f0c6489a33433899bb6e62088121b6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:08.177388 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet d8d6ac842c8040659c2589049eb21c5f (table default.Table_000f52fd1e214fa1931ef3743ce76250 [id=f67a73ffb5574153bc3b52c5b32ecabd]) successfully deleted
W20260925 09:57:08.179690 10386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0fe949ec62d547c9a9acbf96b374edcb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:08.182107 11085 tablet_bootstrap.cc:492] T e3b6b985a76b459dbd1de42ea20d3856 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:08.189947 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet d8d6ac842c8040659c2589049eb21c5f (table default.Table_000f52fd1e214fa1931ef3743ce76250 [id=f67a73ffb5574153bc3b52c5b32ecabd]) successfully deleted
I20260925 09:57:08.193723 11084 tablet_bootstrap.cc:492] T 931769e78f32467b8f27bacb286a7a60 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:08.194226 11084 ts_tablet_manager.cc:1397] T 931769e78f32467b8f27bacb286a7a60 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.120s user 0.019s sys 0.024s
I20260925 09:57:08.194396 11180 tablet.cc:1717] T 840faf042d8749bd958ce1de0207a734 P be97ab5d75934a648d4ab76341193ec0: Skipping requested alter to schema version 2, tablet already version 2
I20260925 09:57:08.197307 11186 tablet.cc:1717] T 840faf042d8749bd958ce1de0207a734 P b8850818e2ee42a082802d0095c08d21: Skipping requested alter to schema version 2, tablet already version 2
I20260925 09:57:08.196653 11084 raft_consensus.cc:348] T 931769e78f32467b8f27bacb286a7a60 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.199702 11084 raft_consensus.cc:374] T 931769e78f32467b8f27bacb286a7a60 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:08.199965 11084 raft_consensus.cc:729] T 931769e78f32467b8f27bacb286a7a60 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:08.200549 11084 consensus_queue.cc:260] T 931769e78f32467b8f27bacb286a7a60 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:08.200141 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet bd89e13e0c534822b3ce0391fff9e3c9 because the tablet was not found. No further retry: Not found: Tablet not found: bd89e13e0c534822b3ce0391fff9e3c9
I20260925 09:57:08.210351 11095 ts_tablet_manager.cc:1905] T bd89e13e0c534822b3ce0391fff9e3c9 P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:08.210379 10387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_8dfe0230298f4ff9b7e34161a9e90f53"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:08.212049 10387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8dfe0230298f4ff9b7e34161a9e90f53 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:08.215261 11188 tablet.cc:1717] T 840faf042d8749bd958ce1de0207a734 P c43461c4ee854ae1ac52b873ccada318: Skipping requested alter to schema version 2, tablet already version 2
I20260925 09:57:08.215934 11085 tablet_bootstrap.cc:654] T e3b6b985a76b459dbd1de42ea20d3856 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:08.219310 11084 ts_tablet_manager.cc:1428] T 931769e78f32467b8f27bacb286a7a60 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.025s user 0.005s sys 0.001s
I20260925 09:57:08.228996 11084 tablet_bootstrap.cc:492] T 04fafb9dcf274806b11b2598e73853e9 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:08.233750 11109 raft_consensus.cc:482] T 8aae639695f44d8c8a61b3ea42ab6ac8 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:08.234269 11109 raft_consensus.cc:504] T 8aae639695f44d8c8a61b3ea42ab6ac8 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.238404 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aae639695f44d8c8a61b3ea42ab6ac8" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:08.241120 11109 leader_election.cc:288] T 8aae639695f44d8c8a61b3ea42ab6ac8 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:08.242005 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aae639695f44d8c8a61b3ea42ab6ac8" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:08.242893 10838 raft_consensus.cc:2379] T 8aae639695f44d8c8a61b3ea42ab6ac8 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b8850818e2ee42a082802d0095c08d21 in current term 1: Already voted for candidate be97ab5d75934a648d4ab76341193ec0 in this term.
I20260925 09:57:08.243992 10621 leader_election.cc:302] T 8aae639695f44d8c8a61b3ea42ab6ac8 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8850818e2ee42a082802d0095c08d21; no voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:08.244882 11084 tablet_bootstrap.cc:654] T 04fafb9dcf274806b11b2598e73853e9 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:08.245059 11109 raft_consensus.cc:3037] T 8aae639695f44d8c8a61b3ea42ab6ac8 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:08.244035 11082 tablet_bootstrap.cc:492] T 4be3f6e81ddf418fabc7b694a2e62f35 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:08.255955 11082 ts_tablet_manager.cc:1397] T 4be3f6e81ddf418fabc7b694a2e62f35 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.218s user 0.012s sys 0.038s
I20260925 09:57:08.258729 11082 raft_consensus.cc:348] T 4be3f6e81ddf418fabc7b694a2e62f35 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.263258 11082 raft_consensus.cc:374] T 4be3f6e81ddf418fabc7b694a2e62f35 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:08.263597 11082 raft_consensus.cc:729] T 4be3f6e81ddf418fabc7b694a2e62f35 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:08.264360 11082 consensus_queue.cc:260] T 4be3f6e81ddf418fabc7b694a2e62f35 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.274339 11157 raft_consensus.cc:482] T a8be01fe42894b96bbdd5ce2403ff1f4 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:08.275002 11157 raft_consensus.cc:504] T a8be01fe42894b96bbdd5ce2403ff1f4 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.278496 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8be01fe42894b96bbdd5ce2403ff1f4" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:08.279501 11157 leader_election.cc:288] T a8be01fe42894b96bbdd5ce2403ff1f4 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:08.281122 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8be01fe42894b96bbdd5ce2403ff1f4" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:08.281654 11082 ts_tablet_manager.cc:1428] T 4be3f6e81ddf418fabc7b694a2e62f35 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.025s user 0.005s sys 0.005s
I20260925 09:57:08.279095 10984 raft_consensus.cc:2454] T a8be01fe42894b96bbdd5ce2403ff1f4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
I20260925 09:57:08.282684 10772 leader_election.cc:302] T a8be01fe42894b96bbdd5ce2403ff1f4 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters:
W20260925 09:57:08.283710 10775 leader_election.cc:341] T a8be01fe42894b96bbdd5ce2403ff1f4 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:08.285404 11157 raft_consensus.cc:2781] T a8be01fe42894b96bbdd5ce2403ff1f4 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:08.285706 11157 raft_consensus.cc:482] T a8be01fe42894b96bbdd5ce2403ff1f4 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:08.286103 11084 tablet_bootstrap.cc:492] T 04fafb9dcf274806b11b2598e73853e9 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:08.286475 11084 ts_tablet_manager.cc:1397] T 04fafb9dcf274806b11b2598e73853e9 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.058s user 0.010s sys 0.000s
I20260925 09:57:08.285982 11157 raft_consensus.cc:3037] T a8be01fe42894b96bbdd5ce2403ff1f4 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:08.289089 11082 tablet_bootstrap.cc:492] T 6b03e5aad59e46a6b0e43b9cb36278d7 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:08.288956 11084 raft_consensus.cc:348] T 04fafb9dcf274806b11b2598e73853e9 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.289574 11084 raft_consensus.cc:374] T 04fafb9dcf274806b11b2598e73853e9 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:08.289825 11084 raft_consensus.cc:729] T 04fafb9dcf274806b11b2598e73853e9 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:08.290364 11084 consensus_queue.cc:260] T 04fafb9dcf274806b11b2598e73853e9 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.302894 10387 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_968581c68bef42468bbdfed05e8dd9ca" } new_table_name: "default.Table_3b9695593bd6426e8932f7b11f01d1ac" modify_external_catalogs: true
I20260925 09:57:08.304373 11082 tablet_bootstrap.cc:654] T 6b03e5aad59e46a6b0e43b9cb36278d7 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:08.308724 11084 ts_tablet_manager.cc:1428] T 04fafb9dcf274806b11b2598e73853e9 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.022s user 0.006s sys 0.000s
I20260925 09:57:08.323982 10392 catalog_manager.cc:5495] T a0976e0c01234b38a7ab934dc1b289c4 P c43461c4ee854ae1ac52b873ccada318 reported cstate change: term changed from 0 to 1, leader changed from <none> to c43461c4ee854ae1ac52b873ccada318 (127.10.9.2). New cstate: current_term: 1 leader_uuid: "c43461c4ee854ae1ac52b873ccada318" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: HEALTHY } } }
I20260925 09:57:08.325023 10961 tablet_service.cc:1467] Processing CreateTablet for tablet d8be238061c146eb96d17a22c1b2104a (DEFAULT_TABLE table=default.Table_2bc55a20487a41b182135205d1a4b65d [id=f97ee2a971ab46abb4cbb37169fb309d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:08.325028 10666 tablet_service.cc:1467] Processing CreateTablet for tablet d8be238061c146eb96d17a22c1b2104a (DEFAULT_TABLE table=default.Table_2bc55a20487a41b182135205d1a4b65d [id=f97ee2a971ab46abb4cbb37169fb309d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:08.325611 10392 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 4be3f6e81ddf418fabc7b694a2e62f35 (table default.Table_1e7379199917430eb9e027fc7bb2a4f9 [id=c2d0336fd84c4dbe95472f458b813fe6]). Expected version 1 got 0
I20260925 09:57:08.326215 10961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8be238061c146eb96d17a22c1b2104a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:08.326261 10392 catalog_manager.cc:5495] T 1969ac471b6f471ebe8e048a96c22837 P c43461c4ee854ae1ac52b873ccada318 reported cstate change: term changed from 0 to 1, leader changed from <none> to c43461c4ee854ae1ac52b873ccada318 (127.10.9.2). New cstate: current_term: 1 leader_uuid: "c43461c4ee854ae1ac52b873ccada318" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: HEALTHY } } }
I20260925 09:57:08.329185 10667 tablet_service.cc:1467] Processing CreateTablet for tablet 8c13a2f288b14d03a3ca4fdd4889cbad (DEFAULT_TABLE table=default.Table_ecec3b643d1c4b76b700090f3a3cab06 [id=52fd37f9b470404ebea6e54191436b12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:08.329828 10960 tablet_service.cc:1467] Processing CreateTablet for tablet aad3cb5e33f64cb885d51f4ea089307b (DEFAULT_TABLE table=default.Table_18ecd2dd5b4e472cb12e72ce8bce84f9 [id=9d2f4f9f46394332bec9f8beda0d9595]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:08.330979 10960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aad3cb5e33f64cb885d51f4ea089307b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:08.330936 10664 tablet_service.cc:1467] Processing CreateTablet for tablet aad3cb5e33f64cb885d51f4ea089307b (DEFAULT_TABLE table=default.Table_18ecd2dd5b4e472cb12e72ce8bce84f9 [id=9d2f4f9f46394332bec9f8beda0d9595]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:08.333887 10668 tablet_service.cc:1467] Processing CreateTablet for tablet 0a2bbe141e564a96be05de86f2993234 (DEFAULT_TABLE table=default.Table_8f64c62cd63d43ce89d446956c0883e3 [id=aed8a2ddc51545ad9cec4d272dcf3daf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:08.335464 11084 tablet_bootstrap.cc:492] T d48d4517c13049389c5b2ba1ab7b14bc P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:08.335608 10665 tablet_service.cc:1467] Processing CreateTablet for tablet e57065f2bb224a56ad1c3518a0c375da (DEFAULT_TABLE table=default.Table_ce8527f03cfc4ae29d76f64597e0b075 [id=7889081642604700bbc6d253ab105260]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:08.338929 10815 tablet_service.cc:1467] Processing CreateTablet for tablet d8be238061c146eb96d17a22c1b2104a (DEFAULT_TABLE table=default.Table_2bc55a20487a41b182135205d1a4b65d [id=f97ee2a971ab46abb4cbb37169fb309d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:08.341606 10962 tablet_service.cc:1467] Processing CreateTablet for tablet 8c13a2f288b14d03a3ca4fdd4889cbad (DEFAULT_TABLE table=default.Table_ecec3b643d1c4b76b700090f3a3cab06 [id=52fd37f9b470404ebea6e54191436b12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:08.343376 10392 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 1969ac471b6f471ebe8e048a96c22837 (table default.Table_4e09a615fb82487baa4783a5f6e2181e [id=f17a11d223204599afb24e2e9d1d9d9d]). Expected version 3 got 0
I20260925 09:57:08.344142 10392 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 6a5121b78e2d4643b99bd8717c6e93ca (table default.Table_5ff0f5bde5954217976f2876e76adaaa [id=aa7e7f5af2724af797170e1f7429d76a]). Expected version 1 got 0
I20260925 09:57:08.344081 10959 tablet_service.cc:1467] Processing CreateTablet for tablet 0a2bbe141e564a96be05de86f2993234 (DEFAULT_TABLE table=default.Table_8f64c62cd63d43ce89d446956c0883e3 [id=aed8a2ddc51545ad9cec4d272dcf3daf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:08.344908 10392 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 840faf042d8749bd958ce1de0207a734 (table default.Table_faa21794a80c407f800038e583c08f16 [id=800eba46046443a7ad196808dec4cdc0]). Expected version 2 got 0
I20260925 09:57:08.343219 10963 tablet_service.cc:1467] Processing CreateTablet for tablet e57065f2bb224a56ad1c3518a0c375da (DEFAULT_TABLE table=default.Table_ce8527f03cfc4ae29d76f64597e0b075 [id=7889081642604700bbc6d253ab105260]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:08.350109 11109 raft_consensus.cc:2726] T 8aae639695f44d8c8a61b3ea42ab6ac8 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:08.339166 10819 tablet_service.cc:1467] Processing CreateTablet for tablet aad3cb5e33f64cb885d51f4ea089307b (DEFAULT_TABLE table=default.Table_18ecd2dd5b4e472cb12e72ce8bce84f9 [id=9d2f4f9f46394332bec9f8beda0d9595]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:08.344085 10818 tablet_service.cc:1467] Processing CreateTablet for tablet 8c13a2f288b14d03a3ca4fdd4889cbad (DEFAULT_TABLE table=default.Table_ecec3b643d1c4b76b700090f3a3cab06 [id=52fd37f9b470404ebea6e54191436b12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:08.345238 10816 tablet_service.cc:1467] Processing CreateTablet for tablet e57065f2bb224a56ad1c3518a0c375da (DEFAULT_TABLE table=default.Table_ce8527f03cfc4ae29d76f64597e0b075 [id=7889081642604700bbc6d253ab105260]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:08.345844 10814 tablet_service.cc:1467] Processing CreateTablet for tablet 0a2bbe141e564a96be05de86f2993234 (DEFAULT_TABLE table=default.Table_8f64c62cd63d43ce89d446956c0883e3 [id=aed8a2ddc51545ad9cec4d272dcf3daf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:08.349419 10815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8be238061c146eb96d17a22c1b2104a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:08.361814 10666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8be238061c146eb96d17a22c1b2104a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:08.363628 10387 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e48d86c20e9e4db990c23359ff1d1a6d has no consensus state
I20260925 09:57:08.370167 10664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aad3cb5e33f64cb885d51f4ea089307b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:08.371897 10667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c13a2f288b14d03a3ca4fdd4889cbad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:08.376116 10816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e57065f2bb224a56ad1c3518a0c375da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:08.380378 10818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c13a2f288b14d03a3ca4fdd4889cbad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:08.381618 11188 tablet.cc:1722] T 6a5121b78e2d4643b99bd8717c6e93ca P c43461c4ee854ae1ac52b873ccada318: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:08.383270 10819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aad3cb5e33f64cb885d51f4ea089307b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:08.347303 10813 tablet_service.cc:1467] Processing CreateTablet for tablet 26b7c37959d042b48234acc79290936f (DEFAULT_TABLE table=default.Table_fe84e50a7371410d81b423c4324b17f2 [id=2b43210a3c1b4f29a1b0f5fdc7819ef2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:08.387547 10958 tablet_service.cc:1467] Processing CreateTablet for tablet 26b7c37959d042b48234acc79290936f (DEFAULT_TABLE table=default.Table_fe84e50a7371410d81b423c4324b17f2 [id=2b43210a3c1b4f29a1b0f5fdc7819ef2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:08.390239 11186 tablet.cc:1722] T 6a5121b78e2d4643b99bd8717c6e93ca P b8850818e2ee42a082802d0095c08d21: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:08.391983 10663 tablet_service.cc:1467] Processing CreateTablet for tablet 26b7c37959d042b48234acc79290936f (DEFAULT_TABLE table=default.Table_fe84e50a7371410d81b423c4324b17f2 [id=2b43210a3c1b4f29a1b0f5fdc7819ef2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:08.393638 10665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e57065f2bb224a56ad1c3518a0c375da. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:08.398675 10378 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 1969ac471b6f471ebe8e048a96c22837 (table default.Table_4e09a615fb82487baa4783a5f6e2181e [id=f17a11d223204599afb24e2e9d1d9d9d], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet 1969ac471b6f471ebe8e048a96c22837 consensus state has no leader
I20260925 09:57:08.399335 10958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26b7c37959d042b48234acc79290936f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:08.400270 10388 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet d8be238061c146eb96d17a22c1b2104a
I20260925 09:57:08.401430 11084 tablet_bootstrap.cc:654] T d48d4517c13049389c5b2ba1ab7b14bc P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:08.401660 10962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c13a2f288b14d03a3ca4fdd4889cbad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:08.408263 10963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e57065f2bb224a56ad1c3518a0c375da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:08.411795 10959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a2bbe141e564a96be05de86f2993234. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:08.417765 10957 tablet_service.cc:1514] Processing DeleteTablet for tablet d8be238061c146eb96d17a22c1b2104a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:08 UTC) from {username='slave'} at 127.0.0.1:53560
W20260925 09:57:08.419296 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet d8be238061c146eb96d17a22c1b2104a because the tablet was not found. No further retry: Not found: Tablet not found: d8be238061c146eb96d17a22c1b2104a
I20260925 09:57:08.420475 10661 tablet_service.cc:1514] Processing DeleteTablet for tablet d8be238061c146eb96d17a22c1b2104a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:08 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:08.421423 10663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26b7c37959d042b48234acc79290936f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:08.423671 11167 raft_consensus.cc:482] T 4be3f6e81ddf418fabc7b694a2e62f35 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:08.430696 11167 raft_consensus.cc:504] T 4be3f6e81ddf418fabc7b694a2e62f35 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.432915 11167 leader_election.cc:288] T 4be3f6e81ddf418fabc7b694a2e62f35 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:08.428524 10814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a2bbe141e564a96be05de86f2993234. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:08.435638 10813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26b7c37959d042b48234acc79290936f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:08.435582 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4be3f6e81ddf418fabc7b694a2e62f35" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:08.435950 11084 tablet_bootstrap.cc:492] T d48d4517c13049389c5b2ba1ab7b14bc P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:08.436203 10984 raft_consensus.cc:2454] T 4be3f6e81ddf418fabc7b694a2e62f35 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
I20260925 09:57:08.436345 11084 ts_tablet_manager.cc:1397] T d48d4517c13049389c5b2ba1ab7b14bc P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.101s user 0.021s sys 0.008s
I20260925 09:57:08.437424 10772 leader_election.cc:302] T 4be3f6e81ddf418fabc7b694a2e62f35 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:08.438880 11084 raft_consensus.cc:348] T d48d4517c13049389c5b2ba1ab7b14bc P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.439566 11084 raft_consensus.cc:374] T d48d4517c13049389c5b2ba1ab7b14bc P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:08.439821 11084 raft_consensus.cc:729] T d48d4517c13049389c5b2ba1ab7b14bc P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:08.440471 11084 consensus_queue.cc:260] T d48d4517c13049389c5b2ba1ab7b14bc P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.443552 10388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_9f1a5af3d45e4ed1b81a8484a2c336fb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:08.446611 10388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9f1a5af3d45e4ed1b81a8484a2c336fb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:08.452708 10817 tablet_service.cc:1514] Processing DeleteTablet for tablet d8be238061c146eb96d17a22c1b2104a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:08 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:08.454109 11084 ts_tablet_manager.cc:1428] T d48d4517c13049389c5b2ba1ab7b14bc P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.017s user 0.003s sys 0.002s
I20260925 09:57:08.454558 11095 ts_tablet_manager.cc:1918] T bd89e13e0c534822b3ce0391fff9e3c9 P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:08.454906 11095 log.cc:1196] T bd89e13e0c534822b3ce0391fff9e3c9 P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/bd89e13e0c534822b3ce0391fff9e3c9
I20260925 09:57:08.455530 11095 ts_tablet_manager.cc:1939] T bd89e13e0c534822b3ce0391fff9e3c9 P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:08.457300 11084 tablet_bootstrap.cc:492] T aad3cb5e33f64cb885d51f4ea089307b P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
W20260925 09:57:08.458869 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet d8be238061c146eb96d17a22c1b2104a because the tablet was not found. No further retry: Not found: Tablet not found: d8be238061c146eb96d17a22c1b2104a
I20260925 09:57:08.460175 11122 raft_consensus.cc:2781] T 4be3f6e81ddf418fabc7b694a2e62f35 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:08.461838 11122 raft_consensus.cc:482] T 4be3f6e81ddf418fabc7b694a2e62f35 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:08.463822 10668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a2bbe141e564a96be05de86f2993234. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:08.466590 11180 tablet.cc:1722] T 6a5121b78e2d4643b99bd8717c6e93ca P be97ab5d75934a648d4ab76341193ec0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:08.463459 11122 raft_consensus.cc:3037] T 4be3f6e81ddf418fabc7b694a2e62f35 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:08.476929 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet bd89e13e0c534822b3ce0391fff9e3c9 (table default.Table_aefaa3aacf0e43e1b32aa184267892d4 [id=e6ae2bf4301141f790fefe7bc560029a]) successfully deleted
I20260925 09:57:08.477237 10386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_beb33698ec744e42b4f6ac2fdd8fe2b9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:08.477927 10387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_861e5470a3b548419a9d96e59a1f9b0e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:08.479005 10386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_beb33698ec744e42b4f6ac2fdd8fe2b9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 09:57:08.479482 10387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_861e5470a3b548419a9d96e59a1f9b0e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:08.490585 10393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_4c02305911f84ebb95187deaf905c14c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:08.492206 10393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4c02305911f84ebb95187deaf905c14c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:08.496078 11134 raft_consensus.cc:482] T d48d4517c13049389c5b2ba1ab7b14bc P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:08.496570 11134 raft_consensus.cc:504] T d48d4517c13049389c5b2ba1ab7b14bc P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.487766 11157 raft_consensus.cc:504] T a8be01fe42894b96bbdd5ce2403ff1f4 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.506179 11157 leader_election.cc:288] T a8be01fe42894b96bbdd5ce2403ff1f4 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:08.507470 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8be01fe42894b96bbdd5ce2403ff1f4" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:08.508268 10984 raft_consensus.cc:3037] T a8be01fe42894b96bbdd5ce2403ff1f4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:08.513044 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d48d4517c13049389c5b2ba1ab7b14bc" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:08.516686 11134 leader_election.cc:288] T d48d4517c13049389c5b2ba1ab7b14bc P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
W20260925 09:57:08.524524 10918 leader_election.cc:341] T d48d4517c13049389c5b2ba1ab7b14bc P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:08.526773 11084 tablet_bootstrap.cc:654] T aad3cb5e33f64cb885d51f4ea089307b P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:08.526691 10390 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 26b7c37959d042b48234acc79290936f
I20260925 09:57:08.533078 10816 tablet_service.cc:1514] Processing DeleteTablet for tablet 26b7c37959d042b48234acc79290936f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:08 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:08.535303 11085 tablet_bootstrap.cc:492] T e3b6b985a76b459dbd1de42ea20d3856 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:08.536051 11085 ts_tablet_manager.cc:1397] T e3b6b985a76b459dbd1de42ea20d3856 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.354s user 0.008s sys 0.006s
W20260925 09:57:08.539784 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 26b7c37959d042b48234acc79290936f because the tablet was not found. No further retry: Not found: Tablet not found: 26b7c37959d042b48234acc79290936f
I20260925 09:57:08.550467 11085 raft_consensus.cc:348] T e3b6b985a76b459dbd1de42ea20d3856 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.551257 11085 raft_consensus.cc:374] T e3b6b985a76b459dbd1de42ea20d3856 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:08.551504 11134 consensus_queue.cc:1035] T 8aae639695f44d8c8a61b3ea42ab6ac8 P be97ab5d75934a648d4ab76341193ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:08.559567 11085 raft_consensus.cc:729] T e3b6b985a76b459dbd1de42ea20d3856 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
W20260925 09:57:08.552824 10775 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 128944us
I20260925 09:57:08.552963 11169 raft_consensus.cc:482] T 6b03e5aad59e46a6b0e43b9cb36278d7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:08.569407 11169 raft_consensus.cc:504] T 6b03e5aad59e46a6b0e43b9cb36278d7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.571291 11169 leader_election.cc:288] T 6b03e5aad59e46a6b0e43b9cb36278d7 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:08.573818 11085 consensus_queue.cc:260] T e3b6b985a76b459dbd1de42ea20d3856 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.580379 11169 raft_consensus.cc:482] T 04fafb9dcf274806b11b2598e73853e9 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:08.575875 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b03e5aad59e46a6b0e43b9cb36278d7" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:08.580785 11169 raft_consensus.cc:504] T 04fafb9dcf274806b11b2598e73853e9 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:08.581732 10918 leader_election.cc:341] T 6b03e5aad59e46a6b0e43b9cb36278d7 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:08.583137 11169 leader_election.cc:288] T 04fafb9dcf274806b11b2598e73853e9 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:08.584187 11122 raft_consensus.cc:504] T 4be3f6e81ddf418fabc7b694a2e62f35 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.586591 11122 leader_election.cc:288] T 4be3f6e81ddf418fabc7b694a2e62f35 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:08.587250 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04fafb9dcf274806b11b2598e73853e9" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:08.587442 10958 tablet_service.cc:1514] Processing DeleteTablet for tablet 26b7c37959d042b48234acc79290936f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:08 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:08.587801 11084 tablet_bootstrap.cc:492] T aad3cb5e33f64cb885d51f4ea089307b P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:08.588513 11084 ts_tablet_manager.cc:1397] T aad3cb5e33f64cb885d51f4ea089307b P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.131s user 0.011s sys 0.005s
I20260925 09:57:08.591064 11084 raft_consensus.cc:348] T aad3cb5e33f64cb885d51f4ea089307b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.591922 11084 raft_consensus.cc:374] T aad3cb5e33f64cb885d51f4ea089307b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:08.592228 11084 raft_consensus.cc:729] T aad3cb5e33f64cb885d51f4ea089307b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:08.592911 11084 consensus_queue.cc:260] T aad3cb5e33f64cb885d51f4ea089307b P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.597115 10390 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_c4ae6258da494589b65be6ede95f4a1d" } new_table_name: "default.Table_6ac888477dde46a4a748e2d8bbe5e0c2" modify_external_catalogs: true
W20260925 09:57:08.597536 10918 leader_election.cc:341] T 04fafb9dcf274806b11b2598e73853e9 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:08.602820 11084 ts_tablet_manager.cc:1428] T aad3cb5e33f64cb885d51f4ea089307b P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.014s user 0.006s sys 0.000s
I20260925 09:57:08.604055 11085 ts_tablet_manager.cc:1428] T e3b6b985a76b459dbd1de42ea20d3856 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.068s user 0.007s sys 0.000s
W20260925 09:57:08.606732 10377 catalog_manager.cc:4821] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 26b7c37959d042b48234acc79290936f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 26b7c37959d042b48234acc79290936f already in progress: creating tablet
I20260925 09:57:08.606769 10982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4be3f6e81ddf418fabc7b694a2e62f35" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:08.607308 10982 raft_consensus.cc:3037] T 4be3f6e81ddf418fabc7b694a2e62f35 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:08.608732 11084 tablet_bootstrap.cc:492] T 26b7c37959d042b48234acc79290936f P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:08.612249 10385 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_c6c124e548d3457fab9199a521cfa2c9" } modify_external_catalogs: true
I20260925 09:57:08.613011 10385 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_c6c124e548d3457fab9199a521cfa2c9" } modify_external_catalogs: true
I20260925 09:57:08.621627 11084 tablet_bootstrap.cc:654] T 26b7c37959d042b48234acc79290936f P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:08.624397 10984 raft_consensus.cc:2454] T a8be01fe42894b96bbdd5ce2403ff1f4 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:08.625558 10772 leader_election.cc:302] T a8be01fe42894b96bbdd5ce2403ff1f4 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:08.628793 11122 raft_consensus.cc:2781] T a8be01fe42894b96bbdd5ce2403ff1f4 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:08.629715 11122 raft_consensus.cc:686] T a8be01fe42894b96bbdd5ce2403ff1f4 P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Becoming Leader. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Running, Role: LEADER
I20260925 09:57:08.630949 11122 consensus_queue.cc:237] T a8be01fe42894b96bbdd5ce2403ff1f4 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.638918 11085 tablet_bootstrap.cc:492] T 6b03e5aad59e46a6b0e43b9cb36278d7 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:08.659581 10982 raft_consensus.cc:2454] T 4be3f6e81ddf418fabc7b694a2e62f35 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:08.661013 11134 raft_consensus.cc:482] T e3b6b985a76b459dbd1de42ea20d3856 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:08.661594 10772 leader_election.cc:302] T 4be3f6e81ddf418fabc7b694a2e62f35 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:08.661481 11134 raft_consensus.cc:504] T e3b6b985a76b459dbd1de42ea20d3856 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.662550 11122 raft_consensus.cc:2781] T 4be3f6e81ddf418fabc7b694a2e62f35 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:08.663533 11134 leader_election.cc:288] T e3b6b985a76b459dbd1de42ea20d3856 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:08.663766 11122 raft_consensus.cc:686] T 4be3f6e81ddf418fabc7b694a2e62f35 P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Becoming Leader. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Running, Role: LEADER
I20260925 09:57:08.664651 11122 consensus_queue.cc:237] T 4be3f6e81ddf418fabc7b694a2e62f35 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.665585 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3b6b985a76b459dbd1de42ea20d3856" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:08.665988 11084 tablet_bootstrap.cc:492] T 26b7c37959d042b48234acc79290936f P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:08.666198 10838 raft_consensus.cc:2454] T e3b6b985a76b459dbd1de42ea20d3856 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be97ab5d75934a648d4ab76341193ec0 in term 0.
I20260925 09:57:08.666375 11084 ts_tablet_manager.cc:1397] T 26b7c37959d042b48234acc79290936f P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.058s user 0.008s sys 0.012s
I20260925 09:57:08.667268 10918 leader_election.cc:302] T e3b6b985a76b459dbd1de42ea20d3856 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:08.668069 11134 raft_consensus.cc:2781] T e3b6b985a76b459dbd1de42ea20d3856 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:08.668394 11134 raft_consensus.cc:482] T e3b6b985a76b459dbd1de42ea20d3856 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:08.668699 11134 raft_consensus.cc:3037] T e3b6b985a76b459dbd1de42ea20d3856 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:08.679679 11084 raft_consensus.cc:348] T 26b7c37959d042b48234acc79290936f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.680354 11084 raft_consensus.cc:374] T 26b7c37959d042b48234acc79290936f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260925 09:57:08.681967 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet d8be238061c146eb96d17a22c1b2104a because the tablet was not found. No further retry: Not found: Tablet not found: d8be238061c146eb96d17a22c1b2104a
I20260925 09:57:08.684532 11085 tablet_bootstrap.cc:654] T 6b03e5aad59e46a6b0e43b9cb36278d7 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:08.691953 11084 raft_consensus.cc:729] T 26b7c37959d042b48234acc79290936f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:08.694406 11084 consensus_queue.cc:260] T 26b7c37959d042b48234acc79290936f P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.706449 11084 ts_tablet_manager.cc:1428] T 26b7c37959d042b48234acc79290936f P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.040s user 0.006s sys 0.001s
I20260925 09:57:08.713639 10668 tablet_service.cc:1514] Processing DeleteTablet for tablet 26b7c37959d042b48234acc79290936f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:08 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:08.716681 11122 consensus_queue.cc:1035] T 1969ac471b6f471ebe8e048a96c22837 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:08.723172 11208 consensus_queue.cc:1035] T 8aae639695f44d8c8a61b3ea42ab6ac8 P be97ab5d75934a648d4ab76341193ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:08.725069 11084 tablet_bootstrap.cc:492] T 0a2bbe141e564a96be05de86f2993234 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:08.728567 11122 consensus_queue.cc:1035] T 1969ac471b6f471ebe8e048a96c22837 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:08.730722 11084 tablet_bootstrap.cc:654] T 0a2bbe141e564a96be05de86f2993234 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:08.731956 11211 raft_consensus.cc:482] T 4be3f6e81ddf418fabc7b694a2e62f35 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:08.732494 11211 raft_consensus.cc:504] T 4be3f6e81ddf418fabc7b694a2e62f35 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.736534 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4be3f6e81ddf418fabc7b694a2e62f35" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:08.737354 10984 raft_consensus.cc:2379] T 4be3f6e81ddf418fabc7b694a2e62f35 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b8850818e2ee42a082802d0095c08d21 in current term 1: Already voted for candidate c43461c4ee854ae1ac52b873ccada318 in this term.
I20260925 09:57:08.737138 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4be3f6e81ddf418fabc7b694a2e62f35" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:08.739482 10621 leader_election.cc:302] T 4be3f6e81ddf418fabc7b694a2e62f35 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8850818e2ee42a082802d0095c08d21; no voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:08.746886 11211 leader_election.cc:288] T 4be3f6e81ddf418fabc7b694a2e62f35 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:08.748555 11211 raft_consensus.cc:3037] T 4be3f6e81ddf418fabc7b694a2e62f35 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:08.759562 11134 raft_consensus.cc:504] T e3b6b985a76b459dbd1de42ea20d3856 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.761432 11134 leader_election.cc:288] T e3b6b985a76b459dbd1de42ea20d3856 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:08.761713 11084 tablet_bootstrap.cc:492] T 0a2bbe141e564a96be05de86f2993234 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:08.762190 11084 ts_tablet_manager.cc:1397] T 0a2bbe141e564a96be05de86f2993234 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.037s user 0.008s sys 0.006s
I20260925 09:57:08.763444 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3b6b985a76b459dbd1de42ea20d3856" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:08.763964 10839 raft_consensus.cc:3037] T e3b6b985a76b459dbd1de42ea20d3856 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:08.764393 11084 raft_consensus.cc:348] T 0a2bbe141e564a96be05de86f2993234 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.765015 11084 raft_consensus.cc:374] T 0a2bbe141e564a96be05de86f2993234 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:08.765247 11084 raft_consensus.cc:729] T 0a2bbe141e564a96be05de86f2993234 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:08.766417 11085 tablet_bootstrap.cc:492] T 6b03e5aad59e46a6b0e43b9cb36278d7 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:08.766772 11085 ts_tablet_manager.cc:1397] T 6b03e5aad59e46a6b0e43b9cb36278d7 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.128s user 0.012s sys 0.004s
I20260925 09:57:08.766674 11084 consensus_queue.cc:260] T 0a2bbe141e564a96be05de86f2993234 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.771109 11084 ts_tablet_manager.cc:1428] T 0a2bbe141e564a96be05de86f2993234 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20260925 09:57:08.772192 11084 tablet_bootstrap.cc:492] T 8c13a2f288b14d03a3ca4fdd4889cbad P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:08.773137 11085 raft_consensus.cc:348] T 6b03e5aad59e46a6b0e43b9cb36278d7 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.773751 11085 raft_consensus.cc:374] T 6b03e5aad59e46a6b0e43b9cb36278d7 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:08.773959 11085 raft_consensus.cc:729] T 6b03e5aad59e46a6b0e43b9cb36278d7 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:08.778337 11085 consensus_queue.cc:260] T 6b03e5aad59e46a6b0e43b9cb36278d7 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.781071 11084 tablet_bootstrap.cc:654] T 8c13a2f288b14d03a3ca4fdd4889cbad P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:08.782958 11085 ts_tablet_manager.cc:1428] T 6b03e5aad59e46a6b0e43b9cb36278d7 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.016s user 0.003s sys 0.003s
I20260925 09:57:08.789532 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4be3f6e81ddf418fabc7b694a2e62f35" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:08.791726 10688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8be01fe42894b96bbdd5ce2403ff1f4" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:08.788048 11085 tablet_bootstrap.cc:492] T 04fafb9dcf274806b11b2598e73853e9 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:08.799046 10685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d48d4517c13049389c5b2ba1ab7b14bc" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:08.799974 11084 tablet_bootstrap.cc:492] T 8c13a2f288b14d03a3ca4fdd4889cbad P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:08.800347 11084 ts_tablet_manager.cc:1397] T 8c13a2f288b14d03a3ca4fdd4889cbad P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.028s user 0.001s sys 0.008s
I20260925 09:57:08.801455 10685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3b6b985a76b459dbd1de42ea20d3856" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
W20260925 09:57:08.801604 10919 leader_election.cc:341] T d48d4517c13049389c5b2ba1ab7b14bc P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:08.802033 10919 leader_election.cc:302] T d48d4517c13049389c5b2ba1ab7b14bc P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:08.802582 11084 raft_consensus.cc:348] T 8c13a2f288b14d03a3ca4fdd4889cbad P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.803220 11084 raft_consensus.cc:374] T 8c13a2f288b14d03a3ca4fdd4889cbad P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:08.803483 11084 raft_consensus.cc:729] T 8c13a2f288b14d03a3ca4fdd4889cbad P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:08.803834 11134 raft_consensus.cc:2726] T d48d4517c13049389c5b2ba1ab7b14bc P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:08.804144 11084 consensus_queue.cc:260] T 8c13a2f288b14d03a3ca4fdd4889cbad P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:08.804687 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 26b7c37959d042b48234acc79290936f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 26b7c37959d042b48234acc79290936f already in progress: creating tablet
I20260925 09:57:08.805467 10682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3b6b985a76b459dbd1de42ea20d3856" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:08.808966 10919 leader_election.cc:341] T e3b6b985a76b459dbd1de42ea20d3856 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:08.808109 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b03e5aad59e46a6b0e43b9cb36278d7" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:08.803140 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4be3f6e81ddf418fabc7b694a2e62f35" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:08.810877 10683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04fafb9dcf274806b11b2598e73853e9" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:08.812835 11214 log.cc:924] Time spent T 8aae639695f44d8c8a61b3ea42ab6ac8 P b8850818e2ee42a082802d0095c08d21: Append to log took a long time: real 0.061s user 0.001s sys 0.000s
W20260925 09:57:08.800422 10379 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.60:36985, user_credentials={real_user=slave}} blocked reactor thread for 132471us
W20260925 09:57:08.796746 10621 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 56412.6us
W20260925 09:57:08.818612 10775 leader_election.cc:341] T a8be01fe42894b96bbdd5ce2403ff1f4 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:08.820238 10919 leader_election.cc:341] T e3b6b985a76b459dbd1de42ea20d3856 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:08.821022 10919 leader_election.cc:341] T 6b03e5aad59e46a6b0e43b9cb36278d7 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:08.821383 10919 leader_election.cc:302] T 6b03e5aad59e46a6b0e43b9cb36278d7 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:08.822077 11134 raft_consensus.cc:2726] T 6b03e5aad59e46a6b0e43b9cb36278d7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:08.822353 10919 leader_election.cc:341] T 04fafb9dcf274806b11b2598e73853e9 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:08.822705 10919 leader_election.cc:302] T 04fafb9dcf274806b11b2598e73853e9 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:08.823310 11134 raft_consensus.cc:2726] T 04fafb9dcf274806b11b2598e73853e9 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:08.824517 11084 ts_tablet_manager.cc:1428] T 8c13a2f288b14d03a3ca4fdd4889cbad P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.024s user 0.006s sys 0.000s
I20260925 09:57:08.824883 11082 tablet_bootstrap.cc:492] T 6b03e5aad59e46a6b0e43b9cb36278d7 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:08.825270 11082 ts_tablet_manager.cc:1397] T 6b03e5aad59e46a6b0e43b9cb36278d7 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.536s user 0.020s sys 0.002s
I20260925 09:57:08.830657 10384 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 6a5121b78e2d4643b99bd8717c6e93ca (table default.Table_5ff0f5bde5954217976f2876e76adaaa [id=aa7e7f5af2724af797170e1f7429d76a]). Expected version 1 got 0
I20260925 09:57:08.833853 10384 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 840faf042d8749bd958ce1de0207a734 (table default.Table_faa21794a80c407f800038e583c08f16 [id=800eba46046443a7ad196808dec4cdc0]). Expected version 2 got 0
I20260925 09:57:08.834455 11084 tablet_bootstrap.cc:492] T e57065f2bb224a56ad1c3518a0c375da P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:08.842763 11084 tablet_bootstrap.cc:654] T e57065f2bb224a56ad1c3518a0c375da P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:08.842327 11082 raft_consensus.cc:348] T 6b03e5aad59e46a6b0e43b9cb36278d7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.843856 11082 raft_consensus.cc:374] T 6b03e5aad59e46a6b0e43b9cb36278d7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:08.844095 11082 raft_consensus.cc:729] T 6b03e5aad59e46a6b0e43b9cb36278d7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
W20260925 09:57:08.844669 10384 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.1:56563 (request call id 19) took 851 ms (client timeout 1000 ms). Trace:
I20260925 09:57:08.847188 11082 consensus_queue.cc:260] T 6b03e5aad59e46a6b0e43b9cb36278d7 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.861594 11211 raft_consensus.cc:2726] T 4be3f6e81ddf418fabc7b694a2e62f35 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:08.862497 10839 raft_consensus.cc:2454] T e3b6b985a76b459dbd1de42ea20d3856 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be97ab5d75934a648d4ab76341193ec0 in term 1.
I20260925 09:57:08.862356 10687 raft_consensus.cc:2454] T 4be3f6e81ddf418fabc7b694a2e62f35 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:08.863679 10918 leader_election.cc:302] T e3b6b985a76b459dbd1de42ea20d3856 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21
I20260925 09:57:08.864084 11134 raft_consensus.cc:482] T 8c13a2f288b14d03a3ca4fdd4889cbad P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:08.864974 11082 ts_tablet_manager.cc:1428] T 6b03e5aad59e46a6b0e43b9cb36278d7 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.039s user 0.005s sys 0.002s
I20260925 09:57:08.864755 11134 raft_consensus.cc:504] T 8c13a2f288b14d03a3ca4fdd4889cbad P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.866076 11082 tablet_bootstrap.cc:492] T a8be01fe42894b96bbdd5ce2403ff1f4 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:08.868242 11206 raft_consensus.cc:2781] T e3b6b985a76b459dbd1de42ea20d3856 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:08.868700 11206 raft_consensus.cc:686] T e3b6b985a76b459dbd1de42ea20d3856 P be97ab5d75934a648d4ab76341193ec0 [term 1 LEADER]: Becoming Leader. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Running, Role: LEADER
I20260925 09:57:08.869450 11206 consensus_queue.cc:237] T e3b6b985a76b459dbd1de42ea20d3856 P be97ab5d75934a648d4ab76341193ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.873203 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c13a2f288b14d03a3ca4fdd4889cbad" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:08.874521 10919 leader_election.cc:341] T 8c13a2f288b14d03a3ca4fdd4889cbad P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:08.876793 11134 leader_election.cc:288] T 8c13a2f288b14d03a3ca4fdd4889cbad P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:08.879215 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c13a2f288b14d03a3ca4fdd4889cbad" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:08.880728 10918 leader_election.cc:341] T 8c13a2f288b14d03a3ca4fdd4889cbad P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:08.881099 10918 leader_election.cc:302] T 8c13a2f288b14d03a3ca4fdd4889cbad P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:08.881269 11167 consensus_queue.cc:1035] T a0976e0c01234b38a7ab934dc1b289c4 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:08.882138 11134 raft_consensus.cc:2726] T 8c13a2f288b14d03a3ca4fdd4889cbad P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:08.887485 11082 tablet_bootstrap.cc:654] T a8be01fe42894b96bbdd5ce2403ff1f4 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:08.902565 11085 tablet_bootstrap.cc:654] T 04fafb9dcf274806b11b2598e73853e9 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:08.845116 10384 rpcz_store.cc:269] 0925 09:57:07.993089 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:07.998098 (+ 5009us) service_pool.cc:227] Handling call
0925 09:57:08.049522 (+ 51424us) spinlock_profiling.cc:243] Waited 50.1 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2ad9c3e 00007fabf0610fb4 00007fabf0610f42 00007fabf0610e71 00007fabf0610dec 00007fabf06100a2 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:08.844607 (+795085us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":2,"spinlock_wait_cycles":115220224}
I20260925 09:57:08.921994 11082 tablet_bootstrap.cc:492] T a8be01fe42894b96bbdd5ce2403ff1f4 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:08.922406 11082 ts_tablet_manager.cc:1397] T a8be01fe42894b96bbdd5ce2403ff1f4 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.057s user 0.006s sys 0.012s
I20260925 09:57:08.925654 11082 raft_consensus.cc:348] T a8be01fe42894b96bbdd5ce2403ff1f4 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.926335 11082 raft_consensus.cc:374] T a8be01fe42894b96bbdd5ce2403ff1f4 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:08.926541 11167 consensus_queue.cc:1035] T a0976e0c01234b38a7ab934dc1b289c4 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:08.926595 11082 raft_consensus.cc:729] T a8be01fe42894b96bbdd5ce2403ff1f4 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:08.931922 11082 consensus_queue.cc:260] T a8be01fe42894b96bbdd5ce2403ff1f4 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.934432 11082 ts_tablet_manager.cc:1428] T a8be01fe42894b96bbdd5ce2403ff1f4 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.012s user 0.001s sys 0.005s
I20260925 09:57:08.936303 11206 raft_consensus.cc:482] T aad3cb5e33f64cb885d51f4ea089307b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:08.936725 11206 raft_consensus.cc:504] T aad3cb5e33f64cb885d51f4ea089307b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.938560 11206 leader_election.cc:288] T aad3cb5e33f64cb885d51f4ea089307b P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:08.940510 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aad3cb5e33f64cb885d51f4ea089307b" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:08.941797 10918 leader_election.cc:341] T aad3cb5e33f64cb885d51f4ea089307b P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:08.947953 11082 tablet_bootstrap.cc:492] T e3b6b985a76b459dbd1de42ea20d3856 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:08.949637 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aad3cb5e33f64cb885d51f4ea089307b" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:08.950855 10919 leader_election.cc:341] T aad3cb5e33f64cb885d51f4ea089307b P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:08.951241 10919 leader_election.cc:302] T aad3cb5e33f64cb885d51f4ea089307b P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:08.952128 11206 raft_consensus.cc:2726] T aad3cb5e33f64cb885d51f4ea089307b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:08.958468 11082 tablet_bootstrap.cc:654] T e3b6b985a76b459dbd1de42ea20d3856 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:08.973278 10390 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e3b6b985a76b459dbd1de42ea20d3856 consensus state has no leader
W20260925 09:57:08.988027 10379 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.62:34455, user_credentials={real_user=slave}} blocked reactor thread for 49894.4us
I20260925 09:57:08.994764 11082 tablet_bootstrap.cc:492] T e3b6b985a76b459dbd1de42ea20d3856 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:08.995131 11082 ts_tablet_manager.cc:1397] T e3b6b985a76b459dbd1de42ea20d3856 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.047s user 0.009s sys 0.004s
I20260925 09:57:08.997112 11082 raft_consensus.cc:348] T e3b6b985a76b459dbd1de42ea20d3856 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.997709 11082 raft_consensus.cc:374] T e3b6b985a76b459dbd1de42ea20d3856 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:08.997836 11206 raft_consensus.cc:482] T 0a2bbe141e564a96be05de86f2993234 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:08.997929 11082 raft_consensus.cc:729] T e3b6b985a76b459dbd1de42ea20d3856 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:08.998279 11206 raft_consensus.cc:504] T 0a2bbe141e564a96be05de86f2993234 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:08.998674 11082 consensus_queue.cc:260] T e3b6b985a76b459dbd1de42ea20d3856 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.000411 11206 leader_election.cc:288] T 0a2bbe141e564a96be05de86f2993234 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:09.001857 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a2bbe141e564a96be05de86f2993234" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:09.002859 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a2bbe141e564a96be05de86f2993234" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:09.003249 11082 ts_tablet_manager.cc:1428] T e3b6b985a76b459dbd1de42ea20d3856 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.008s user 0.005s sys 0.001s
I20260925 09:57:09.004055 11082 tablet_bootstrap.cc:492] T d48d4517c13049389c5b2ba1ab7b14bc P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
W20260925 09:57:09.004886 10919 leader_election.cc:341] T 0a2bbe141e564a96be05de86f2993234 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:09.006750 10918 leader_election.cc:341] T 0a2bbe141e564a96be05de86f2993234 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:09.007367 10918 leader_election.cc:302] T 0a2bbe141e564a96be05de86f2993234 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:09.009804 11206 raft_consensus.cc:2726] T 0a2bbe141e564a96be05de86f2993234 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:09.013429 11082 tablet_bootstrap.cc:654] T d48d4517c13049389c5b2ba1ab7b14bc P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:09.035768 10388 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_ecec3b643d1c4b76b700090f3a3cab06" } modify_external_catalogs: true
I20260925 09:57:09.036437 10388 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_ecec3b643d1c4b76b700090f3a3cab06" } modify_external_catalogs: true
I20260925 09:57:09.042906 11082 tablet_bootstrap.cc:492] T d48d4517c13049389c5b2ba1ab7b14bc P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:09.043552 11082 ts_tablet_manager.cc:1397] T d48d4517c13049389c5b2ba1ab7b14bc P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.040s user 0.007s sys 0.006s
I20260925 09:57:09.045562 11082 raft_consensus.cc:348] T d48d4517c13049389c5b2ba1ab7b14bc P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.046167 11082 raft_consensus.cc:374] T d48d4517c13049389c5b2ba1ab7b14bc P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:09.046380 11082 raft_consensus.cc:729] T d48d4517c13049389c5b2ba1ab7b14bc P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:09.046906 11082 consensus_queue.cc:260] T d48d4517c13049389c5b2ba1ab7b14bc P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.068638 11082 ts_tablet_manager.cc:1428] T d48d4517c13049389c5b2ba1ab7b14bc P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.025s user 0.005s sys 0.000s
I20260925 09:57:09.071065 10385 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet f2803c986c0d461b8a66fc796ca8c9dd
I20260925 09:57:09.075875 11082 tablet_bootstrap.cc:492] T 931769e78f32467b8f27bacb286a7a60 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:09.081003 11206 raft_consensus.cc:482] T 931769e78f32467b8f27bacb286a7a60 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:09.081426 11206 raft_consensus.cc:504] T 931769e78f32467b8f27bacb286a7a60 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.083335 11206 leader_election.cc:288] T 931769e78f32467b8f27bacb286a7a60 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:09.083725 11082 tablet_bootstrap.cc:654] T 931769e78f32467b8f27bacb286a7a60 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:09.084978 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "931769e78f32467b8f27bacb286a7a60" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:09.088056 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "931769e78f32467b8f27bacb286a7a60" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:09.089291 11211 raft_consensus.cc:482] T a8be01fe42894b96bbdd5ce2403ff1f4 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:09.089685 11211 raft_consensus.cc:504] T a8be01fe42894b96bbdd5ce2403ff1f4 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:09.089785 10918 leader_election.cc:341] T 931769e78f32467b8f27bacb286a7a60 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:09.091164 10919 leader_election.cc:341] T 931769e78f32467b8f27bacb286a7a60 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:09.091769 10919 leader_election.cc:302] T 931769e78f32467b8f27bacb286a7a60 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:09.092125 11211 leader_election.cc:288] T a8be01fe42894b96bbdd5ce2403ff1f4 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:09.093372 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8be01fe42894b96bbdd5ce2403ff1f4" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:09.094250 10984 raft_consensus.cc:2379] T a8be01fe42894b96bbdd5ce2403ff1f4 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b8850818e2ee42a082802d0095c08d21 in current term 1: Already voted for candidate c43461c4ee854ae1ac52b873ccada318 in this term.
I20260925 09:57:09.093389 11206 raft_consensus.cc:2726] T 931769e78f32467b8f27bacb286a7a60 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:09.103750 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8be01fe42894b96bbdd5ce2403ff1f4" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:09.106513 10621 leader_election.cc:302] T a8be01fe42894b96bbdd5ce2403ff1f4 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8850818e2ee42a082802d0095c08d21; no voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:09.108237 11211 raft_consensus.cc:3037] T a8be01fe42894b96bbdd5ce2403ff1f4 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:09.109634 11084 tablet_bootstrap.cc:492] T e57065f2bb224a56ad1c3518a0c375da P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:09.110131 11084 ts_tablet_manager.cc:1397] T e57065f2bb224a56ad1c3518a0c375da P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.276s user 0.041s sys 0.039s
I20260925 09:57:09.117027 11084 raft_consensus.cc:348] T e57065f2bb224a56ad1c3518a0c375da P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.119500 11084 raft_consensus.cc:374] T e57065f2bb224a56ad1c3518a0c375da P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:09.119903 11085 tablet_bootstrap.cc:492] T 04fafb9dcf274806b11b2598e73853e9 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:09.119859 11084 raft_consensus.cc:729] T e57065f2bb224a56ad1c3518a0c375da P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:09.120283 11085 ts_tablet_manager.cc:1397] T 04fafb9dcf274806b11b2598e73853e9 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.332s user 0.010s sys 0.002s
I20260925 09:57:09.120752 11084 consensus_queue.cc:260] T e57065f2bb224a56ad1c3518a0c375da P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.131542 11085 raft_consensus.cc:348] T 04fafb9dcf274806b11b2598e73853e9 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.132310 11085 raft_consensus.cc:374] T 04fafb9dcf274806b11b2598e73853e9 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:09.132565 11085 raft_consensus.cc:729] T 04fafb9dcf274806b11b2598e73853e9 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:09.139359 11084 ts_tablet_manager.cc:1428] T e57065f2bb224a56ad1c3518a0c375da P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.029s user 0.005s sys 0.005s
I20260925 09:57:09.142736 11195 raft_consensus.cc:482] T 6b03e5aad59e46a6b0e43b9cb36278d7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:09.143323 11195 raft_consensus.cc:504] T 6b03e5aad59e46a6b0e43b9cb36278d7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.145186 11084 tablet_bootstrap.cc:492] T d8be238061c146eb96d17a22c1b2104a P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:09.145831 11195 leader_election.cc:288] T 6b03e5aad59e46a6b0e43b9cb36278d7 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:09.146883 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b03e5aad59e46a6b0e43b9cb36278d7" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:09.147512 10984 raft_consensus.cc:2454] T 6b03e5aad59e46a6b0e43b9cb36278d7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:09.148643 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b03e5aad59e46a6b0e43b9cb36278d7" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:09.148950 10619 leader_election.cc:302] T 6b03e5aad59e46a6b0e43b9cb36278d7 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:09.149408 10839 raft_consensus.cc:2454] T 6b03e5aad59e46a6b0e43b9cb36278d7 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:09.150043 10385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_fb1abc10cc744e82a45c0c93258c8305"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:09.151711 10385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fb1abc10cc744e82a45c0c93258c8305 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:09.153333 10390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_a0c3cadc623f413bb77d0ddb87cafa8c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:09.156951 11082 tablet_bootstrap.cc:492] T 931769e78f32467b8f27bacb286a7a60 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:09.157315 11082 ts_tablet_manager.cc:1397] T 931769e78f32467b8f27bacb286a7a60 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.082s user 0.019s sys 0.011s
W20260925 09:57:09.155331 10390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a0c3cadc623f413bb77d0ddb87cafa8c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:09.159432 11082 raft_consensus.cc:348] T 931769e78f32467b8f27bacb286a7a60 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.160028 11082 raft_consensus.cc:374] T 931769e78f32467b8f27bacb286a7a60 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:09.160288 11082 raft_consensus.cc:729] T 931769e78f32467b8f27bacb286a7a60 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:09.160935 11082 consensus_queue.cc:260] T 931769e78f32467b8f27bacb286a7a60 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.162251 11215 raft_consensus.cc:2781] T 6b03e5aad59e46a6b0e43b9cb36278d7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:09.162628 11215 raft_consensus.cc:482] T 6b03e5aad59e46a6b0e43b9cb36278d7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:09.162940 11215 raft_consensus.cc:3037] T 6b03e5aad59e46a6b0e43b9cb36278d7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:09.171860 11082 ts_tablet_manager.cc:1428] T 931769e78f32467b8f27bacb286a7a60 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.014s user 0.005s sys 0.000s
I20260925 09:57:09.172590 11082 tablet_bootstrap.cc:492] T 59f0c6489a33433899bb6e62088121b6 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:09.172857 11211 raft_consensus.cc:2726] T a8be01fe42894b96bbdd5ce2403ff1f4 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:09.178323 11084 tablet_bootstrap.cc:654] T d8be238061c146eb96d17a22c1b2104a P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:09.178726 11082 tablet_bootstrap.cc:654] T 59f0c6489a33433899bb6e62088121b6 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:09.186775 10386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_7d9b5b59509847c4953652343f89d5e8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:09.189286 10386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7d9b5b59509847c4953652343f89d5e8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:09.203886 10389 catalog_manager.cc:5495] T 8aae639695f44d8c8a61b3ea42ab6ac8 P be97ab5d75934a648d4ab76341193ec0 reported cstate change: term changed from 0 to 1, leader changed from <none> to be97ab5d75934a648d4ab76341193ec0 (127.10.9.3). New cstate: current_term: 1 leader_uuid: "be97ab5d75934a648d4ab76341193ec0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: HEALTHY } } }
I20260925 09:57:09.209769 10389 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 6a5121b78e2d4643b99bd8717c6e93ca (table default.Table_5ff0f5bde5954217976f2876e76adaaa [id=aa7e7f5af2724af797170e1f7429d76a]). Expected version 1 got 0
I20260925 09:57:09.210506 10389 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 840faf042d8749bd958ce1de0207a734 (table default.Table_faa21794a80c407f800038e583c08f16 [id=800eba46046443a7ad196808dec4cdc0]). Expected version 2 got 0
I20260925 09:57:09.214124 10813 tablet_service.cc:1467] Processing CreateTablet for tablet b7646a1cde1b45b9bf7928c3e13bcbd3 (DEFAULT_TABLE table=default.Table_0fe949ec62d547c9a9acbf96b374edcb [id=11e4b3a3251f4fa4a78de075f8ecb6f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:09.216676 10813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7646a1cde1b45b9bf7928c3e13bcbd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:09.225759 10961 tablet_service.cc:1467] Processing CreateTablet for tablet b7646a1cde1b45b9bf7928c3e13bcbd3 (DEFAULT_TABLE table=default.Table_0fe949ec62d547c9a9acbf96b374edcb [id=11e4b3a3251f4fa4a78de075f8ecb6f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:09.226943 10961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7646a1cde1b45b9bf7928c3e13bcbd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:09.227094 10819 tablet_service.cc:1467] Processing CreateTablet for tablet 8b281fc20e6f42239fee80d05da88ab6 (DEFAULT_TABLE table=default.Table_8dfe0230298f4ff9b7e34161a9e90f53 [id=50788524d4b24e429c978c44428dc892]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:09.228296 10819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b281fc20e6f42239fee80d05da88ab6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:09.228379 10668 tablet_service.cc:1467] Processing CreateTablet for tablet b7646a1cde1b45b9bf7928c3e13bcbd3 (DEFAULT_TABLE table=default.Table_0fe949ec62d547c9a9acbf96b374edcb [id=11e4b3a3251f4fa4a78de075f8ecb6f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:09.229501 10668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7646a1cde1b45b9bf7928c3e13bcbd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:09.232197 10963 tablet_service.cc:1467] Processing CreateTablet for tablet 2d661d5000894c45ac3d1c27f0e72889 (DEFAULT_TABLE table=default.Table_c6db4ef50ad442fcb28e2646fd6b7571 [id=756dfeff38d64529944858afa495ab49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:09.233316 10963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d661d5000894c45ac3d1c27f0e72889. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:09.234388 10667 tablet_service.cc:1467] Processing CreateTablet for tablet 2d661d5000894c45ac3d1c27f0e72889 (DEFAULT_TABLE table=default.Table_c6db4ef50ad442fcb28e2646fd6b7571 [id=756dfeff38d64529944858afa495ab49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:09.236158 10962 tablet_service.cc:1467] Processing CreateTablet for tablet 8b281fc20e6f42239fee80d05da88ab6 (DEFAULT_TABLE table=default.Table_8dfe0230298f4ff9b7e34161a9e90f53 [id=50788524d4b24e429c978c44428dc892]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:09.237118 10667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d661d5000894c45ac3d1c27f0e72889. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:09.237466 10962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b281fc20e6f42239fee80d05da88ab6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:09.240095 10818 tablet_service.cc:1467] Processing CreateTablet for tablet 2d661d5000894c45ac3d1c27f0e72889 (DEFAULT_TABLE table=default.Table_c6db4ef50ad442fcb28e2646fd6b7571 [id=756dfeff38d64529944858afa495ab49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:09.241044 10818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d661d5000894c45ac3d1c27f0e72889. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:09.244462 10663 tablet_service.cc:1467] Processing CreateTablet for tablet 8b281fc20e6f42239fee80d05da88ab6 (DEFAULT_TABLE table=default.Table_8dfe0230298f4ff9b7e34161a9e90f53 [id=50788524d4b24e429c978c44428dc892]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:09.245815 10663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b281fc20e6f42239fee80d05da88ab6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:09.252373 10390 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_18ecd2dd5b4e472cb12e72ce8bce84f9" } new_table_name: "default.Table_2d7acf0d8022443090ba12cd87956502" modify_external_catalogs: true
I20260925 09:57:09.265585 11206 raft_consensus.cc:482] T 26b7c37959d042b48234acc79290936f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:09.266047 11206 raft_consensus.cc:504] T 26b7c37959d042b48234acc79290936f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.267671 11082 tablet_bootstrap.cc:492] T 59f0c6489a33433899bb6e62088121b6 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:09.268445 11206 leader_election.cc:288] T 26b7c37959d042b48234acc79290936f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:09.269260 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26b7c37959d042b48234acc79290936f" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:09.270658 10919 leader_election.cc:341] T 26b7c37959d042b48234acc79290936f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:09.272143 11082 ts_tablet_manager.cc:1397] T 59f0c6489a33433899bb6e62088121b6 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.100s user 0.015s sys 0.026s
I20260925 09:57:09.275703 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26b7c37959d042b48234acc79290936f" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:09.276603 10959 tablet_service.cc:1467] Processing CreateTablet for tablet e48d86c20e9e4db990c23359ff1d1a6d (DEFAULT_TABLE table=default.Table_3b9695593bd6426e8932f7b11f01d1ac [id=c524d4b2bbb1495cbe85a6f1b3d595d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:09.277643 10959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e48d86c20e9e4db990c23359ff1d1a6d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:09.278847 10918 leader_election.cc:341] T 26b7c37959d042b48234acc79290936f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:09.279405 10918 leader_election.cc:302] T 26b7c37959d042b48234acc79290936f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:09.281491 11206 raft_consensus.cc:2726] T 26b7c37959d042b48234acc79290936f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:09.282183 10819 tablet_service.cc:1467] Processing CreateTablet for tablet e48d86c20e9e4db990c23359ff1d1a6d (DEFAULT_TABLE table=default.Table_3b9695593bd6426e8932f7b11f01d1ac [id=c524d4b2bbb1495cbe85a6f1b3d595d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:09.285498 10819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e48d86c20e9e4db990c23359ff1d1a6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:09.289007 11082 raft_consensus.cc:348] T 59f0c6489a33433899bb6e62088121b6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.289664 11082 raft_consensus.cc:374] T 59f0c6489a33433899bb6e62088121b6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:09.289920 11082 raft_consensus.cc:729] T 59f0c6489a33433899bb6e62088121b6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:09.290443 11082 consensus_queue.cc:260] T 59f0c6489a33433899bb6e62088121b6 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.291575 11215 raft_consensus.cc:504] T 6b03e5aad59e46a6b0e43b9cb36278d7 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.292877 10385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_6c59a202a45f43aaa4e1de75e372d0ff"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:09.294018 11215 leader_election.cc:288] T 6b03e5aad59e46a6b0e43b9cb36278d7 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
W20260925 09:57:09.294916 10385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6c59a202a45f43aaa4e1de75e372d0ff in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:09.303591 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b03e5aad59e46a6b0e43b9cb36278d7" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:09.304198 10984 raft_consensus.cc:3037] T 6b03e5aad59e46a6b0e43b9cb36278d7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:09.308279 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b03e5aad59e46a6b0e43b9cb36278d7" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:09.308861 10839 raft_consensus.cc:3037] T 6b03e5aad59e46a6b0e43b9cb36278d7 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:09.311892 10963 tablet_service.cc:1467] Processing CreateTablet for tablet 0e882284a014455192375daad391efaf (DEFAULT_TABLE table=default.Table_62ea48a919ec4159bcc86ebb048e3545 [id=051a096f481f4f20a59081fbd08b3b1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:09.311885 11085 consensus_queue.cc:260] T 04fafb9dcf274806b11b2598e73853e9 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.312825 10963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e882284a014455192375daad391efaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:09.313787 11085 ts_tablet_manager.cc:1428] T 04fafb9dcf274806b11b2598e73853e9 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.193s user 0.001s sys 0.003s
I20260925 09:57:09.314606 11085 tablet_bootstrap.cc:492] T 931769e78f32467b8f27bacb286a7a60 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:09.332732 10818 tablet_service.cc:1467] Processing CreateTablet for tablet 0e882284a014455192375daad391efaf (DEFAULT_TABLE table=default.Table_62ea48a919ec4159bcc86ebb048e3545 [id=051a096f481f4f20a59081fbd08b3b1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:09.334308 10818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e882284a014455192375daad391efaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:09.336952 10666 tablet_service.cc:1467] Processing CreateTablet for tablet e48d86c20e9e4db990c23359ff1d1a6d (DEFAULT_TABLE table=default.Table_3b9695593bd6426e8932f7b11f01d1ac [id=c524d4b2bbb1495cbe85a6f1b3d595d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:09.338023 10666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e48d86c20e9e4db990c23359ff1d1a6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:09.339529 11215 raft_consensus.cc:482] T 931769e78f32467b8f27bacb286a7a60 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:09.340119 11084 tablet_bootstrap.cc:492] T d8be238061c146eb96d17a22c1b2104a P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:09.339982 11215 raft_consensus.cc:504] T 931769e78f32467b8f27bacb286a7a60 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.340520 11084 ts_tablet_manager.cc:1397] T d8be238061c146eb96d17a22c1b2104a P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.195s user 0.021s sys 0.012s
I20260925 09:57:09.342218 11085 tablet_bootstrap.cc:654] T 931769e78f32467b8f27bacb286a7a60 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:09.342195 10664 tablet_service.cc:1467] Processing CreateTablet for tablet 0e882284a014455192375daad391efaf (DEFAULT_TABLE table=default.Table_62ea48a919ec4159bcc86ebb048e3545 [id=051a096f481f4f20a59081fbd08b3b1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:09.342792 11082 ts_tablet_manager.cc:1428] T 59f0c6489a33433899bb6e62088121b6 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.070s user 0.003s sys 0.003s
I20260925 09:57:09.342676 11084 raft_consensus.cc:348] T d8be238061c146eb96d17a22c1b2104a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.343230 10664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e882284a014455192375daad391efaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:09.343364 11084 raft_consensus.cc:374] T d8be238061c146eb96d17a22c1b2104a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:09.343626 11084 raft_consensus.cc:729] T d8be238061c146eb96d17a22c1b2104a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:09.343693 11082 tablet_bootstrap.cc:492] T 04fafb9dcf274806b11b2598e73853e9 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:09.344161 11084 consensus_queue.cc:260] T d8be238061c146eb96d17a22c1b2104a P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.352967 11134 raft_consensus.cc:482] T e57065f2bb224a56ad1c3518a0c375da P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:09.353411 11134 raft_consensus.cc:504] T e57065f2bb224a56ad1c3518a0c375da P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.354616 11084 ts_tablet_manager.cc:1428] T d8be238061c146eb96d17a22c1b2104a P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.014s user 0.002s sys 0.004s
I20260925 09:57:09.355355 11084 tablet_bootstrap.cc:492] T b7646a1cde1b45b9bf7928c3e13bcbd3 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:09.356894 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e57065f2bb224a56ad1c3518a0c375da" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:09.358860 10919 leader_election.cc:341] T e57065f2bb224a56ad1c3518a0c375da P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:09.359045 11082 tablet_bootstrap.cc:654] T 04fafb9dcf274806b11b2598e73853e9 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:09.355305 11134 leader_election.cc:288] T e57065f2bb224a56ad1c3518a0c375da P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:09.361850 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e57065f2bb224a56ad1c3518a0c375da" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:09.369007 10918 leader_election.cc:341] T e57065f2bb224a56ad1c3518a0c375da P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:09.370762 11085 tablet_bootstrap.cc:492] T 931769e78f32467b8f27bacb286a7a60 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:09.371142 11085 ts_tablet_manager.cc:1397] T 931769e78f32467b8f27bacb286a7a60 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.057s user 0.007s sys 0.005s
I20260925 09:57:09.374579 11085 raft_consensus.cc:348] T 931769e78f32467b8f27bacb286a7a60 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.375604 11085 raft_consensus.cc:374] T 931769e78f32467b8f27bacb286a7a60 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:09.375952 11085 raft_consensus.cc:729] T 931769e78f32467b8f27bacb286a7a60 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:09.376662 11085 consensus_queue.cc:260] T 931769e78f32467b8f27bacb286a7a60 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.378314 11215 leader_election.cc:288] T 931769e78f32467b8f27bacb286a7a60 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:09.381393 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "931769e78f32467b8f27bacb286a7a60" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:09.381980 10838 raft_consensus.cc:2454] T 931769e78f32467b8f27bacb286a7a60 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:09.384974 11085 ts_tablet_manager.cc:1428] T 931769e78f32467b8f27bacb286a7a60 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.014s user 0.006s sys 0.001s
I20260925 09:57:09.385776 10621 leader_election.cc:302] T 931769e78f32467b8f27bacb286a7a60 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:09.387135 11215 raft_consensus.cc:2781] T 931769e78f32467b8f27bacb286a7a60 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:09.387543 11215 raft_consensus.cc:482] T 931769e78f32467b8f27bacb286a7a60 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:09.387921 11215 raft_consensus.cc:3037] T 931769e78f32467b8f27bacb286a7a60 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:09.375159 10984 raft_consensus.cc:2454] T 6b03e5aad59e46a6b0e43b9cb36278d7 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:09.394143 10619 leader_election.cc:302] T 6b03e5aad59e46a6b0e43b9cb36278d7 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:09.394958 11195 raft_consensus.cc:2781] T 6b03e5aad59e46a6b0e43b9cb36278d7 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:09.395166 11085 tablet_bootstrap.cc:492] T d48d4517c13049389c5b2ba1ab7b14bc P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:09.395464 11195 raft_consensus.cc:686] T 6b03e5aad59e46a6b0e43b9cb36278d7 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:09.396360 11195 consensus_queue.cc:237] T 6b03e5aad59e46a6b0e43b9cb36278d7 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.379855 10982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "931769e78f32467b8f27bacb286a7a60" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:09.384119 11084 tablet_bootstrap.cc:654] T b7646a1cde1b45b9bf7928c3e13bcbd3 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:09.400020 10982 raft_consensus.cc:2454] T 931769e78f32467b8f27bacb286a7a60 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:09.402503 10918 leader_election.cc:302] T e57065f2bb224a56ad1c3518a0c375da P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:09.403911 11206 raft_consensus.cc:2726] T e57065f2bb224a56ad1c3518a0c375da P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:09.401908 11085 tablet_bootstrap.cc:654] T d48d4517c13049389c5b2ba1ab7b14bc P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:09.426692 11084 tablet_bootstrap.cc:492] T b7646a1cde1b45b9bf7928c3e13bcbd3 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:09.427089 11084 ts_tablet_manager.cc:1397] T b7646a1cde1b45b9bf7928c3e13bcbd3 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.072s user 0.005s sys 0.005s
I20260925 09:57:09.429116 11084 raft_consensus.cc:348] T b7646a1cde1b45b9bf7928c3e13bcbd3 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.429750 11084 raft_consensus.cc:374] T b7646a1cde1b45b9bf7928c3e13bcbd3 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:09.429981 11084 raft_consensus.cc:729] T b7646a1cde1b45b9bf7928c3e13bcbd3 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:09.430512 11084 consensus_queue.cc:260] T b7646a1cde1b45b9bf7928c3e13bcbd3 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.438136 11085 tablet_bootstrap.cc:492] T d48d4517c13049389c5b2ba1ab7b14bc P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:09.438392 11084 ts_tablet_manager.cc:1428] T b7646a1cde1b45b9bf7928c3e13bcbd3 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.011s user 0.005s sys 0.001s
I20260925 09:57:09.438524 11085 ts_tablet_manager.cc:1397] T d48d4517c13049389c5b2ba1ab7b14bc P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.044s user 0.004s sys 0.008s
I20260925 09:57:09.439194 10839 raft_consensus.cc:2454] T 6b03e5aad59e46a6b0e43b9cb36278d7 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:09.439149 11084 tablet_bootstrap.cc:492] T 2d661d5000894c45ac3d1c27f0e72889 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:09.441026 11085 raft_consensus.cc:348] T d48d4517c13049389c5b2ba1ab7b14bc P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.456051 11085 raft_consensus.cc:374] T d48d4517c13049389c5b2ba1ab7b14bc P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:09.456339 11085 raft_consensus.cc:729] T d48d4517c13049389c5b2ba1ab7b14bc P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:09.456908 11085 consensus_queue.cc:260] T d48d4517c13049389c5b2ba1ab7b14bc P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.458793 11085 ts_tablet_manager.cc:1428] T d48d4517c13049389c5b2ba1ab7b14bc P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.020s user 0.006s sys 0.001s
I20260925 09:57:09.470327 11085 tablet_bootstrap.cc:492] T 59f0c6489a33433899bb6e62088121b6 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:09.470717 11082 tablet_bootstrap.cc:492] T 04fafb9dcf274806b11b2598e73853e9 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:09.471095 11082 ts_tablet_manager.cc:1397] T 04fafb9dcf274806b11b2598e73853e9 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.128s user 0.014s sys 0.007s
I20260925 09:57:09.473313 11082 raft_consensus.cc:348] T 04fafb9dcf274806b11b2598e73853e9 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.473917 11082 raft_consensus.cc:374] T 04fafb9dcf274806b11b2598e73853e9 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:09.474151 11082 raft_consensus.cc:729] T 04fafb9dcf274806b11b2598e73853e9 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:09.475522 11084 tablet_bootstrap.cc:654] T 2d661d5000894c45ac3d1c27f0e72889 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:09.475929 11085 tablet_bootstrap.cc:654] T 59f0c6489a33433899bb6e62088121b6 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:09.477387 11215 raft_consensus.cc:504] T 931769e78f32467b8f27bacb286a7a60 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.478299 11082 consensus_queue.cc:260] T 04fafb9dcf274806b11b2598e73853e9 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.484970 11082 ts_tablet_manager.cc:1428] T 04fafb9dcf274806b11b2598e73853e9 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.009s user 0.005s sys 0.001s
I20260925 09:57:09.486166 11082 tablet_bootstrap.cc:492] T 0a2bbe141e564a96be05de86f2993234 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:09.496091 11082 tablet_bootstrap.cc:654] T 0a2bbe141e564a96be05de86f2993234 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:09.498077 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "931769e78f32467b8f27bacb286a7a60" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:09.501060 10982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "931769e78f32467b8f27bacb286a7a60" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:09.501644 10982 raft_consensus.cc:3037] T 931769e78f32467b8f27bacb286a7a60 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:09.503247 11215 leader_election.cc:288] T 931769e78f32467b8f27bacb286a7a60 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:09.499115 10839 raft_consensus.cc:3037] T 931769e78f32467b8f27bacb286a7a60 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:09.507439 11167 consensus_queue.cc:1035] T 4be3f6e81ddf418fabc7b694a2e62f35 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:09.516467 11082 tablet_bootstrap.cc:492] T 0a2bbe141e564a96be05de86f2993234 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:09.516853 11082 ts_tablet_manager.cc:1397] T 0a2bbe141e564a96be05de86f2993234 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.031s user 0.013s sys 0.000s
I20260925 09:57:09.518105 11085 tablet_bootstrap.cc:492] T 59f0c6489a33433899bb6e62088121b6 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:09.518534 11085 ts_tablet_manager.cc:1397] T 59f0c6489a33433899bb6e62088121b6 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.048s user 0.014s sys 0.000s
I20260925 09:57:09.519321 11082 raft_consensus.cc:348] T 0a2bbe141e564a96be05de86f2993234 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.519955 11082 raft_consensus.cc:374] T 0a2bbe141e564a96be05de86f2993234 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:09.520197 11082 raft_consensus.cc:729] T 0a2bbe141e564a96be05de86f2993234 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:09.521209 11085 raft_consensus.cc:348] T 59f0c6489a33433899bb6e62088121b6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.522343 11085 raft_consensus.cc:374] T 59f0c6489a33433899bb6e62088121b6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:09.522749 11085 raft_consensus.cc:729] T 59f0c6489a33433899bb6e62088121b6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:09.528450 11084 tablet_bootstrap.cc:492] T 2d661d5000894c45ac3d1c27f0e72889 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:09.528872 11084 ts_tablet_manager.cc:1397] T 2d661d5000894c45ac3d1c27f0e72889 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.090s user 0.004s sys 0.004s
I20260925 09:57:09.529564 11082 consensus_queue.cc:260] T 0a2bbe141e564a96be05de86f2993234 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.531497 11082 ts_tablet_manager.cc:1428] T 0a2bbe141e564a96be05de86f2993234 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.014s user 0.006s sys 0.000s
I20260925 09:57:09.531090 11084 raft_consensus.cc:348] T 2d661d5000894c45ac3d1c27f0e72889 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.531745 11084 raft_consensus.cc:374] T 2d661d5000894c45ac3d1c27f0e72889 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:09.532083 11084 raft_consensus.cc:729] T 2d661d5000894c45ac3d1c27f0e72889 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:09.532618 11082 tablet_bootstrap.cc:492] T e57065f2bb224a56ad1c3518a0c375da P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:09.532652 11084 consensus_queue.cc:260] T 2d661d5000894c45ac3d1c27f0e72889 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.532452 10384 catalog_manager.cc:5495] T e3b6b985a76b459dbd1de42ea20d3856 P be97ab5d75934a648d4ab76341193ec0 reported cstate change: term changed from 0 to 1, leader changed from <none> to be97ab5d75934a648d4ab76341193ec0 (127.10.9.3). New cstate: current_term: 1 leader_uuid: "be97ab5d75934a648d4ab76341193ec0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: HEALTHY } } }
I20260925 09:57:09.535086 11084 ts_tablet_manager.cc:1428] T 2d661d5000894c45ac3d1c27f0e72889 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260925 09:57:09.535085 10384 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet e3b6b985a76b459dbd1de42ea20d3856 (table default.Table_6ac888477dde46a4a748e2d8bbe5e0c2 [id=5e5f72cecad44b5e9bfd60320ee42243]). Expected version 1 got 0
I20260925 09:57:09.536391 11084 tablet_bootstrap.cc:492] T 8b281fc20e6f42239fee80d05da88ab6 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:09.537644 10384 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 1969ac471b6f471ebe8e048a96c22837 (table default.Table_4e09a615fb82487baa4783a5f6e2181e [id=f17a11d223204599afb24e2e9d1d9d9d]). Expected version 3 got 0
I20260925 09:57:09.537899 11082 tablet_bootstrap.cc:654] T e57065f2bb224a56ad1c3518a0c375da P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:09.538828 10388 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 8c13a2f288b14d03a3ca4fdd4889cbad
I20260925 09:57:09.540863 10664 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c13a2f288b14d03a3ca4fdd4889cbad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:09 UTC) from {username='slave'} at 127.0.0.1:37876
W20260925 09:57:09.541610 10390 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet aad3cb5e33f64cb885d51f4ea089307b consensus state has no leader
I20260925 09:57:09.542073 10963 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c13a2f288b14d03a3ca4fdd4889cbad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:09 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:09.543931 11085 consensus_queue.cc:260] T 59f0c6489a33433899bb6e62088121b6 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.552680 10818 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c13a2f288b14d03a3ca4fdd4889cbad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:09 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:09.554373 11245 tablet_replica.cc:330] T 8c13a2f288b14d03a3ca4fdd4889cbad P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:09.554788 11085 ts_tablet_manager.cc:1428] T 59f0c6489a33433899bb6e62088121b6 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.036s user 0.006s sys 0.001s
I20260925 09:57:09.558969 11085 tablet_bootstrap.cc:492] T d8be238061c146eb96d17a22c1b2104a P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:09.559969 11245 raft_consensus.cc:2229] T 8c13a2f288b14d03a3ca4fdd4889cbad P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:09.562611 11084 tablet_bootstrap.cc:654] T 8b281fc20e6f42239fee80d05da88ab6 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:09.561610 11245 raft_consensus.cc:2258] T 8c13a2f288b14d03a3ca4fdd4889cbad P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:09.564639 11245 ts_tablet_manager.cc:1905] T 8c13a2f288b14d03a3ca4fdd4889cbad P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 09:57:09.574002 10379 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet aad3cb5e33f64cb885d51f4ea089307b (table default.Table_2d7acf0d8022443090ba12cd87956502 [id=9d2f4f9f46394332bec9f8beda0d9595], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet aad3cb5e33f64cb885d51f4ea089307b consensus state has no leader
I20260925 09:57:09.575765 11085 tablet_bootstrap.cc:654] T d8be238061c146eb96d17a22c1b2104a P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:09.577719 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 8c13a2f288b14d03a3ca4fdd4889cbad because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8c13a2f288b14d03a3ca4fdd4889cbad already in progress: creating tablet
I20260925 09:57:09.580595 11134 raft_consensus.cc:482] T 2d661d5000894c45ac3d1c27f0e72889 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:09.581053 11134 raft_consensus.cc:504] T 2d661d5000894c45ac3d1c27f0e72889 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.583248 11134 leader_election.cc:288] T 2d661d5000894c45ac3d1c27f0e72889 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:09.585590 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d661d5000894c45ac3d1c27f0e72889" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:09.589512 10918 leader_election.cc:341] T 2d661d5000894c45ac3d1c27f0e72889 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:09.595333 10681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d661d5000894c45ac3d1c27f0e72889" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:09.597227 10919 leader_election.cc:341] T 2d661d5000894c45ac3d1c27f0e72889 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:09.597455 11084 tablet_bootstrap.cc:492] T 8b281fc20e6f42239fee80d05da88ab6 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:09.597630 10919 leader_election.cc:302] T 2d661d5000894c45ac3d1c27f0e72889 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:09.597982 11084 ts_tablet_manager.cc:1397] T 8b281fc20e6f42239fee80d05da88ab6 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.062s user 0.009s sys 0.000s
I20260925 09:57:09.598876 11195 raft_consensus.cc:482] T e3b6b985a76b459dbd1de42ea20d3856 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:09.599344 11195 raft_consensus.cc:504] T e3b6b985a76b459dbd1de42ea20d3856 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.600378 11084 raft_consensus.cc:348] T 8b281fc20e6f42239fee80d05da88ab6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.601019 11084 raft_consensus.cc:374] T 8b281fc20e6f42239fee80d05da88ab6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:09.601263 11084 raft_consensus.cc:729] T 8b281fc20e6f42239fee80d05da88ab6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:09.602886 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3b6b985a76b459dbd1de42ea20d3856" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:09.603909 11195 leader_election.cc:288] T e3b6b985a76b459dbd1de42ea20d3856 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:09.604115 11082 tablet_bootstrap.cc:492] T e57065f2bb224a56ad1c3518a0c375da P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:09.604809 11082 ts_tablet_manager.cc:1397] T e57065f2bb224a56ad1c3518a0c375da P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.072s user 0.006s sys 0.011s
I20260925 09:57:09.605168 11206 raft_consensus.cc:2726] T 2d661d5000894c45ac3d1c27f0e72889 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:09.605902 11084 consensus_queue.cc:260] T 8b281fc20e6f42239fee80d05da88ab6 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.607558 11082 raft_consensus.cc:348] T e57065f2bb224a56ad1c3518a0c375da P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.608434 11082 raft_consensus.cc:374] T e57065f2bb224a56ad1c3518a0c375da P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:09.608716 11082 raft_consensus.cc:729] T e57065f2bb224a56ad1c3518a0c375da P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:09.608814 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3b6b985a76b459dbd1de42ea20d3856" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:09.609561 10838 raft_consensus.cc:2379] T e3b6b985a76b459dbd1de42ea20d3856 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b8850818e2ee42a082802d0095c08d21 in current term 1: Already voted for candidate be97ab5d75934a648d4ab76341193ec0 in this term.
I20260925 09:57:09.609982 11085 tablet_bootstrap.cc:492] T d8be238061c146eb96d17a22c1b2104a P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:09.609403 11082 consensus_queue.cc:260] T e57065f2bb224a56ad1c3518a0c375da P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.610390 11085 ts_tablet_manager.cc:1397] T d8be238061c146eb96d17a22c1b2104a P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.052s user 0.009s sys 0.001s
I20260925 09:57:09.612298 11082 ts_tablet_manager.cc:1428] T e57065f2bb224a56ad1c3518a0c375da P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.007s user 0.003s sys 0.002s
I20260925 09:57:09.613070 11082 tablet_bootstrap.cc:492] T aad3cb5e33f64cb885d51f4ea089307b P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:09.613109 11085 raft_consensus.cc:348] T d8be238061c146eb96d17a22c1b2104a P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.613767 11085 raft_consensus.cc:374] T d8be238061c146eb96d17a22c1b2104a P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:09.614005 11085 raft_consensus.cc:729] T d8be238061c146eb96d17a22c1b2104a P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:09.614728 11085 consensus_queue.cc:260] T d8be238061c146eb96d17a22c1b2104a P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:09.615965 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 8c13a2f288b14d03a3ca4fdd4889cbad because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8c13a2f288b14d03a3ca4fdd4889cbad already in progress: creating tablet
I20260925 09:57:09.623824 11084 ts_tablet_manager.cc:1428] T 8b281fc20e6f42239fee80d05da88ab6 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.026s user 0.006s sys 0.000s
I20260925 09:57:09.623793 10621 leader_election.cc:302] T e3b6b985a76b459dbd1de42ea20d3856 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8850818e2ee42a082802d0095c08d21; no voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:09.624738 11215 raft_consensus.cc:3037] T e3b6b985a76b459dbd1de42ea20d3856 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:09.631143 11084 tablet_bootstrap.cc:492] T e48d86c20e9e4db990c23359ff1d1a6d P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:09.636245 10390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_70f0b47292de423097986b9a1c54f854"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:09.638370 10390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_70f0b47292de423097986b9a1c54f854 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:09.640457 10386 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_62ea48a919ec4159bcc86ebb048e3545" } modify_external_catalogs: true
I20260925 09:57:09.641800 11082 tablet_bootstrap.cc:654] T aad3cb5e33f64cb885d51f4ea089307b P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:09.635079 10388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_0af2360d761342cf9f057be5f265aa41"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:09.650274 11109 raft_consensus.cc:482] T e57065f2bb224a56ad1c3518a0c375da P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:09.650435 11084 tablet_bootstrap.cc:654] T e48d86c20e9e4db990c23359ff1d1a6d P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:09.650744 10388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0af2360d761342cf9f057be5f265aa41 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:09.650205 11195 raft_consensus.cc:482] T 04fafb9dcf274806b11b2598e73853e9 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:09.650722 11109 raft_consensus.cc:504] T e57065f2bb224a56ad1c3518a0c375da P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.651381 11195 raft_consensus.cc:504] T 04fafb9dcf274806b11b2598e73853e9 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.652606 11109 leader_election.cc:288] T e57065f2bb224a56ad1c3518a0c375da P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:09.653110 11195 leader_election.cc:288] T 04fafb9dcf274806b11b2598e73853e9 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:09.654065 10386 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_62ea48a919ec4159bcc86ebb048e3545" } modify_external_catalogs: true
I20260925 09:57:09.657893 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e57065f2bb224a56ad1c3518a0c375da" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:09.658862 10984 raft_consensus.cc:2454] T e57065f2bb224a56ad1c3518a0c375da P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:09.660399 10983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04fafb9dcf274806b11b2598e73853e9" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:09.661275 10983 raft_consensus.cc:2454] T 04fafb9dcf274806b11b2598e73853e9 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:09.664105 10619 leader_election.cc:302] T e57065f2bb224a56ad1c3518a0c375da P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:09.665481 11082 tablet_bootstrap.cc:492] T aad3cb5e33f64cb885d51f4ea089307b P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:09.665931 11082 ts_tablet_manager.cc:1397] T aad3cb5e33f64cb885d51f4ea089307b P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.053s user 0.008s sys 0.001s
I20260925 09:57:09.665723 10619 leader_election.cc:302] T 04fafb9dcf274806b11b2598e73853e9 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:09.667135 11195 raft_consensus.cc:2781] T e57065f2bb224a56ad1c3518a0c375da P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:09.667609 11195 raft_consensus.cc:482] T e57065f2bb224a56ad1c3518a0c375da P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:09.667944 11195 raft_consensus.cc:3037] T e57065f2bb224a56ad1c3518a0c375da P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:09.668828 11082 raft_consensus.cc:348] T aad3cb5e33f64cb885d51f4ea089307b P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.669641 11082 raft_consensus.cc:374] T aad3cb5e33f64cb885d51f4ea089307b P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:09.669884 11082 raft_consensus.cc:729] T aad3cb5e33f64cb885d51f4ea089307b P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:09.671665 11082 consensus_queue.cc:260] T aad3cb5e33f64cb885d51f4ea089307b P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.675451 10982 raft_consensus.cc:2454] T 931769e78f32467b8f27bacb286a7a60 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:09.676514 11211 raft_consensus.cc:2781] T 04fafb9dcf274806b11b2598e73853e9 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:09.676849 11211 raft_consensus.cc:482] T 04fafb9dcf274806b11b2598e73853e9 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:09.677163 11211 raft_consensus.cc:3037] T 04fafb9dcf274806b11b2598e73853e9 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:09.678355 11215 raft_consensus.cc:2726] T e3b6b985a76b459dbd1de42ea20d3856 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:09.683066 11082 ts_tablet_manager.cc:1428] T aad3cb5e33f64cb885d51f4ea089307b P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.017s user 0.006s sys 0.001s
I20260925 09:57:09.683900 11082 tablet_bootstrap.cc:492] T d8be238061c146eb96d17a22c1b2104a P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:09.696372 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e57065f2bb224a56ad1c3518a0c375da" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:09.699330 10619 leader_election.cc:302] T 931769e78f32467b8f27bacb286a7a60 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:09.699528 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04fafb9dcf274806b11b2598e73853e9" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:09.697512 10839 raft_consensus.cc:2454] T 931769e78f32467b8f27bacb286a7a60 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:09.703852 10837 raft_consensus.cc:2454] T 04fafb9dcf274806b11b2598e73853e9 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:09.704231 11134 consensus_queue.cc:1035] T e3b6b985a76b459dbd1de42ea20d3856 P be97ab5d75934a648d4ab76341193ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:09.713851 11249 consensus_queue.cc:1035] T 4be3f6e81ddf418fabc7b694a2e62f35 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:09.702198 11082 tablet_bootstrap.cc:654] T d8be238061c146eb96d17a22c1b2104a P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:09.715050 11109 raft_consensus.cc:2781] T 931769e78f32467b8f27bacb286a7a60 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:09.715615 11109 raft_consensus.cc:686] T 931769e78f32467b8f27bacb286a7a60 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:09.716452 11109 consensus_queue.cc:237] T 931769e78f32467b8f27bacb286a7a60 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.726347 11085 ts_tablet_manager.cc:1428] T d8be238061c146eb96d17a22c1b2104a P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.116s user 0.007s sys 0.000s
I20260925 09:57:09.731650 11085 tablet_bootstrap.cc:492] T e57065f2bb224a56ad1c3518a0c375da P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:09.739725 10385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_5b846352cfb74a06967986d1eabc2b7a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:09.741916 10385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5b846352cfb74a06967986d1eabc2b7a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:09.752079 11134 consensus_queue.cc:1035] T e3b6b985a76b459dbd1de42ea20d3856 P be97ab5d75934a648d4ab76341193ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:09.756381 11082 tablet_bootstrap.cc:492] T d8be238061c146eb96d17a22c1b2104a P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
W20260925 09:57:09.756248 10621 leader_election.cc:341] T e57065f2bb224a56ad1c3518a0c375da P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:09.756769 11082 ts_tablet_manager.cc:1397] T d8be238061c146eb96d17a22c1b2104a P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.073s user 0.014s sys 0.004s
I20260925 09:57:09.757936 11215 raft_consensus.cc:482] T 59f0c6489a33433899bb6e62088121b6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:09.758342 11215 raft_consensus.cc:504] T 59f0c6489a33433899bb6e62088121b6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.758868 11082 raft_consensus.cc:348] T d8be238061c146eb96d17a22c1b2104a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.759485 11082 raft_consensus.cc:374] T d8be238061c146eb96d17a22c1b2104a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:09.759708 11082 raft_consensus.cc:729] T d8be238061c146eb96d17a22c1b2104a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:09.760099 11215 leader_election.cc:288] T 59f0c6489a33433899bb6e62088121b6 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:09.760251 11082 consensus_queue.cc:260] T d8be238061c146eb96d17a22c1b2104a P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.761871 10982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59f0c6489a33433899bb6e62088121b6" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:09.762490 10982 raft_consensus.cc:2454] T 59f0c6489a33433899bb6e62088121b6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:09.762694 11082 ts_tablet_manager.cc:1428] T d8be238061c146eb96d17a22c1b2104a P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260925 09:57:09.763921 11082 tablet_bootstrap.cc:492] T 26b7c37959d042b48234acc79290936f P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:09.764349 10619 leader_election.cc:302] T 59f0c6489a33433899bb6e62088121b6 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:09.768252 11215 raft_consensus.cc:2781] T 59f0c6489a33433899bb6e62088121b6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:09.768632 11215 raft_consensus.cc:482] T 59f0c6489a33433899bb6e62088121b6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:09.768798 11082 tablet_bootstrap.cc:654] T 26b7c37959d042b48234acc79290936f P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:09.769016 11215 raft_consensus.cc:3037] T 59f0c6489a33433899bb6e62088121b6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:09.783723 11195 raft_consensus.cc:504] T e57065f2bb224a56ad1c3518a0c375da P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.798439 11134 raft_consensus.cc:482] T b7646a1cde1b45b9bf7928c3e13bcbd3 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:09.799082 11134 raft_consensus.cc:504] T b7646a1cde1b45b9bf7928c3e13bcbd3 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.802311 11134 leader_election.cc:288] T b7646a1cde1b45b9bf7928c3e13bcbd3 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:09.802248 10681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7646a1cde1b45b9bf7928c3e13bcbd3" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:09.803570 10919 leader_election.cc:341] T b7646a1cde1b45b9bf7928c3e13bcbd3 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:09.804687 10390 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_b9a40e813ba14ce89495f9edffeb4ee0" } modify_external_catalogs: true
I20260925 09:57:09.805939 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7646a1cde1b45b9bf7928c3e13bcbd3" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:09.808759 10918 leader_election.cc:341] T b7646a1cde1b45b9bf7928c3e13bcbd3 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:09.809124 10918 leader_election.cc:302] T b7646a1cde1b45b9bf7928c3e13bcbd3 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:09.810108 11134 raft_consensus.cc:2726] T b7646a1cde1b45b9bf7928c3e13bcbd3 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:09.818864 10390 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_b9a40e813ba14ce89495f9edffeb4ee0" } modify_external_catalogs: true
I20260925 09:57:09.808106 10385 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_6ac888477dde46a4a748e2d8bbe5e0c2" } new_table_name: "default.Table_44abad8e8def4bf7be1d01c50a435298" modify_external_catalogs: true
I20260925 09:57:09.832165 11167 consensus_queue.cc:1035] T a8be01fe42894b96bbdd5ce2403ff1f4 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:09.838876 10983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e57065f2bb224a56ad1c3518a0c375da" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:09.839742 10983 raft_consensus.cc:3037] T e57065f2bb224a56ad1c3518a0c375da P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:09.841853 10388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_8e0c5caa8df14066af83c49a50a172bf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:09.853005 11195 leader_election.cc:288] T e57065f2bb224a56ad1c3518a0c375da P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:09.853880 11195 raft_consensus.cc:482] T d48d4517c13049389c5b2ba1ab7b14bc P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:09.854867 11195 raft_consensus.cc:504] T d48d4517c13049389c5b2ba1ab7b14bc P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.823331 11245 ts_tablet_manager.cc:1918] T 8c13a2f288b14d03a3ca4fdd4889cbad P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:09.876845 10982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d48d4517c13049389c5b2ba1ab7b14bc" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:09.877920 10982 raft_consensus.cc:2454] T d48d4517c13049389c5b2ba1ab7b14bc P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:09.879142 10619 leader_election.cc:302] T d48d4517c13049389c5b2ba1ab7b14bc P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:09.880091 11109 raft_consensus.cc:2781] T d48d4517c13049389c5b2ba1ab7b14bc P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:09.880522 11109 raft_consensus.cc:482] T d48d4517c13049389c5b2ba1ab7b14bc P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:09.881067 11109 raft_consensus.cc:3037] T d48d4517c13049389c5b2ba1ab7b14bc P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:09.861459 11245 log.cc:1196] T 8c13a2f288b14d03a3ca4fdd4889cbad P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/8c13a2f288b14d03a3ca4fdd4889cbad
I20260925 09:57:09.888422 11195 leader_election.cc:288] T d48d4517c13049389c5b2ba1ab7b14bc P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:09.891569 11167 consensus_queue.cc:1035] T a8be01fe42894b96bbdd5ce2403ff1f4 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:09.897742 10983 raft_consensus.cc:2454] T e57065f2bb224a56ad1c3518a0c375da P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:09.909960 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59f0c6489a33433899bb6e62088121b6" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:09.910673 10838 raft_consensus.cc:2454] T 59f0c6489a33433899bb6e62088121b6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:09.899240 11211 raft_consensus.cc:504] T 04fafb9dcf274806b11b2598e73853e9 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.900949 10619 leader_election.cc:302] T e57065f2bb224a56ad1c3518a0c375da P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:09.918871 11245 ts_tablet_manager.cc:1939] T 8c13a2f288b14d03a3ca4fdd4889cbad P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:09.921875 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 8c13a2f288b14d03a3ca4fdd4889cbad (table default.Table_ecec3b643d1c4b76b700090f3a3cab06 [id=52fd37f9b470404ebea6e54191436b12]) successfully deleted
W20260925 09:57:09.927441 10388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8e0c5caa8df14066af83c49a50a172bf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:09.927604 11211 leader_election.cc:288] T 04fafb9dcf274806b11b2598e73853e9 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:09.928224 11211 raft_consensus.cc:2781] T e57065f2bb224a56ad1c3518a0c375da P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:09.928706 11211 raft_consensus.cc:686] T e57065f2bb224a56ad1c3518a0c375da P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:09.932690 10983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04fafb9dcf274806b11b2598e73853e9" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:09.933388 10983 raft_consensus.cc:3037] T 04fafb9dcf274806b11b2598e73853e9 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:09.941413 11084 tablet_bootstrap.cc:492] T e48d86c20e9e4db990c23359ff1d1a6d P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:09.941834 11084 ts_tablet_manager.cc:1397] T e48d86c20e9e4db990c23359ff1d1a6d P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.311s user 0.032s sys 0.033s
I20260925 09:57:09.913041 11263 raft_consensus.cc:482] T aad3cb5e33f64cb885d51f4ea089307b P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:09.943044 11263 raft_consensus.cc:504] T aad3cb5e33f64cb885d51f4ea089307b P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.955145 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e57065f2bb224a56ad1c3518a0c375da" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:09.957218 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d48d4517c13049389c5b2ba1ab7b14bc" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:09.957861 10838 raft_consensus.cc:2454] T d48d4517c13049389c5b2ba1ab7b14bc P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
W20260925 09:57:09.958966 10621 leader_election.cc:341] T e57065f2bb224a56ad1c3518a0c375da P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:09.946732 11084 raft_consensus.cc:348] T e48d86c20e9e4db990c23359ff1d1a6d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.959961 11084 raft_consensus.cc:374] T e48d86c20e9e4db990c23359ff1d1a6d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:09.960230 11084 raft_consensus.cc:729] T e48d86c20e9e4db990c23359ff1d1a6d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:09.961009 11084 consensus_queue.cc:260] T e48d86c20e9e4db990c23359ff1d1a6d P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.962146 11195 raft_consensus.cc:482] T d8be238061c146eb96d17a22c1b2104a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:09.965234 11195 raft_consensus.cc:504] T d8be238061c146eb96d17a22c1b2104a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.971758 11211 consensus_queue.cc:237] T e57065f2bb224a56ad1c3518a0c375da P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:09.975818 11195 leader_election.cc:288] T d8be238061c146eb96d17a22c1b2104a P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:09.977080 10982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8be238061c146eb96d17a22c1b2104a" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:09.978019 10982 raft_consensus.cc:2454] T d8be238061c146eb96d17a22c1b2104a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:09.979096 10619 leader_election.cc:302] T d8be238061c146eb96d17a22c1b2104a P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:09.980015 11195 raft_consensus.cc:2781] T d8be238061c146eb96d17a22c1b2104a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:09.980418 11195 raft_consensus.cc:482] T d8be238061c146eb96d17a22c1b2104a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:09.980799 11195 raft_consensus.cc:3037] T d8be238061c146eb96d17a22c1b2104a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:09.963241 11084 ts_tablet_manager.cc:1428] T e48d86c20e9e4db990c23359ff1d1a6d P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.021s user 0.005s sys 0.000s
I20260925 09:57:09.982510 11084 tablet_bootstrap.cc:492] T 0e882284a014455192375daad391efaf P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
W20260925 09:57:09.983843 10775 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 99669.9us
W20260925 09:57:09.970105 10379 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.60:36985, user_credentials={real_user=slave}} blocked reactor thread for 103039us
I20260925 09:57:10.006995 11215 raft_consensus.cc:504] T 59f0c6489a33433899bb6e62088121b6 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:10.009171 11215 leader_election.cc:288] T 59f0c6489a33433899bb6e62088121b6 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:10.005342 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04fafb9dcf274806b11b2598e73853e9" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:10.011778 10838 raft_consensus.cc:3037] T 04fafb9dcf274806b11b2598e73853e9 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:10.021040 11085 tablet_bootstrap.cc:654] T e57065f2bb224a56ad1c3518a0c375da P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:10.033003 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8be238061c146eb96d17a22c1b2104a" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:10.033620 10837 raft_consensus.cc:2454] T d8be238061c146eb96d17a22c1b2104a P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:10.034518 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59f0c6489a33433899bb6e62088121b6" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:10.035075 10839 raft_consensus.cc:3037] T 59f0c6489a33433899bb6e62088121b6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:10.038008 10983 raft_consensus.cc:2454] T 04fafb9dcf274806b11b2598e73853e9 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:10.042352 10982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59f0c6489a33433899bb6e62088121b6" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:10.043164 11206 raft_consensus.cc:482] T e48d86c20e9e4db990c23359ff1d1a6d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:10.043499 10619 leader_election.cc:302] T 04fafb9dcf274806b11b2598e73853e9 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:10.043697 11206 raft_consensus.cc:504] T e48d86c20e9e4db990c23359ff1d1a6d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:10.045761 11206 leader_election.cc:288] T e48d86c20e9e4db990c23359ff1d1a6d P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:10.046305 11211 raft_consensus.cc:2781] T 04fafb9dcf274806b11b2598e73853e9 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:10.046762 11211 raft_consensus.cc:686] T 04fafb9dcf274806b11b2598e73853e9 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:10.047873 11211 consensus_queue.cc:237] T 04fafb9dcf274806b11b2598e73853e9 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:10.049573 11263 leader_election.cc:288] T aad3cb5e33f64cb885d51f4ea089307b P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:10.042989 10982 raft_consensus.cc:3037] T 59f0c6489a33433899bb6e62088121b6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:10.051429 10680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e48d86c20e9e4db990c23359ff1d1a6d" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:10.053570 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e48d86c20e9e4db990c23359ff1d1a6d" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:10.057564 10919 leader_election.cc:341] T e48d86c20e9e4db990c23359ff1d1a6d P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:10.060227 10918 leader_election.cc:341] T e48d86c20e9e4db990c23359ff1d1a6d P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:10.060618 10918 leader_election.cc:302] T e48d86c20e9e4db990c23359ff1d1a6d P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:10.073694 11279 tablet.cc:1722] T 1969ac471b6f471ebe8e048a96c22837 P c43461c4ee854ae1ac52b873ccada318: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260925 09:57:10.077030 11282 tablet.cc:1722] T 1969ac471b6f471ebe8e048a96c22837 P be97ab5d75934a648d4ab76341193ec0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260925 09:57:10.082219 11084 tablet_bootstrap.cc:654] T 0e882284a014455192375daad391efaf P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:10.106443 11109 raft_consensus.cc:504] T d48d4517c13049389c5b2ba1ab7b14bc P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:10.108649 11109 leader_election.cc:288] T d48d4517c13049389c5b2ba1ab7b14bc P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:10.109300 10839 raft_consensus.cc:2454] T 59f0c6489a33433899bb6e62088121b6 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:10.118202 10963 tablet_service.cc:1467] Processing CreateTablet for tablet 4c4e8790282a49fd849a34ca11bef5bf (DEFAULT_TABLE table=default.Table_beb33698ec744e42b4f6ac2fdd8fe2b9 [id=9c778da25cde488a953f712100b2fbb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:10.119513 10664 tablet_service.cc:1467] Processing CreateTablet for tablet 4c4e8790282a49fd849a34ca11bef5bf (DEFAULT_TABLE table=default.Table_beb33698ec744e42b4f6ac2fdd8fe2b9 [id=9c778da25cde488a953f712100b2fbb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:10.120601 10664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c4e8790282a49fd849a34ca11bef5bf. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:10.122328 10774 outbound_call.cc:322] RPC callback for RPC call kudu.master.MasterService.Ping -> {remote=127.10.9.60:36985, user_credentials={real_user=slave}} blocked reactor thread for 89183.9us
I20260925 09:57:10.122970 10982 raft_consensus.cc:2454] T 59f0c6489a33433899bb6e62088121b6 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:10.124285 10621 leader_election.cc:302] T 59f0c6489a33433899bb6e62088121b6 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:10.127681 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d48d4517c13049389c5b2ba1ab7b14bc" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:10.128253 10837 raft_consensus.cc:3037] T d48d4517c13049389c5b2ba1ab7b14bc P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:10.127830 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aad3cb5e33f64cb885d51f4ea089307b" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:10.129401 11109 raft_consensus.cc:2781] T 59f0c6489a33433899bb6e62088121b6 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:10.128064 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d48d4517c13049389c5b2ba1ab7b14bc" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:10.130740 11109 raft_consensus.cc:686] T 59f0c6489a33433899bb6e62088121b6 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:10.131031 10984 raft_consensus.cc:3037] T d48d4517c13049389c5b2ba1ab7b14bc P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:10.131757 11109 consensus_queue.cc:237] T 59f0c6489a33433899bb6e62088121b6 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:10.136240 10983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aad3cb5e33f64cb885d51f4ea089307b" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:10.136865 10983 raft_consensus.cc:2454] T aad3cb5e33f64cb885d51f4ea089307b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:10.136889 10393 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 1969ac471b6f471ebe8e048a96c22837 (table default.Table_4e09a615fb82487baa4783a5f6e2181e [id=f17a11d223204599afb24e2e9d1d9d9d]). Expected version 3 got 0
I20260925 09:57:10.138235 10619 leader_election.cc:302] T aad3cb5e33f64cb885d51f4ea089307b P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:10.137840 10393 catalog_manager.cc:5495] T 4be3f6e81ddf418fabc7b694a2e62f35 P c43461c4ee854ae1ac52b873ccada318 reported cstate change: term changed from 0 to 1, leader changed from <none> to c43461c4ee854ae1ac52b873ccada318 (127.10.9.2). New cstate: current_term: 1 leader_uuid: "c43461c4ee854ae1ac52b873ccada318" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:10.138953 11211 raft_consensus.cc:2781] T aad3cb5e33f64cb885d51f4ea089307b P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:10.139254 11211 raft_consensus.cc:482] T aad3cb5e33f64cb885d51f4ea089307b P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:10.139628 11211 raft_consensus.cc:3037] T aad3cb5e33f64cb885d51f4ea089307b P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:10.151495 10959 tablet_service.cc:1467] Processing CreateTablet for tablet 25db96e00e6646eaa1190b85c989958f (DEFAULT_TABLE table=default.Table_9f1a5af3d45e4ed1b81a8484a2c336fb [id=0f93ec50f6a94cedaf6cee59e4bb4227]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:10.151865 10963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c4e8790282a49fd849a34ca11bef5bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:10.152558 10959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25db96e00e6646eaa1190b85c989958f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:10.152297 10666 tablet_service.cc:1467] Processing CreateTablet for tablet 25db96e00e6646eaa1190b85c989958f (DEFAULT_TABLE table=default.Table_9f1a5af3d45e4ed1b81a8484a2c336fb [id=0f93ec50f6a94cedaf6cee59e4bb4227]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:10.153369 10666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25db96e00e6646eaa1190b85c989958f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:10.155936 10667 tablet_service.cc:1467] Processing CreateTablet for tablet 83b0cf21f2a144719a2e13990e1921c0 (DEFAULT_TABLE table=default.Table_a0c3cadc623f413bb77d0ddb87cafa8c [id=e176feb88d2643cf85a1f1d821133b6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:10.156922 10667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83b0cf21f2a144719a2e13990e1921c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:10.157459 10962 tablet_service.cc:1467] Processing CreateTablet for tablet 83b0cf21f2a144719a2e13990e1921c0 (DEFAULT_TABLE table=default.Table_a0c3cadc623f413bb77d0ddb87cafa8c [id=e176feb88d2643cf85a1f1d821133b6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:10.158470 10962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83b0cf21f2a144719a2e13990e1921c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:10.139123 10393 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 4be3f6e81ddf418fabc7b694a2e62f35 (table default.Table_1e7379199917430eb9e027fc7bb2a4f9 [id=c2d0336fd84c4dbe95472f458b813fe6]). Expected version 1 got 0
I20260925 09:57:10.160775 10393 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet e3b6b985a76b459dbd1de42ea20d3856 (table default.Table_6ac888477dde46a4a748e2d8bbe5e0c2 [id=5e5f72cecad44b5e9bfd60320ee42243]). Expected version 1 got 0
I20260925 09:57:10.167929 11084 tablet_bootstrap.cc:492] T 0e882284a014455192375daad391efaf P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:10.168329 11084 ts_tablet_manager.cc:1397] T 0e882284a014455192375daad391efaf P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.186s user 0.011s sys 0.002s
I20260925 09:57:10.169919 11082 tablet_bootstrap.cc:492] T 26b7c37959d042b48234acc79290936f P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:10.170312 11082 ts_tablet_manager.cc:1397] T 26b7c37959d042b48234acc79290936f P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.407s user 0.032s sys 0.067s
I20260925 09:57:10.177714 10668 tablet_service.cc:1467] Processing CreateTablet for tablet 3fa527403d98425ea5cff9215d3a5e7f (DEFAULT_TABLE table=default.Table_fb1abc10cc744e82a45c0c93258c8305 [id=6e427b8eeb0d44c992c99c28f7e8af8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:10.178247 11082 raft_consensus.cc:348] T 26b7c37959d042b48234acc79290936f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:10.178999 11082 raft_consensus.cc:374] T 26b7c37959d042b48234acc79290936f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:10.179227 11082 raft_consensus.cc:729] T 26b7c37959d042b48234acc79290936f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:10.179782 11082 consensus_queue.cc:260] T 26b7c37959d042b48234acc79290936f P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:10.180440 10961 tablet_service.cc:1467] Processing CreateTablet for tablet 3fa527403d98425ea5cff9215d3a5e7f (DEFAULT_TABLE table=default.Table_fb1abc10cc744e82a45c0c93258c8305 [id=6e427b8eeb0d44c992c99c28f7e8af8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:10.180871 10668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fa527403d98425ea5cff9215d3a5e7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:10.181506 10961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fa527403d98425ea5cff9215d3a5e7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:10.190408 11082 ts_tablet_manager.cc:1428] T 26b7c37959d042b48234acc79290936f P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.020s user 0.007s sys 0.000s
I20260925 09:57:10.191627 11082 tablet_bootstrap.cc:492] T 8c13a2f288b14d03a3ca4fdd4889cbad P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:10.194067 11085 tablet_bootstrap.cc:492] T e57065f2bb224a56ad1c3518a0c375da P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
W20260925 09:57:10.195379 10621 leader_election.cc:341] T aad3cb5e33f64cb885d51f4ea089307b P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:10.194512 11085 ts_tablet_manager.cc:1397] T e57065f2bb224a56ad1c3518a0c375da P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.463s user 0.009s sys 0.002s
I20260925 09:57:10.161569 10393 catalog_manager.cc:5495] T a8be01fe42894b96bbdd5ce2403ff1f4 P c43461c4ee854ae1ac52b873ccada318 reported cstate change: term changed from 0 to 1, leader changed from <none> to c43461c4ee854ae1ac52b873ccada318 (127.10.9.2). New cstate: current_term: 1 leader_uuid: "c43461c4ee854ae1ac52b873ccada318" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:10.207727 11195 raft_consensus.cc:504] T d8be238061c146eb96d17a22c1b2104a P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:10.206506 10385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_6b025220a9aa49ecb47a7e17824f0a71"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:10.209702 11195 leader_election.cc:288] T d8be238061c146eb96d17a22c1b2104a P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
W20260925 09:57:10.210095 10385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6b025220a9aa49ecb47a7e17824f0a71 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:10.212726 10837 raft_consensus.cc:2454] T d48d4517c13049389c5b2ba1ab7b14bc P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:10.199930 11085 raft_consensus.cc:348] T e57065f2bb224a56ad1c3518a0c375da P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:10.214547 11085 raft_consensus.cc:374] T e57065f2bb224a56ad1c3518a0c375da P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:10.214803 11085 raft_consensus.cc:729] T e57065f2bb224a56ad1c3518a0c375da P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:10.215845 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8be238061c146eb96d17a22c1b2104a" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:10.216518 10837 raft_consensus.cc:3037] T d8be238061c146eb96d17a22c1b2104a P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:10.217290 10984 raft_consensus.cc:2454] T d48d4517c13049389c5b2ba1ab7b14bc P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:10.219694 11085 consensus_queue.cc:260] T e57065f2bb224a56ad1c3518a0c375da P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:10.224965 11085 ts_tablet_manager.cc:1428] T e57065f2bb224a56ad1c3518a0c375da P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.028s user 0.004s sys 0.000s
I20260925 09:57:10.225986 11085 tablet_bootstrap.cc:492] T 0a2bbe141e564a96be05de86f2993234 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:10.230291 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8be238061c146eb96d17a22c1b2104a" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:10.231603 10984 raft_consensus.cc:3037] T d8be238061c146eb96d17a22c1b2104a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:57:10.234432 10379 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.62:34455, user_credentials={real_user=slave}} blocked reactor thread for 136843us
I20260925 09:57:10.236804 11206 raft_consensus.cc:2726] T e48d86c20e9e4db990c23359ff1d1a6d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:10.238569 11084 raft_consensus.cc:348] T 0e882284a014455192375daad391efaf P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:10.239179 11084 raft_consensus.cc:374] T 0e882284a014455192375daad391efaf P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:10.239449 11084 raft_consensus.cc:729] T 0e882284a014455192375daad391efaf P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:10.240219 11084 consensus_queue.cc:260] T 0e882284a014455192375daad391efaf P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:10.242739 10918 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 182530us
I20260925 09:57:10.260802 11085 tablet_bootstrap.cc:654] T 0a2bbe141e564a96be05de86f2993234 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:10.283124 10818 tablet_service.cc:1467] Processing CreateTablet for tablet 4c4e8790282a49fd849a34ca11bef5bf (DEFAULT_TABLE table=default.Table_beb33698ec744e42b4f6ac2fdd8fe2b9 [id=9c778da25cde488a953f712100b2fbb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:10.284390 10818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c4e8790282a49fd849a34ca11bef5bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:10.244230 11084 ts_tablet_manager.cc:1428] T 0e882284a014455192375daad391efaf P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.076s user 0.004s sys 0.003s
I20260925 09:57:10.292397 11084 tablet_bootstrap.cc:492] T 25db96e00e6646eaa1190b85c989958f P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:10.293118 10388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_d586d012b3534ee79591128c9aac74ac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:10.295177 11282 tablet_replica.cc:1402] Time spent flushing the Log queue took a long time: real 0.217s user 0.000s sys 0.001s
I20260925 09:57:10.250886 10958 tablet_service.cc:1467] Processing CreateTablet for tablet c07c6205d1284a6fbb369c010a226da7 (DEFAULT_TABLE table=default.Table_861e5470a3b548419a9d96e59a1f9b0e [id=2d6bc0e2aaee4b1483b01fc678505bb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:10.304322 10958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c07c6205d1284a6fbb369c010a226da7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:10.308806 10837 raft_consensus.cc:2454] T d8be238061c146eb96d17a22c1b2104a P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:10.316891 10813 tablet_service.cc:1467] Processing CreateTablet for tablet 25db96e00e6646eaa1190b85c989958f (DEFAULT_TABLE table=default.Table_9f1a5af3d45e4ed1b81a8484a2c336fb [id=0f93ec50f6a94cedaf6cee59e4bb4227]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:10.317457 10814 tablet_service.cc:1467] Processing CreateTablet for tablet 3fa527403d98425ea5cff9215d3a5e7f (DEFAULT_TABLE table=default.Table_fb1abc10cc744e82a45c0c93258c8305 [id=6e427b8eeb0d44c992c99c28f7e8af8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:10.317956 10813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25db96e00e6646eaa1190b85c989958f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:10.318432 10814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fa527403d98425ea5cff9215d3a5e7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:10.320360 10819 tablet_service.cc:1467] Processing CreateTablet for tablet 83b0cf21f2a144719a2e13990e1921c0 (DEFAULT_TABLE table=default.Table_a0c3cadc623f413bb77d0ddb87cafa8c [id=e176feb88d2643cf85a1f1d821133b6c]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 09:57:10.321056 10388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d586d012b3534ee79591128c9aac74ac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:10.321313 10819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83b0cf21f2a144719a2e13990e1921c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:10.328562 11085 tablet_bootstrap.cc:492] T 0a2bbe141e564a96be05de86f2993234 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:10.315876 11084 tablet_bootstrap.cc:654] T 25db96e00e6646eaa1190b85c989958f P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:10.337093 11085 ts_tablet_manager.cc:1397] T 0a2bbe141e564a96be05de86f2993234 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.111s user 0.007s sys 0.006s
I20260925 09:57:10.339282 11085 raft_consensus.cc:348] T 0a2bbe141e564a96be05de86f2993234 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:10.339952 11085 raft_consensus.cc:374] T 0a2bbe141e564a96be05de86f2993234 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:10.340193 11085 raft_consensus.cc:729] T 0a2bbe141e564a96be05de86f2993234 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:10.342707 11085 consensus_queue.cc:260] T 0a2bbe141e564a96be05de86f2993234 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:10.346894 11085 ts_tablet_manager.cc:1428] T 0a2bbe141e564a96be05de86f2993234 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20260925 09:57:10.349671 11085 tablet_bootstrap.cc:492] T aad3cb5e33f64cb885d51f4ea089307b P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:10.356096 11085 tablet_bootstrap.cc:654] T aad3cb5e33f64cb885d51f4ea089307b P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:10.359951 10816 tablet_service.cc:1467] Processing CreateTablet for tablet c07c6205d1284a6fbb369c010a226da7 (DEFAULT_TABLE table=default.Table_861e5470a3b548419a9d96e59a1f9b0e [id=2d6bc0e2aaee4b1483b01fc678505bb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:10.360961 10816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c07c6205d1284a6fbb369c010a226da7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:10.379909 10958 tablet_service.cc:1467] Processing CreateTablet for tablet fd73d0d52350444499abae34e8c02fab (DEFAULT_TABLE table=default.Table_4c02305911f84ebb95187deaf905c14c [id=82e36829eee74495bec16d263e259c09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:10.381397 10958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd73d0d52350444499abae34e8c02fab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:10.384042 10386 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 0e882284a014455192375daad391efaf
I20260925 09:57:10.385599 10386 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:35140 (request call id 83) took 1107 ms. Trace:
I20260925 09:57:10.391909 10390 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 931769e78f32467b8f27bacb286a7a60
I20260925 09:57:10.398245 10963 tablet_service.cc:1514] Processing DeleteTablet for tablet 931769e78f32467b8f27bacb286a7a60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:09 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:10.399271 11245 tablet_replica.cc:330] T 931769e78f32467b8f27bacb286a7a60 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:10.400489 11245 raft_consensus.cc:2229] T 931769e78f32467b8f27bacb286a7a60 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:10.401182 11245 raft_consensus.cc:2258] T 931769e78f32467b8f27bacb286a7a60 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:10.405519 10838 raft_consensus.cc:2454] T 04fafb9dcf274806b11b2598e73853e9 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:10.405819 10962 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e882284a014455192375daad391efaf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:09 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:10.406525 11245 ts_tablet_manager.cc:1905] T 931769e78f32467b8f27bacb286a7a60 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:10.385937 10386 rpcz_store.cc:276] 0925 09:57:09.278080 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:09.528887 (+250807us) service_pool.cc:227] Handling call
0925 09:57:09.640324 (+111437us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2adb466 00007fabf061cb34 00007fabf061cac2 00007fabf061c9f1 00007fabf061c96c 00007fabf061bc22 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:09.654767 (+ 14443us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:57:09.659673 (+ 4906us) catalog_manager.cc:2752] Modifying in-memory table state
0925 09:57:09.659819 (+ 146us) catalog_manager.cc:2761] Locking tablets
0925 09:57:10.108434 (+448615us) catalog_manager.cc:2775] Removing table and tablets from system table
0925 09:57:10.383560 (+275126us) catalog_manager.cc:2793] Removing table from by-name map
0925 09:57:10.383819 (+ 259us) catalog_manager.cc:2809] Committing in-memory state
0925 09:57:10.383948 (+ 129us) catalog_manager.cc:2813] Aborting table tasks
0925 09:57:10.385540 (+ 1592us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:10.117320 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:10.117659 (+ 339us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:10.117683 (+ 24us) write_op.cc:435] Acquired schema lock
0925 09:57:10.117695 (+ 12us) tablet.cc:660] Decoding operations
0925 09:57:10.117983 (+ 288us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:10.118021 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:10.118042 (+ 21us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:10.118321 (+ 279us) tablet.cc:699] Row locks acquired
0925 09:57:10.118335 (+ 14us) write_op.cc:260] PREPARE: finished
0925 09:57:10.118439 (+ 104us) write_op.cc:270] Start()
0925 09:57:10.118516 (+ 77us) write_op.cc:276] Timestamp: P: 1790330230118393 usec, L: 0
0925 09:57:10.118532 (+ 16us) op_driver.cc:331] REPLICATION: starting
0925 09:57:10.118966 (+ 434us) log.cc:842] Serialized 721 byte log entry
0925 09:57:10.379763 (+260797us) op_driver.cc:444] REPLICATION: finished
0925 09:57:10.380638 (+ 875us) write_op.cc:301] APPLY: starting
0925 09:57:10.380697 (+ 59us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:10.380806 (+ 109us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:10.380897 (+ 91us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:10.381106 (+ 209us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:10.381259 (+ 153us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 09:57:10.381300 (+ 41us) write_op.cc:312] APPLY: finished
0925 09:57:10.381479 (+ 179us) log.cc:842] Serialized 28 byte log entry
0925 09:57:10.381618 (+ 139us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:10.381685 (+ 67us) write_op.cc:454] Released schema lock
0925 09:57:10.381717 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:10.381927 (+ 210us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:10.381951 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":253200640,"child_traces":[["op",{"apply.queue_time_us":715,"apply.run_cpu_time_us":1518,"apply.run_wall_time_us":1518,"num_ops":2,"prepare.queue_time_us":6715,"prepare.run_cpu_time_us":1892,"prepare.run_wall_time_us":1903,"raft.queue_time_us":371,"raft.run_cpu_time_us":11,"raft.run_wall_time_us":12,"replication_time_us":261138}]]}
W20260925 09:57:10.424806 10918 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 123043us
W20260925 09:57:10.427345 10379 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.62:34455, user_credentials={real_user=slave}} blocked reactor thread for 47778.8us
I20260925 09:57:10.451896 10815 tablet_service.cc:1514] Processing DeleteTablet for tablet 931769e78f32467b8f27bacb286a7a60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:09 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:10.455171 10814 tablet_service.cc:1467] Processing CreateTablet for tablet fd73d0d52350444499abae34e8c02fab (DEFAULT_TABLE table=default.Table_4c02305911f84ebb95187deaf905c14c [id=82e36829eee74495bec16d263e259c09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:10.456954 10814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd73d0d52350444499abae34e8c02fab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:10.430130 11287 tablet.cc:1722] T e3b6b985a76b459dbd1de42ea20d3856 P be97ab5d75934a648d4ab76341193ec0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:10.456089 10819 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e882284a014455192375daad391efaf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:09 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:10.465416 11085 tablet_bootstrap.cc:492] T aad3cb5e33f64cb885d51f4ea089307b P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:10.465809 11085 ts_tablet_manager.cc:1397] T aad3cb5e33f64cb885d51f4ea089307b P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.116s user 0.034s sys 0.028s
I20260925 09:57:10.467905 11085 raft_consensus.cc:348] T aad3cb5e33f64cb885d51f4ea089307b P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:10.468617 11085 raft_consensus.cc:374] T aad3cb5e33f64cb885d51f4ea089307b P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:10.468879 11085 raft_consensus.cc:729] T aad3cb5e33f64cb885d51f4ea089307b P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:10.469167 11292 tablet_replica.cc:330] T 931769e78f32467b8f27bacb286a7a60 P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:10.469954 11292 raft_consensus.cc:2229] T 931769e78f32467b8f27bacb286a7a60 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:10.470363 11292 raft_consensus.cc:2258] T 931769e78f32467b8f27bacb286a7a60 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:10.470129 10385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_0108f9fbea174357ae1e8d04519bbaf0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:10.470783 11084 tablet_bootstrap.cc:492] T 25db96e00e6646eaa1190b85c989958f P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:10.471135 11084 ts_tablet_manager.cc:1397] T 25db96e00e6646eaa1190b85c989958f P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.179s user 0.025s sys 0.017s
I20260925 09:57:10.473727 11279 tablet.cc:1722] T e3b6b985a76b459dbd1de42ea20d3856 P c43461c4ee854ae1ac52b873ccada318: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:10.473227 11084 raft_consensus.cc:348] T 25db96e00e6646eaa1190b85c989958f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:10.475620 11084 raft_consensus.cc:374] T 25db96e00e6646eaa1190b85c989958f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:10.475976 11084 raft_consensus.cc:729] T 25db96e00e6646eaa1190b85c989958f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:10.476568 11084 consensus_queue.cc:260] T 25db96e00e6646eaa1190b85c989958f P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:10.477293 11085 consensus_queue.cc:260] T aad3cb5e33f64cb885d51f4ea089307b P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:10.479228 11085 ts_tablet_manager.cc:1428] T aad3cb5e33f64cb885d51f4ea089307b P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.013s user 0.004s sys 0.000s
I20260925 09:57:10.481053 11084 ts_tablet_manager.cc:1428] T 25db96e00e6646eaa1190b85c989958f P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.010s user 0.002s sys 0.003s
I20260925 09:57:10.481937 11085 tablet_bootstrap.cc:492] T 8c13a2f288b14d03a3ca4fdd4889cbad P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:10.482725 11084 tablet_bootstrap.cc:492] T 4c4e8790282a49fd849a34ca11bef5bf P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:10.487527 11084 tablet_bootstrap.cc:654] T 4c4e8790282a49fd849a34ca11bef5bf P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:10.504444 11292 ts_tablet_manager.cc:1905] T 931769e78f32467b8f27bacb286a7a60 P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:10.505620 10386 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_0fe949ec62d547c9a9acbf96b374edcb" } new_table_name: "default.Table_9f64de2cbc4c4968bc0cfe51d0a9a487" modify_external_catalogs: true
W20260925 09:57:10.513341 10385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0108f9fbea174357ae1e8d04519bbaf0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:10.525110 11206 raft_consensus.cc:482] T 8b281fc20e6f42239fee80d05da88ab6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:10.525750 11206 raft_consensus.cc:504] T 8b281fc20e6f42239fee80d05da88ab6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:10.527900 11206 leader_election.cc:288] T 8b281fc20e6f42239fee80d05da88ab6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:10.529872 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b281fc20e6f42239fee80d05da88ab6" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:10.531319 10918 leader_election.cc:341] T 8b281fc20e6f42239fee80d05da88ab6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:10.535866 11084 tablet_bootstrap.cc:492] T 4c4e8790282a49fd849a34ca11bef5bf P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:10.536129 11085 tablet_bootstrap.cc:654] T 8c13a2f288b14d03a3ca4fdd4889cbad P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:10.536427 11084 ts_tablet_manager.cc:1397] T 4c4e8790282a49fd849a34ca11bef5bf P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.054s user 0.036s sys 0.006s
I20260925 09:57:10.538782 11084 raft_consensus.cc:348] T 4c4e8790282a49fd849a34ca11bef5bf P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:10.539467 11084 raft_consensus.cc:374] T 4c4e8790282a49fd849a34ca11bef5bf P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:10.539790 11084 raft_consensus.cc:729] T 4c4e8790282a49fd849a34ca11bef5bf P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:10.540422 11084 consensus_queue.cc:260] T 4c4e8790282a49fd849a34ca11bef5bf P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:10.543533 11084 ts_tablet_manager.cc:1428] T 4c4e8790282a49fd849a34ca11bef5bf P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260925 09:57:10.546090 11084 tablet_bootstrap.cc:492] T 83b0cf21f2a144719a2e13990e1921c0 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:10.551128 11084 tablet_bootstrap.cc:654] T 83b0cf21f2a144719a2e13990e1921c0 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:10.562261 11084 tablet_bootstrap.cc:492] T 83b0cf21f2a144719a2e13990e1921c0 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:10.562703 11084 ts_tablet_manager.cc:1397] T 83b0cf21f2a144719a2e13990e1921c0 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.006s
I20260925 09:57:10.564810 11084 raft_consensus.cc:348] T 83b0cf21f2a144719a2e13990e1921c0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:10.565582 11084 raft_consensus.cc:374] T 83b0cf21f2a144719a2e13990e1921c0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:10.565864 11084 raft_consensus.cc:729] T 83b0cf21f2a144719a2e13990e1921c0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:10.566449 11084 consensus_queue.cc:260] T 83b0cf21f2a144719a2e13990e1921c0 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:10.570385 11084 ts_tablet_manager.cc:1428] T 83b0cf21f2a144719a2e13990e1921c0 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.007s user 0.000s sys 0.004s
I20260925 09:57:10.570544 11085 tablet_bootstrap.cc:492] T 8c13a2f288b14d03a3ca4fdd4889cbad P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:10.570960 11085 ts_tablet_manager.cc:1397] T 8c13a2f288b14d03a3ca4fdd4889cbad P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.089s user 0.008s sys 0.003s
I20260925 09:57:10.571226 11084 tablet_bootstrap.cc:492] T c07c6205d1284a6fbb369c010a226da7 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:10.576951 11084 tablet_bootstrap.cc:654] T c07c6205d1284a6fbb369c010a226da7 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:10.576457 11281 tablet.cc:1722] T 1969ac471b6f471ebe8e048a96c22837 P b8850818e2ee42a082802d0095c08d21: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260925 09:57:10.573683 11085 raft_consensus.cc:348] T 8c13a2f288b14d03a3ca4fdd4889cbad P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:10.577839 11272 raft_consensus.cc:482] T 0a2bbe141e564a96be05de86f2993234 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:10.578280 11272 raft_consensus.cc:504] T 0a2bbe141e564a96be05de86f2993234 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:10.578696 11085 raft_consensus.cc:374] T 8c13a2f288b14d03a3ca4fdd4889cbad P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:10.578928 11085 raft_consensus.cc:729] T 8c13a2f288b14d03a3ca4fdd4889cbad P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:10.580046 11272 leader_election.cc:288] T 0a2bbe141e564a96be05de86f2993234 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:10.588344 11084 tablet_bootstrap.cc:492] T c07c6205d1284a6fbb369c010a226da7 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:10.588717 11084 ts_tablet_manager.cc:1397] T c07c6205d1284a6fbb369c010a226da7 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.018s user 0.011s sys 0.003s
I20260925 09:57:10.590785 10665 tablet_service.cc:1467] Processing CreateTablet for tablet fd73d0d52350444499abae34e8c02fab (DEFAULT_TABLE table=default.Table_4c02305911f84ebb95187deaf905c14c [id=82e36829eee74495bec16d263e259c09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:10.588866 11085 consensus_queue.cc:260] T 8c13a2f288b14d03a3ca4fdd4889cbad P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:10.591378 11084 raft_consensus.cc:348] T c07c6205d1284a6fbb369c010a226da7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:10.592175 11084 raft_consensus.cc:374] T c07c6205d1284a6fbb369c010a226da7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:10.592536 11084 raft_consensus.cc:729] T c07c6205d1284a6fbb369c010a226da7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:10.593127 11084 consensus_queue.cc:260] T c07c6205d1284a6fbb369c010a226da7 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:10.596155 11084 ts_tablet_manager.cc:1428] T c07c6205d1284a6fbb369c010a226da7 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260925 09:57:10.596837 11084 tablet_bootstrap.cc:492] T fd73d0d52350444499abae34e8c02fab P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:10.597885 10387 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 1969ac471b6f471ebe8e048a96c22837 (table default.Table_4e09a615fb82487baa4783a5f6e2181e [id=f17a11d223204599afb24e2e9d1d9d9d]). Expected version 3 got 0
I20260925 09:57:10.598490 10387 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 4be3f6e81ddf418fabc7b694a2e62f35 (table default.Table_1e7379199917430eb9e027fc7bb2a4f9 [id=c2d0336fd84c4dbe95472f458b813fe6]). Expected version 1 got 0
I20260925 09:57:10.598546 10659 tablet_service.cc:1514] Processing DeleteTablet for tablet 931769e78f32467b8f27bacb286a7a60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:09 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:10.599694 10661 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e882284a014455192375daad391efaf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:09 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:10.600508 10663 tablet_service.cc:1467] Processing CreateTablet for tablet c07c6205d1284a6fbb369c010a226da7 (DEFAULT_TABLE table=default.Table_861e5470a3b548419a9d96e59a1f9b0e [id=2d6bc0e2aaee4b1483b01fc678505bb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:10.471951 10619 leader_election.cc:302] T d48d4517c13049389c5b2ba1ab7b14bc P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:10.601569 11084 tablet_bootstrap.cc:654] T fd73d0d52350444499abae34e8c02fab P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:10.601424 10619 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 375109us
W20260925 09:57:10.601905 10621 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 373130us
I20260925 09:57:10.603099 10982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a2bbe141e564a96be05de86f2993234" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:10.603699 10982 raft_consensus.cc:2454] T 0a2bbe141e564a96be05de86f2993234 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:10.604081 11109 raft_consensus.cc:2781] T d48d4517c13049389c5b2ba1ab7b14bc P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:10.605288 10619 leader_election.cc:302] T 0a2bbe141e564a96be05de86f2993234 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:10.612542 11211 raft_consensus.cc:504] T aad3cb5e33f64cb885d51f4ea089307b P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:10.613601 11085 ts_tablet_manager.cc:1428] T 8c13a2f288b14d03a3ca4fdd4889cbad P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.042s user 0.003s sys 0.001s
I20260925 09:57:10.614337 11245 ts_tablet_manager.cc:1918] T 931769e78f32467b8f27bacb286a7a60 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:10.614472 11085 tablet_bootstrap.cc:492] T 26b7c37959d042b48234acc79290936f P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:10.614480 11211 leader_election.cc:288] T aad3cb5e33f64cb885d51f4ea089307b P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:10.614699 11245 log.cc:1196] T 931769e78f32467b8f27bacb286a7a60 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/931769e78f32467b8f27bacb286a7a60
I20260925 09:57:10.616794 11245 ts_tablet_manager.cc:1939] T 931769e78f32467b8f27bacb286a7a60 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:10.629204 10984 raft_consensus.cc:2454] T d8be238061c146eb96d17a22c1b2104a P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:10.632781 11082 tablet_bootstrap.cc:654] T 8c13a2f288b14d03a3ca4fdd4889cbad P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:10.634263 11085 tablet_bootstrap.cc:654] T 26b7c37959d042b48234acc79290936f P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:10.617895 10390 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet e48d86c20e9e4db990c23359ff1d1a6d (table default.Table_3b9695593bd6426e8932f7b11f01d1ac [id=c524d4b2bbb1495cbe85a6f1b3d595d2]). Expected version 1 got 0
I20260925 09:57:10.636790 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a2bbe141e564a96be05de86f2993234" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:10.639037 10838 raft_consensus.cc:2454] T 0a2bbe141e564a96be05de86f2993234 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:10.640836 10390 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet e3b6b985a76b459dbd1de42ea20d3856 (table default.Table_44abad8e8def4bf7be1d01c50a435298 [id=5e5f72cecad44b5e9bfd60320ee42243]). Expected version 2 got 0
W20260925 09:57:10.643803 10378 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet e48d86c20e9e4db990c23359ff1d1a6d (table default.Table_3b9695593bd6426e8932f7b11f01d1ac [id=c524d4b2bbb1495cbe85a6f1b3d595d2], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet e48d86c20e9e4db990c23359ff1d1a6d consensus state has no leader
I20260925 09:57:10.659775 10390 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet aad3cb5e33f64cb885d51f4ea089307b (table default.Table_2d7acf0d8022443090ba12cd87956502 [id=9d2f4f9f46394332bec9f8beda0d9595]). Expected version 1 got 0
I20260925 09:57:10.662269 11249 raft_consensus.cc:482] T 8c13a2f288b14d03a3ca4fdd4889cbad P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:10.662755 11249 raft_consensus.cc:504] T 8c13a2f288b14d03a3ca4fdd4889cbad P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:10.665213 11249 leader_election.cc:288] T 8c13a2f288b14d03a3ca4fdd4889cbad P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:10.666683 10390 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 4be3f6e81ddf418fabc7b694a2e62f35 (table default.Table_1e7379199917430eb9e027fc7bb2a4f9 [id=c2d0336fd84c4dbe95472f458b813fe6]). Expected version 1 got 0
W20260925 09:57:10.690369 10772 leader_election.cc:341] T 8c13a2f288b14d03a3ca4fdd4889cbad P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 8c13a2f288b14d03a3ca4fdd4889cbad
I20260925 09:57:10.702698 11085 tablet_bootstrap.cc:492] T 26b7c37959d042b48234acc79290936f P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:10.704239 11085 ts_tablet_manager.cc:1397] T 26b7c37959d042b48234acc79290936f P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.090s user 0.013s sys 0.010s
I20260925 09:57:10.704739 10390 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 1969ac471b6f471ebe8e048a96c22837 (table default.Table_4e09a615fb82487baa4783a5f6e2181e [id=f17a11d223204599afb24e2e9d1d9d9d]). Expected version 3 got 0
W20260925 09:57:10.708541 10386 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b7646a1cde1b45b9bf7928c3e13bcbd3 consensus state has no leader
I20260925 09:57:10.713915 11245 tablet_replica.cc:330] T 0e882284a014455192375daad391efaf P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:10.715994 11245 raft_consensus.cc:2229] T 0e882284a014455192375daad391efaf P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:10.716441 11245 raft_consensus.cc:2258] T 0e882284a014455192375daad391efaf P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:10.718080 11245 ts_tablet_manager.cc:1905] T 0e882284a014455192375daad391efaf P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:10.718206 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 931769e78f32467b8f27bacb286a7a60 (table default.Table_b9a40e813ba14ce89495f9edffeb4ee0 [id=68697340a8b046d79b795c15a68e1c6f]) successfully deleted
I20260925 09:57:10.725477 11287 tablet.cc:1722] T e3b6b985a76b459dbd1de42ea20d3856 P be97ab5d75934a648d4ab76341193ec0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 09:57:10.731192 11279 tablet.cc:1722] T e3b6b985a76b459dbd1de42ea20d3856 P c43461c4ee854ae1ac52b873ccada318: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 09:57:10.732088 11085 raft_consensus.cc:348] T 26b7c37959d042b48234acc79290936f P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:10.732687 11085 raft_consensus.cc:374] T 26b7c37959d042b48234acc79290936f P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:10.732149 10385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_d3efd9bb6b174cdfbdbd1af72107b558"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:10.732926 11085 raft_consensus.cc:729] T 26b7c37959d042b48234acc79290936f P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:10.733520 11085 consensus_queue.cc:260] T 26b7c37959d042b48234acc79290936f P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:10.734040 10385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d3efd9bb6b174cdfbdbd1af72107b558 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:10.737744 10386 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_faa21794a80c407f800038e583c08f16" } modify_external_catalogs: true
I20260925 09:57:10.738420 10386 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_faa21794a80c407f800038e583c08f16" } modify_external_catalogs: true
I20260925 09:57:10.738883 11085 ts_tablet_manager.cc:1428] T 26b7c37959d042b48234acc79290936f P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.030s user 0.002s sys 0.000s
I20260925 09:57:10.742475 11085 tablet_bootstrap.cc:492] T 8b281fc20e6f42239fee80d05da88ab6 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:10.687064 10683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b281fc20e6f42239fee80d05da88ab6" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:10.745179 10379 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.9.60:36985: connect: Connection refused (error 111) [suppressed 9 similar messages]
I20260925 09:57:10.749224 11085 tablet_bootstrap.cc:654] T 8b281fc20e6f42239fee80d05da88ab6 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:10.769161 10619 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 163947us
I20260925 09:57:10.775797 11292 ts_tablet_manager.cc:1918] T 931769e78f32467b8f27bacb286a7a60 P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:10.779809 11292 log.cc:1196] T 931769e78f32467b8f27bacb286a7a60 P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/931769e78f32467b8f27bacb286a7a60
I20260925 09:57:10.784859 11292 ts_tablet_manager.cc:1939] T 931769e78f32467b8f27bacb286a7a60 P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:10.784965 10385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_7eaf247fb74f497d80a88b67fa301cab"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:10.787185 10385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7eaf247fb74f497d80a88b67fa301cab in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 09:57:10.797262 10919 leader_election.cc:341] T 8b281fc20e6f42239fee80d05da88ab6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:10.797693 10919 leader_election.cc:302] T 8b281fc20e6f42239fee80d05da88ab6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:10.798833 11134 raft_consensus.cc:2726] T 8b281fc20e6f42239fee80d05da88ab6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:10.827627 11272 raft_consensus.cc:2781] T 0a2bbe141e564a96be05de86f2993234 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:10.831645 11109 raft_consensus.cc:686] T d48d4517c13049389c5b2ba1ab7b14bc P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
W20260925 09:57:10.839713 10379 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 -> Peer b8be057e0b6d40a3a4b90f2b1f4a0fa2 (127.10.9.60:36985): Couldn't send request to peer b8be057e0b6d40a3a4b90f2b1f4a0fa2. Status: Network error: Client connection negotiation failed: client connection to 127.10.9.60:36985: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260925 09:57:10.840270 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 931769e78f32467b8f27bacb286a7a60 (table default.Table_b9a40e813ba14ce89495f9edffeb4ee0 [id=68697340a8b046d79b795c15a68e1c6f]) successfully deleted
W20260925 09:57:10.840902 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 0e882284a014455192375daad391efaf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0e882284a014455192375daad391efaf already in progress: creating tablet
I20260925 09:57:10.843931 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aad3cb5e33f64cb885d51f4ea089307b" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:10.844782 10984 raft_consensus.cc:3037] T aad3cb5e33f64cb885d51f4ea089307b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:10.828253 11272 raft_consensus.cc:482] T 0a2bbe141e564a96be05de86f2993234 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:10.843106 10619 leader_election.cc:302] T d8be238061c146eb96d17a22c1b2104a P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:10.833113 11109 consensus_queue.cc:237] T d48d4517c13049389c5b2ba1ab7b14bc P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:10.858736 10683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c13a2f288b14d03a3ca4fdd4889cbad" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:10.852696 10665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd73d0d52350444499abae34e8c02fab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:10.862308 10663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c07c6205d1284a6fbb369c010a226da7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:10.867975 10619 consensus_peers.cc:487] T 931769e78f32467b8f27bacb286a7a60 P b8850818e2ee42a082802d0095c08d21 -> Peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Couldn't send request to peer be97ab5d75934a648d4ab76341193ec0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 931769e78f32467b8f27bacb286a7a60. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:10.854308 11298 tablet_replica.cc:330] T 931769e78f32467b8f27bacb286a7a60 P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:10.878090 11084 tablet_bootstrap.cc:492] T fd73d0d52350444499abae34e8c02fab P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:10.878484 11084 ts_tablet_manager.cc:1397] T fd73d0d52350444499abae34e8c02fab P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.282s user 0.015s sys 0.042s
I20260925 09:57:10.880743 11084 raft_consensus.cc:348] T fd73d0d52350444499abae34e8c02fab P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:10.881377 11084 raft_consensus.cc:374] T fd73d0d52350444499abae34e8c02fab P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:10.881606 11084 raft_consensus.cc:729] T fd73d0d52350444499abae34e8c02fab P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:10.882267 11084 consensus_queue.cc:260] T fd73d0d52350444499abae34e8c02fab P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:10.884519 11084 ts_tablet_manager.cc:1428] T fd73d0d52350444499abae34e8c02fab P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260925 09:57:10.885492 11084 tablet_bootstrap.cc:492] T 3fa527403d98425ea5cff9215d3a5e7f P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:10.896458 11300 raft_consensus.cc:2781] T d8be238061c146eb96d17a22c1b2104a P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:10.899643 10984 raft_consensus.cc:2454] T aad3cb5e33f64cb885d51f4ea089307b P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:10.847415 11272 raft_consensus.cc:3037] T 0a2bbe141e564a96be05de86f2993234 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:10.922469 10388 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 4be3f6e81ddf418fabc7b694a2e62f35 (table default.Table_1e7379199917430eb9e027fc7bb2a4f9 [id=c2d0336fd84c4dbe95472f458b813fe6]). Expected version 1 got 0
I20260925 09:57:10.923045 10816 tablet_service.cc:1467] Processing CreateTablet for tablet 833fac37485141f6a18104eaea907b0e (DEFAULT_TABLE table=default.Table_8e0c5caa8df14066af83c49a50a172bf [id=fc3b3c18f00c42559612468a21e96694]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:10.924023 10388 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet e3b6b985a76b459dbd1de42ea20d3856 (table default.Table_44abad8e8def4bf7be1d01c50a435298 [id=5e5f72cecad44b5e9bfd60320ee42243]). Expected version 2 got 0
I20260925 09:57:10.924201 10816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 833fac37485141f6a18104eaea907b0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:10.940807 10667 tablet_service.cc:1467] Processing CreateTablet for tablet 833fac37485141f6a18104eaea907b0e (DEFAULT_TABLE table=default.Table_8e0c5caa8df14066af83c49a50a172bf [id=fc3b3c18f00c42559612468a21e96694]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:10.942029 10667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 833fac37485141f6a18104eaea907b0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:10.943718 10619 leader_election.cc:302] T aad3cb5e33f64cb885d51f4ea089307b P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:10.952600 11245 ts_tablet_manager.cc:1918] T 0e882284a014455192375daad391efaf P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:10.953078 11245 log.cc:1196] T 0e882284a014455192375daad391efaf P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/0e882284a014455192375daad391efaf
I20260925 09:57:10.953852 11245 ts_tablet_manager.cc:1939] T 0e882284a014455192375daad391efaf P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:10.959893 11298 raft_consensus.cc:2229] T 931769e78f32467b8f27bacb286a7a60 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:10.960480 11084 tablet_bootstrap.cc:654] T 3fa527403d98425ea5cff9215d3a5e7f P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:10.967239 11082 tablet_bootstrap.cc:492] T 8c13a2f288b14d03a3ca4fdd4889cbad P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:10.972916 10961 tablet_service.cc:1467] Processing CreateTablet for tablet 833fac37485141f6a18104eaea907b0e (DEFAULT_TABLE table=default.Table_8e0c5caa8df14066af83c49a50a172bf [id=fc3b3c18f00c42559612468a21e96694]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:10.974017 11082 ts_tablet_manager.cc:1397] T 8c13a2f288b14d03a3ca4fdd4889cbad P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.783s user 0.009s sys 0.011s
I20260925 09:57:10.974099 10961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 833fac37485141f6a18104eaea907b0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:10.953799 11211 consensus_queue.cc:1035] T 6b03e5aad59e46a6b0e43b9cb36278d7 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20260925 09:57:10.985071 11300 raft_consensus.cc:686] T d8be238061c146eb96d17a22c1b2104a P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:10.990100 10385 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 1969ac471b6f471ebe8e048a96c22837 (table default.Table_4e09a615fb82487baa4783a5f6e2181e [id=f17a11d223204599afb24e2e9d1d9d9d]). Expected version 3 got 0
I20260925 09:57:10.992717 11301 raft_consensus.cc:482] T 26b7c37959d042b48234acc79290936f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:10.993206 11301 raft_consensus.cc:504] T 26b7c37959d042b48234acc79290936f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:10.996398 10958 tablet_service.cc:1467] Processing CreateTablet for tablet 69f84509cafc44dd9444218738ebc2fe (DEFAULT_TABLE table=default.Table_5b846352cfb74a06967986d1eabc2b7a [id=036df81a04334b7eb71d2bba8e74dad2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:10.996791 11298 pending_rounds.cc:62] T 931769e78f32467b8f27bacb286a7a60 P b8850818e2ee42a082802d0095c08d21: Trying to abort 1 pending ops.
I20260925 09:57:10.997032 11298 pending_rounds.cc:68] T 931769e78f32467b8f27bacb286a7a60 P b8850818e2ee42a082802d0095c08d21: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192621026533376 op_type: NO_OP noop_request { }
I20260925 09:57:10.997442 11298 raft_consensus.cc:2866] T 931769e78f32467b8f27bacb286a7a60 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:57:10.997640 10958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69f84509cafc44dd9444218738ebc2fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:10.997802 11298 raft_consensus.cc:2258] T 931769e78f32467b8f27bacb286a7a60 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:10.998246 11301 leader_election.cc:288] T 26b7c37959d042b48234acc79290936f P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:10.996272 11211 raft_consensus.cc:2781] T aad3cb5e33f64cb885d51f4ea089307b P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:10.997931 11167 raft_consensus.cc:482] T e57065f2bb224a56ad1c3518a0c375da P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:11.000543 11082 raft_consensus.cc:348] T 8c13a2f288b14d03a3ca4fdd4889cbad P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:10.999644 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aad3cb5e33f64cb885d51f4ea089307b" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:11.001174 11082 raft_consensus.cc:374] T 8c13a2f288b14d03a3ca4fdd4889cbad P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:11.001421 11082 raft_consensus.cc:729] T 8c13a2f288b14d03a3ca4fdd4889cbad P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:11.001619 10838 raft_consensus.cc:3037] T aad3cb5e33f64cb885d51f4ea089307b P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:11.003690 11082 consensus_queue.cc:260] T 8c13a2f288b14d03a3ca4fdd4889cbad P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:10.998916 11300 consensus_queue.cc:237] T d8be238061c146eb96d17a22c1b2104a P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:10.986873 10619 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 85619.4us
I20260925 09:57:11.010452 11167 raft_consensus.cc:504] T e57065f2bb224a56ad1c3518a0c375da P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:11.010645 10664 tablet_service.cc:1467] Processing CreateTablet for tablet 69f84509cafc44dd9444218738ebc2fe (DEFAULT_TABLE table=default.Table_5b846352cfb74a06967986d1eabc2b7a [id=036df81a04334b7eb71d2bba8e74dad2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:11.011713 10664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69f84509cafc44dd9444218738ebc2fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:11.012343 11167 leader_election.cc:288] T e57065f2bb224a56ad1c3518a0c375da P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:11.014678 10981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e57065f2bb224a56ad1c3518a0c375da" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:11.014887 10982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26b7c37959d042b48234acc79290936f" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:11.015522 10981 raft_consensus.cc:2379] T e57065f2bb224a56ad1c3518a0c375da P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c43461c4ee854ae1ac52b873ccada318 in current term 1: Already voted for candidate b8850818e2ee42a082802d0095c08d21 in this term.
I20260925 09:57:11.015415 10982 raft_consensus.cc:2454] T 26b7c37959d042b48234acc79290936f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:11.026854 11082 ts_tablet_manager.cc:1428] T 8c13a2f288b14d03a3ca4fdd4889cbad P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.053s user 0.005s sys 0.000s
I20260925 09:57:11.028959 10619 leader_election.cc:302] T 26b7c37959d042b48234acc79290936f P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:11.030772 11299 raft_consensus.cc:2781] T 26b7c37959d042b48234acc79290936f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:11.031113 11299 raft_consensus.cc:482] T 26b7c37959d042b48234acc79290936f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:11.031479 11299 raft_consensus.cc:3037] T 26b7c37959d042b48234acc79290936f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:57:11.032490 10775 leader_election.cc:341] T 8c13a2f288b14d03a3ca4fdd4889cbad P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:11.033047 10775 leader_election.cc:302] T 8c13a2f288b14d03a3ca4fdd4889cbad P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:11.033937 11249 raft_consensus.cc:2726] T 8c13a2f288b14d03a3ca4fdd4889cbad P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:11.004619 11211 raft_consensus.cc:686] T aad3cb5e33f64cb885d51f4ea089307b P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:11.036841 11085 tablet_bootstrap.cc:492] T 8b281fc20e6f42239fee80d05da88ab6 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:11.037249 11085 ts_tablet_manager.cc:1397] T 8b281fc20e6f42239fee80d05da88ab6 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.295s user 0.015s sys 0.023s
I20260925 09:57:11.042285 11082 tablet_bootstrap.cc:492] T 8b281fc20e6f42239fee80d05da88ab6 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:11.049129 10838 raft_consensus.cc:2454] T aad3cb5e33f64cb885d51f4ea089307b P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:11.053560 10814 tablet_service.cc:1467] Processing CreateTablet for tablet 69f84509cafc44dd9444218738ebc2fe (DEFAULT_TABLE table=default.Table_5b846352cfb74a06967986d1eabc2b7a [id=036df81a04334b7eb71d2bba8e74dad2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:11.054638 10814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69f84509cafc44dd9444218738ebc2fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:11.078233 10666 tablet_service.cc:1467] Processing CreateTablet for tablet 3be0eccc65564bfba3d6e5c398c61fb6 (DEFAULT_TABLE table=default.Table_0af2360d761342cf9f057be5f265aa41 [id=99ba81c4d0dd497398e048a05759419a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:11.080190 10815 tablet_service.cc:1467] Processing CreateTablet for tablet 3be0eccc65564bfba3d6e5c398c61fb6 (DEFAULT_TABLE table=default.Table_0af2360d761342cf9f057be5f265aa41 [id=99ba81c4d0dd497398e048a05759419a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:11.081243 10815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3be0eccc65564bfba3d6e5c398c61fb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:11.055352 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 0e882284a014455192375daad391efaf (table default.Table_62ea48a919ec4159bcc86ebb048e3545 [id=051a096f481f4f20a59081fbd08b3b1f]) successfully deleted
I20260925 09:57:11.039237 11085 raft_consensus.cc:348] T 8b281fc20e6f42239fee80d05da88ab6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:11.083896 10385 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 4be3f6e81ddf418fabc7b694a2e62f35 (table default.Table_1e7379199917430eb9e027fc7bb2a4f9 [id=c2d0336fd84c4dbe95472f458b813fe6]). Expected version 1 got 0
I20260925 09:57:11.084290 11085 raft_consensus.cc:374] T 8b281fc20e6f42239fee80d05da88ab6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:11.084606 11085 raft_consensus.cc:729] T 8b281fc20e6f42239fee80d05da88ab6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:11.085425 10385 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet e3b6b985a76b459dbd1de42ea20d3856 (table default.Table_44abad8e8def4bf7be1d01c50a435298 [id=5e5f72cecad44b5e9bfd60320ee42243]). Expected version 2 got 0
I20260925 09:57:11.086160 10385 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet aad3cb5e33f64cb885d51f4ea089307b (table default.Table_2d7acf0d8022443090ba12cd87956502 [id=9d2f4f9f46394332bec9f8beda0d9595]). Expected version 1 got 0
I20260925 09:57:11.086292 11279 tablet.cc:1722] T 4be3f6e81ddf418fabc7b694a2e62f35 P c43461c4ee854ae1ac52b873ccada318: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:11.088281 10962 tablet_service.cc:1467] Processing CreateTablet for tablet 3be0eccc65564bfba3d6e5c398c61fb6 (DEFAULT_TABLE table=default.Table_0af2360d761342cf9f057be5f265aa41 [id=99ba81c4d0dd497398e048a05759419a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:11.089358 10962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3be0eccc65564bfba3d6e5c398c61fb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:11.089041 10385 catalog_manager.cc:5495] T e57065f2bb224a56ad1c3518a0c375da P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:11.095229 10819 tablet_service.cc:1467] Processing CreateTablet for tablet c16e096038084b2b9148390b6e773294 (DEFAULT_TABLE table=default.Table_70f0b47292de423097986b9a1c54f854 [id=8f2eaee8273640fb9ef73ccbe51782f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:11.079381 10666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3be0eccc65564bfba3d6e5c398c61fb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:11.096187 10819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c16e096038084b2b9148390b6e773294. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:11.096151 10963 tablet_service.cc:1467] Processing CreateTablet for tablet c16e096038084b2b9148390b6e773294 (DEFAULT_TABLE table=default.Table_70f0b47292de423097986b9a1c54f854 [id=8f2eaee8273640fb9ef73ccbe51782f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:11.097048 10963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c16e096038084b2b9148390b6e773294. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:11.097169 11084 tablet_bootstrap.cc:492] T 3fa527403d98425ea5cff9215d3a5e7f P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:11.097574 11084 ts_tablet_manager.cc:1397] T 3fa527403d98425ea5cff9215d3a5e7f P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.212s user 0.014s sys 0.042s
I20260925 09:57:11.100795 11084 raft_consensus.cc:348] T 3fa527403d98425ea5cff9215d3a5e7f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:11.101428 11084 raft_consensus.cc:374] T 3fa527403d98425ea5cff9215d3a5e7f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:11.101665 11084 raft_consensus.cc:729] T 3fa527403d98425ea5cff9215d3a5e7f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:11.100978 10659 tablet_service.cc:1467] Processing CreateTablet for tablet c16e096038084b2b9148390b6e773294 (DEFAULT_TABLE table=default.Table_70f0b47292de423097986b9a1c54f854 [id=8f2eaee8273640fb9ef73ccbe51782f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:11.102527 11298 ts_tablet_manager.cc:1905] T 931769e78f32467b8f27bacb286a7a60 P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:11.102248 11084 consensus_queue.cc:260] T 3fa527403d98425ea5cff9215d3a5e7f P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:11.109866 11134 raft_consensus.cc:482] T 25db96e00e6646eaa1190b85c989958f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:11.110379 11134 raft_consensus.cc:504] T 25db96e00e6646eaa1190b85c989958f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:11.112325 11134 leader_election.cc:288] T 25db96e00e6646eaa1190b85c989958f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:11.116465 11287 tablet.cc:1722] T 4be3f6e81ddf418fabc7b694a2e62f35 P be97ab5d75934a648d4ab76341193ec0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:11.119725 10813 tablet_service.cc:1467] Processing CreateTablet for tablet 47a9fb9392244e1d89f8af2bfb42edd6 (DEFAULT_TABLE table=default.Table_7d9b5b59509847c4953652343f89d5e8 [id=0ce6ba53ad84442392c39911a4cccfea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:11.120786 10813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47a9fb9392244e1d89f8af2bfb42edd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:11.122474 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25db96e00e6646eaa1190b85c989958f" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:11.123762 10918 leader_election.cc:341] T 25db96e00e6646eaa1190b85c989958f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:11.138842 11084 ts_tablet_manager.cc:1428] T 3fa527403d98425ea5cff9215d3a5e7f P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.041s user 0.005s sys 0.000s
I20260925 09:57:11.079968 11272 raft_consensus.cc:504] T 0a2bbe141e564a96be05de86f2993234 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:11.144945 10668 tablet_service.cc:1467] Processing CreateTablet for tablet 47a9fb9392244e1d89f8af2bfb42edd6 (DEFAULT_TABLE table=default.Table_7d9b5b59509847c4953652343f89d5e8 [id=0ce6ba53ad84442392c39911a4cccfea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:11.147156 10814 tablet_service.cc:1467] Processing CreateTablet for tablet 28d60531970a4ab58ce5ef99cb3a96bf (DEFAULT_TABLE table=default.Table_6c59a202a45f43aaa4e1de75e372d0ff [id=6851500c7dba44f1bf353bb79bafd97a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:11.148561 10814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28d60531970a4ab58ce5ef99cb3a96bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:11.149130 11272 leader_election.cc:288] T 0a2bbe141e564a96be05de86f2993234 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:11.149750 11084 tablet_bootstrap.cc:492] T 833fac37485141f6a18104eaea907b0e P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:11.151096 10958 tablet_service.cc:1467] Processing CreateTablet for tablet 28d60531970a4ab58ce5ef99cb3a96bf (DEFAULT_TABLE table=default.Table_6c59a202a45f43aaa4e1de75e372d0ff [id=6851500c7dba44f1bf353bb79bafd97a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:11.152402 10387 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: default.Table_faa21794a80c407f800038e583c08f16 [id=800eba46046443a7ad196808dec4cdc0] alter complete (version 2)
I20260925 09:57:11.152887 10393 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: default.Table_5ff0f5bde5954217976f2876e76adaaa [id=aa7e7f5af2724af797170e1f7429d76a] alter complete (version 1)
I20260925 09:57:11.153303 10391 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:35140 (request call id 63) took 3429 ms. Trace:
I20260925 09:57:11.153658 10391 rpcz_store.cc:276] 0925 09:57:07.724183 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:07.724344 (+ 161us) service_pool.cc:227] Handling call
0925 09:57:07.725444 (+ 1100us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:57:07.726419 (+ 975us) catalog_manager.cc:3616] Apply alter schema
0925 09:57:07.727763 (+ 1344us) catalog_manager.cc:3639] Acquired catalog manager lock
0925 09:57:07.728070 (+ 307us) catalog_manager.cc:3786] Updating metadata on disk
0925 09:57:08.131122 (+403052us) catalog_manager.cc:3817] Committing alterations to in-memory state
0925 09:57:11.153244 (+3022122us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:08.116066 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:08.116436 (+ 370us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:08.116461 (+ 25us) write_op.cc:435] Acquired schema lock
0925 09:57:08.116474 (+ 13us) tablet.cc:660] Decoding operations
0925 09:57:08.116731 (+ 257us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:08.116774 (+ 43us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:08.116799 (+ 25us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:08.116963 (+ 164us) tablet.cc:699] Row locks acquired
0925 09:57:08.116977 (+ 14us) write_op.cc:260] PREPARE: finished
0925 09:57:08.117076 (+ 99us) write_op.cc:270] Start()
0925 09:57:08.117158 (+ 82us) write_op.cc:276] Timestamp: P: 1790330228117021 usec, L: 0
0925 09:57:08.117173 (+ 15us) op_driver.cc:331] REPLICATION: starting
0925 09:57:08.117628 (+ 455us) log.cc:842] Serialized 404 byte log entry
0925 09:57:08.129051 (+ 11423us) op_driver.cc:444] REPLICATION: finished
0925 09:57:08.129438 (+ 387us) write_op.cc:301] APPLY: starting
0925 09:57:08.129498 (+ 60us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:08.129588 (+ 90us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:08.129703 (+ 115us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:08.129842 (+ 139us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:08.129971 (+ 129us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:08.130017 (+ 46us) write_op.cc:312] APPLY: finished
0925 09:57:08.130187 (+ 170us) log.cc:842] Serialized 22 byte log entry
0925 09:57:08.130833 (+ 646us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:08.130898 (+ 65us) write_op.cc:454] Released schema lock
0925 09:57:08.130930 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:08.131118 (+ 188us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:08.131145 (+ 27us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":219,"apply.run_cpu_time_us":1361,"apply.run_wall_time_us":1860,"num_ops":1,"prepare.queue_time_us":345560,"prepare.run_cpu_time_us":2028,"prepare.run_wall_time_us":48167,"raft.queue_time_us":238,"raft.run_cpu_time_us":923,"raft.run_wall_time_us":2017,"replication_time_us":11776}]]}
I20260925 09:57:11.160590 10391 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_2d7acf0d8022443090ba12cd87956502" } modify_external_catalogs: true
I20260925 09:57:11.161244 10391 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_2d7acf0d8022443090ba12cd87956502" } modify_external_catalogs: true
I20260925 09:57:11.162038 10388 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: default.Table_4e09a615fb82487baa4783a5f6e2181e [id=f17a11d223204599afb24e2e9d1d9d9d] alter complete (version 3)
I20260925 09:57:11.165784 10961 tablet_service.cc:1467] Processing CreateTablet for tablet 47a9fb9392244e1d89f8af2bfb42edd6 (DEFAULT_TABLE table=default.Table_7d9b5b59509847c4953652343f89d5e8 [id=0ce6ba53ad84442392c39911a4cccfea]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 09:57:11.171573 10393 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.2:40761 (request call id 17) took 2115 ms (client timeout 1000 ms). Trace:
W20260925 09:57:11.175349 10387 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.1:56563 (request call id 20) took 2212 ms (client timeout 1000 ms). Trace:
I20260925 09:57:11.183167 10818 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b03e5aad59e46a6b0e43b9cb36278d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:07 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:11.184028 11292 tablet_replica.cc:330] T 6b03e5aad59e46a6b0e43b9cb36278d7 P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:11.148597 10661 tablet_service.cc:1467] Processing CreateTablet for tablet 28d60531970a4ab58ce5ef99cb3a96bf (DEFAULT_TABLE table=default.Table_6c59a202a45f43aaa4e1de75e372d0ff [id=6851500c7dba44f1bf353bb79bafd97a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:11.188241 11292 raft_consensus.cc:2229] T 6b03e5aad59e46a6b0e43b9cb36278d7 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:11.141083 11211 consensus_queue.cc:237] T aad3cb5e33f64cb885d51f4ea089307b P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:11.188689 11292 raft_consensus.cc:2258] T 6b03e5aad59e46a6b0e43b9cb36278d7 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 09:57:11.179965 10387 rpcz_store.cc:269] 0925 09:57:08.963531 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:08.963918 (+ 387us) service_pool.cc:227] Handling call
0925 09:57:09.038719 (+ 74801us) spinlock_profiling.cc:243] Waited 72.1 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2ad9c3e 00007fabf0610fb4 00007fabf0610f42 00007fabf0610e71 00007fabf0610dec 00007fabf06100a2 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:11.175288 (+2136569us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:10.618337 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:10.618695 (+ 358us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:10.618718 (+ 23us) write_op.cc:435] Acquired schema lock
0925 09:57:10.618731 (+ 13us) tablet.cc:660] Decoding operations
0925 09:57:10.618982 (+ 251us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:10.619017 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:10.619037 (+ 20us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:10.619167 (+ 130us) tablet.cc:699] Row locks acquired
0925 09:57:10.619180 (+ 13us) write_op.cc:260] PREPARE: finished
0925 09:57:10.619294 (+ 114us) write_op.cc:270] Start()
0925 09:57:10.619371 (+ 77us) write_op.cc:276] Timestamp: P: 1790330230625554 usec, L: 0
0925 09:57:10.619419 (+ 48us) op_driver.cc:331] REPLICATION: starting
0925 09:57:10.619880 (+ 461us) log.cc:842] Serialized 355 byte log entry
0925 09:57:10.670052 (+ 50172us) op_driver.cc:444] REPLICATION: finished
0925 09:57:10.675717 (+ 5665us) write_op.cc:301] APPLY: starting
0925 09:57:10.675776 (+ 59us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:10.675878 (+ 102us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:10.675985 (+ 107us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:10.676127 (+ 142us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:10.676276 (+ 149us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:10.676345 (+ 69us) write_op.cc:312] APPLY: finished
0925 09:57:10.676509 (+ 164us) log.cc:842] Serialized 22 byte log entry
0925 09:57:10.676615 (+ 106us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:10.676687 (+ 72us) write_op.cc:454] Released schema lock
0925 09:57:10.676721 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:10.676895 (+ 174us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:10.676928 (+ 33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2191,"reported_tablets":8,"spinlock_wait_cycles":165823360,"child_traces":[["op",{"apply.queue_time_us":5531,"apply.run_cpu_time_us":1368,"apply.run_wall_time_us":1367,"num_ops":1,"prepare.queue_time_us":6846,"prepare.run_cpu_time_us":1717,"prepare.run_wall_time_us":1726,"replication_time_us":50564,"thread_start_us":60974,"threads_started":1}]]}
I20260925 09:57:11.190583 11292 ts_tablet_manager.cc:1905] T 6b03e5aad59e46a6b0e43b9cb36278d7 P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:11.085211 11085 consensus_queue.cc:260] T 8b281fc20e6f42239fee80d05da88ab6 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:11.171887 10393 rpcz_store.cc:269] 0925 09:57:09.056005 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:09.056110 (+ 105us) service_pool.cc:227] Handling call
0925 09:57:11.171512 (+2115402us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:10.341592 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:10.342069 (+ 477us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:10.342106 (+ 37us) write_op.cc:435] Acquired schema lock
0925 09:57:10.342119 (+ 13us) tablet.cc:660] Decoding operations
0925 09:57:10.342464 (+ 345us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:10.342505 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:10.342531 (+ 26us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:10.342804 (+ 273us) tablet.cc:699] Row locks acquired
0925 09:57:10.342818 (+ 14us) write_op.cc:260] PREPARE: finished
0925 09:57:10.342923 (+ 105us) write_op.cc:270] Start()
0925 09:57:10.343020 (+ 97us) write_op.cc:276] Timestamp: P: 1790330230342852 usec, L: 0
0925 09:57:10.343035 (+ 15us) op_driver.cc:331] REPLICATION: starting
0925 09:57:10.343629 (+ 594us) log.cc:842] Serialized 894 byte log entry
0925 09:57:10.585572 (+241943us) op_driver.cc:444] REPLICATION: finished
0925 09:57:10.586130 (+ 558us) write_op.cc:301] APPLY: starting
0925 09:57:10.586199 (+ 69us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:10.586319 (+ 120us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:10.586407 (+ 88us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:10.586607 (+ 200us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:10.586743 (+ 136us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 09:57:10.586789 (+ 46us) write_op.cc:312] APPLY: finished
0925 09:57:10.586965 (+ 176us) log.cc:842] Serialized 28 byte log entry
0925 09:57:10.587103 (+ 138us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:10.587159 (+ 56us) write_op.cc:454] Released schema lock
0925 09:57:10.587196 (+ 37us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:10.587362 (+ 166us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:10.587403 (+ 41us) write_op.cc:454] Released schema lock
Related trace 'op':
0925 09:57:10.599990 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:10.600379 (+ 389us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:10.600401 (+ 22us) write_op.cc:435] Acquired schema lock
0925 09:57:10.600415 (+ 14us) tablet.cc:660] Decoding operations
0925 09:57:10.600652 (+ 237us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:10.600685 (+ 33us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:10.600706 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:10.600837 (+ 131us) tablet.cc:699] Row locks acquired
0925 09:57:10.600851 (+ 14us) write_op.cc:260] PREPARE: finished
0925 09:57:10.600950 (+ 99us) write_op.cc:270] Start()
0925 09:57:10.601027 (+ 77us) write_op.cc:276] Timestamp: P: 1790330230607214 usec, L: 0
0925 09:57:10.601042 (+ 15us) op_driver.cc:331] REPLICATION: starting
0925 09:57:10.601504 (+ 462us) log.cc:842] Serialized 355 byte log entry
0925 09:57:10.669848 (+ 68344us) op_driver.cc:444] REPLICATION: finished
0925 09:57:10.674083 (+ 4235us) write_op.cc:301] APPLY: starting
0925 09:57:10.674143 (+ 60us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:10.674247 (+ 104us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:10.674417 (+ 170us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:10.674571 (+ 154us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:10.674724 (+ 153us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:10.674784 (+ 60us) write_op.cc:312] APPLY: finished
0925 09:57:10.674954 (+ 170us) log.cc:842] Serialized 22 byte log entry
0925 09:57:10.675089 (+ 135us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:10.675149 (+ 60us) write_op.cc:454] Released schema lock
0925 09:57:10.675190 (+ 41us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:10.675400 (+ 210us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:10.675428 (+ 28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":809,"reported_tablets":9,"child_traces":[["op",{"apply.queue_time_us":390,"apply.run_cpu_time_us":1455,"apply.run_wall_time_us":1454,"num_ops":2,"prepare.queue_time_us":90369,"prepare.run_cpu_time_us":2126,"prepare.run_wall_time_us":3253,"replication_time_us":242459}],["op",{"apply.queue_time_us":4090,"apply.run_cpu_time_us":1509,"apply.run_wall_time_us":1509,"num_ops":1,"prepare.queue_time_us":6364,"prepare.run_cpu_time_us":1701,"prepare.run_wall_time_us":1702,"replication_time_us":68736}]]}
I20260925 09:57:11.192571 10393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_7d0a974dfa9a48c496f3128d28fe1d15"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:11.198089 10393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7d0a974dfa9a48c496f3128d28fe1d15 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:11.201967 10958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28d60531970a4ab58ce5ef99cb3a96bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:11.204932 10819 tablet_service.cc:1467] Processing CreateTablet for tablet 91c09118b43b41eca8f7040323ba76f7 (DEFAULT_TABLE table=default.Table_6b025220a9aa49ecb47a7e17824f0a71 [id=453beec106a64428b163c7db8132fc21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:11.211685 10961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47a9fb9392244e1d89f8af2bfb42edd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:11.212420 11082 tablet_bootstrap.cc:654] T 8b281fc20e6f42239fee80d05da88ab6 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:11.214218 10658 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b03e5aad59e46a6b0e43b9cb36278d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:07 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:11.213603 11134 raft_consensus.cc:482] T 83b0cf21f2a144719a2e13990e1921c0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:11.214792 11134 raft_consensus.cc:504] T 83b0cf21f2a144719a2e13990e1921c0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:11.153959 11299 raft_consensus.cc:504] T 26b7c37959d042b48234acc79290936f P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:11.217068 11134 leader_election.cc:288] T 83b0cf21f2a144719a2e13990e1921c0 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:11.217059 10658 tablet_service.cc:1467] Processing CreateTablet for tablet 91c09118b43b41eca8f7040323ba76f7 (DEFAULT_TABLE table=default.Table_6b025220a9aa49ecb47a7e17824f0a71 [id=453beec106a64428b163c7db8132fc21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:11.218793 10683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25db96e00e6646eaa1190b85c989958f" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:11.219851 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83b0cf21f2a144719a2e13990e1921c0" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:11.221397 10919 leader_election.cc:341] T 25db96e00e6646eaa1190b85c989958f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:11.221799 10919 leader_election.cc:302] T 25db96e00e6646eaa1190b85c989958f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:11.223048 11206 raft_consensus.cc:2726] T 25db96e00e6646eaa1190b85c989958f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:11.223572 10919 leader_election.cc:341] T 83b0cf21f2a144719a2e13990e1921c0 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:11.223749 11109 consensus_queue.cc:1035] T e57065f2bb224a56ad1c3518a0c375da P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:11.225205 11281 tablet.cc:1722] T e3b6b985a76b459dbd1de42ea20d3856 P b8850818e2ee42a082802d0095c08d21: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:11.227698 10668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47a9fb9392244e1d89f8af2bfb42edd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:11.229437 10959 tablet_service.cc:1467] Processing CreateTablet for tablet 91c09118b43b41eca8f7040323ba76f7 (DEFAULT_TABLE table=default.Table_6b025220a9aa49ecb47a7e17824f0a71 [id=453beec106a64428b163c7db8132fc21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:11.230563 10959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91c09118b43b41eca8f7040323ba76f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:11.236416 10661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28d60531970a4ab58ce5ef99cb3a96bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:11.193948 10387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_3bc500239afb4c5eae08ecf1401ae72f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:11.214393 10819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91c09118b43b41eca8f7040323ba76f7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:11.243522 10387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3bc500239afb4c5eae08ecf1401ae72f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:11.244807 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a2bbe141e564a96be05de86f2993234" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:11.245425 10984 raft_consensus.cc:3037] T 0a2bbe141e564a96be05de86f2993234 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:11.248131 10659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c16e096038084b2b9148390b6e773294. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:11.250005 11084 tablet_bootstrap.cc:654] T 833fac37485141f6a18104eaea907b0e P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:11.252385 10658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91c09118b43b41eca8f7040323ba76f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:11.268853 10393 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_6c59a202a45f43aaa4e1de75e372d0ff" } new_table_name: "default.Table_c3a609c3cdbf4990910b8784a7412d34" modify_external_catalogs: true
W20260925 09:57:11.281785 10392 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.2:40761 (request call id 16) took 3237 ms (client timeout 1000 ms). Trace:
W20260925 09:57:11.285112 10389 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.3:41469 (request call id 10) took 3278 ms (client timeout 1000 ms). Trace:
I20260925 09:57:11.302052 10386 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 840faf042d8749bd958ce1de0207a734
I20260925 09:57:11.305650 10386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_c2ce8fea3dc743379c1912445278a487"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:11.323863 10386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c2ce8fea3dc743379c1912445278a487 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 09:57:11.285466 10389 rpcz_store.cc:269] 0925 09:57:08.007190 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:08.007314 (+ 124us) service_pool.cc:227] Handling call
0925 09:57:08.028117 (+ 20803us) spinlock_profiling.cc:243] Waited 18.8 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2ad9c3e 00007fabf0610fb4 00007fabf0610f42 00007fabf0610e71 00007fabf0610dec 00007fabf06100a2 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:11.285049 (+3256932us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:09.335881 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:09.336272 (+ 391us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:09.336303 (+ 31us) write_op.cc:435] Acquired schema lock
0925 09:57:09.336320 (+ 17us) tablet.cc:660] Decoding operations
0925 09:57:09.336599 (+ 279us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:09.336640 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:09.336658 (+ 18us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:09.336796 (+ 138us) tablet.cc:699] Row locks acquired
0925 09:57:09.336812 (+ 16us) write_op.cc:260] PREPARE: finished
0925 09:57:09.336913 (+ 101us) write_op.cc:270] Start()
0925 09:57:09.336988 (+ 75us) write_op.cc:276] Timestamp: P: 1790330229336834 usec, L: 0
0925 09:57:09.337003 (+ 15us) op_driver.cc:331] REPLICATION: starting
0925 09:57:09.337418 (+ 415us) log.cc:842] Serialized 523 byte log entry
0925 09:57:09.521626 (+184208us) op_driver.cc:444] REPLICATION: finished
0925 09:57:09.524947 (+ 3321us) write_op.cc:301] APPLY: starting
0925 09:57:09.524990 (+ 43us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:09.525060 (+ 70us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:09.525143 (+ 83us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:09.525247 (+ 104us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:09.525975 (+ 728us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:09.526016 (+ 41us) write_op.cc:312] APPLY: finished
0925 09:57:09.526143 (+ 127us) log.cc:842] Serialized 22 byte log entry
0925 09:57:09.526243 (+ 100us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:09.526284 (+ 41us) write_op.cc:454] Released schema lock
0925 09:57:09.526314 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:09.526449 (+ 135us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:09.526471 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":4,"spinlock_wait_cycles":43257216,"child_traces":[["op",{"apply.queue_time_us":3209,"apply.run_cpu_time_us":1636,"apply.run_wall_time_us":1635,"num_ops":1,"prepare.queue_time_us":70437,"prepare.run_cpu_time_us":1853,"prepare.run_wall_time_us":10996,"raft.queue_time_us":100,"raft.run_cpu_time_us":17,"raft.run_wall_time_us":18,"replication_time_us":184569,"thread_start_us":91358,"threads_started":1}]]}
I20260925 09:57:11.325238 11299 leader_election.cc:288] T 26b7c37959d042b48234acc79290936f P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:11.325915 10389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_077f38b3329348828ef80c0f4e0c972f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:11.326242 10664 tablet_service.cc:1514] Processing DeleteTablet for tablet 840faf042d8749bd958ce1de0207a734 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:11 UTC) from {username='slave'} at 127.0.0.1:37876
W20260925 09:57:11.328619 10389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_077f38b3329348828ef80c0f4e0c972f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 09:57:11.282181 10392 rpcz_store.cc:269] 0925 09:57:08.044120 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:08.044589 (+ 469us) service_pool.cc:227] Handling call
0925 09:57:11.281719 (+3237130us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:08.555602 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:08.556100 (+ 498us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:08.556135 (+ 35us) write_op.cc:435] Acquired schema lock
0925 09:57:08.556152 (+ 17us) tablet.cc:660] Decoding operations
0925 09:57:08.556559 (+ 407us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:08.556613 (+ 54us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:08.556646 (+ 33us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:08.556985 (+ 339us) tablet.cc:699] Row locks acquired
0925 09:57:08.557003 (+ 18us) write_op.cc:260] PREPARE: finished
0925 09:57:08.557138 (+ 135us) write_op.cc:270] Start()
0925 09:57:08.557242 (+ 104us) write_op.cc:276] Timestamp: P: 1790330228557081 usec, L: 0
0925 09:57:08.557260 (+ 18us) op_driver.cc:331] REPLICATION: starting
0925 09:57:08.557890 (+ 630us) log.cc:842] Serialized 894 byte log entry
0925 09:57:08.815802 (+257912us) op_driver.cc:444] REPLICATION: finished
0925 09:57:08.816847 (+ 1045us) write_op.cc:301] APPLY: starting
0925 09:57:08.816900 (+ 53us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:08.817002 (+ 102us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:08.817103 (+ 101us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:08.817312 (+ 209us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:08.817452 (+ 140us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 09:57:08.817494 (+ 42us) write_op.cc:312] APPLY: finished
0925 09:57:08.817659 (+ 165us) log.cc:842] Serialized 28 byte log entry
0925 09:57:08.817799 (+ 140us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:08.817863 (+ 64us) write_op.cc:454] Released schema lock
0925 09:57:08.817896 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:08.818062 (+ 166us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:08.818088 (+ 26us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":6,"child_traces":[["op",{"apply.queue_time_us":902,"apply.run_cpu_time_us":1424,"apply.run_wall_time_us":1424,"num_ops":2,"prepare.queue_time_us":207007,"prepare.run_cpu_time_us":2667,"prepare.run_wall_time_us":47203,"raft.queue_time_us":154,"raft.run_cpu_time_us":611,"raft.run_wall_time_us":610,"replication_time_us":258465}]]}
W20260925 09:57:11.316859 10388 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.2:40761 (request call id 18) took 755 ms (client timeout 1000 ms). Trace:
W20260925 09:57:11.329676 10388 rpcz_store.cc:269] 0925 09:57:10.561704 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:10.591909 (+ 30205us) service_pool.cc:227] Handling call
0925 09:57:11.316807 (+724898us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:10.970658 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:10.971081 (+ 423us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:10.971114 (+ 33us) write_op.cc:435] Acquired schema lock
0925 09:57:10.971128 (+ 14us) tablet.cc:660] Decoding operations
0925 09:57:10.971364 (+ 236us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:10.971428 (+ 64us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:10.971448 (+ 20us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:10.971589 (+ 141us) tablet.cc:699] Row locks acquired
0925 09:57:10.971602 (+ 13us) write_op.cc:260] PREPARE: finished
0925 09:57:10.971717 (+ 115us) write_op.cc:270] Start()
0925 09:57:10.971802 (+ 85us) write_op.cc:276] Timestamp: P: 1790330230977950 usec, L: 0
0925 09:57:10.971817 (+ 15us) op_driver.cc:331] REPLICATION: starting
0925 09:57:10.972243 (+ 426us) log.cc:842] Serialized 355 byte log entry
0925 09:57:11.055039 (+ 82796us) op_driver.cc:444] REPLICATION: finished
0925 09:57:11.055569 (+ 530us) write_op.cc:301] APPLY: starting
0925 09:57:11.055622 (+ 53us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:11.055727 (+ 105us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:11.055850 (+ 123us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:11.056030 (+ 180us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:11.056204 (+ 174us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:11.056266 (+ 62us) write_op.cc:312] APPLY: finished
0925 09:57:11.056478 (+ 212us) log.cc:842] Serialized 22 byte log entry
0925 09:57:11.056656 (+ 178us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:11.056747 (+ 91us) write_op.cc:454] Released schema lock
0925 09:57:11.056780 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:11.056949 (+ 169us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:11.056983 (+ 34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2325,"reported_tablets":14,"child_traces":[["op",{"apply.queue_time_us":352,"apply.run_cpu_time_us":1595,"apply.run_wall_time_us":1594,"num_ops":1,"prepare.queue_time_us":1508,"prepare.run_cpu_time_us":1834,"prepare.run_wall_time_us":1834,"raft.queue_time_us":1278,"raft.run_cpu_time_us":733,"raft.run_wall_time_us":731,"replication_time_us":83137}]]}
I20260925 09:57:11.335265 11134 raft_consensus.cc:482] T fd73d0d52350444499abae34e8c02fab P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:11.351053 10955 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c13a2f288b14d03a3ca4fdd4889cbad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:09 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:11.351963 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e57065f2bb224a56ad1c3518a0c375da" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:11.352072 11281 tablet.cc:1722] T e3b6b985a76b459dbd1de42ea20d3856 P b8850818e2ee42a082802d0095c08d21: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 09:57:11.356796 10954 tablet_service.cc:1514] Processing DeleteTablet for tablet 26b7c37959d042b48234acc79290936f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:08 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:11.358343 10956 tablet_service.cc:1514] Processing DeleteTablet for tablet d48d4517c13049389c5b2ba1ab7b14bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:07 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:11.359733 10953 tablet_service.cc:1514] Processing DeleteTablet for tablet 840faf042d8749bd958ce1de0207a734 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:11 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:11.363760 10981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26b7c37959d042b48234acc79290936f" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:11.368916 10981 raft_consensus.cc:3037] T 26b7c37959d042b48234acc79290936f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:11.372002 11245 tablet_replica.cc:330] T 26b7c37959d042b48234acc79290936f P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:11.284592 11298 ts_tablet_manager.cc:1918] T 931769e78f32467b8f27bacb286a7a60 P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:11.371611 11299 consensus_queue.cc:1035] T 04fafb9dcf274806b11b2598e73853e9 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:11.371430 10960 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b03e5aad59e46a6b0e43b9cb36278d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:07 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:11.353802 11134 raft_consensus.cc:504] T fd73d0d52350444499abae34e8c02fab P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:11.388479 11084 tablet_bootstrap.cc:492] T 833fac37485141f6a18104eaea907b0e P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:11.388996 11084 ts_tablet_manager.cc:1397] T 833fac37485141f6a18104eaea907b0e P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.249s user 0.013s sys 0.003s
I20260925 09:57:11.388846 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd73d0d52350444499abae34e8c02fab" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:11.391537 10919 leader_election.cc:341] T fd73d0d52350444499abae34e8c02fab P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:11.398286 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83b0cf21f2a144719a2e13990e1921c0" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:11.398860 11134 leader_election.cc:288] T fd73d0d52350444499abae34e8c02fab P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:11.406116 10981 raft_consensus.cc:2454] T 26b7c37959d042b48234acc79290936f P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:11.406589 11245 raft_consensus.cc:2229] T 26b7c37959d042b48234acc79290936f P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260925 09:57:11.418624 10918 leader_election.cc:341] T 83b0cf21f2a144719a2e13990e1921c0 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:11.419036 10918 leader_election.cc:302] T 83b0cf21f2a144719a2e13990e1921c0 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:11.421252 11245 raft_consensus.cc:2258] T 26b7c37959d042b48234acc79290936f P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:11.422843 11134 raft_consensus.cc:2726] T 83b0cf21f2a144719a2e13990e1921c0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:11.423660 11245 ts_tablet_manager.cc:1905] T 26b7c37959d042b48234acc79290936f P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:11.430560 10953 tablet_service.cc:1514] Processing DeleteTablet for tablet 931769e78f32467b8f27bacb286a7a60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:09 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:11.433131 10953 tablet_service.cc:1514] Processing DeleteTablet for tablet d8be238061c146eb96d17a22c1b2104a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:08 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:11.434499 11134 raft_consensus.cc:482] T c07c6205d1284a6fbb369c010a226da7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:11.435465 11134 raft_consensus.cc:504] T c07c6205d1284a6fbb369c010a226da7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:11.439471 11134 leader_election.cc:288] T c07c6205d1284a6fbb369c010a226da7 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:11.443708 10818 tablet_service.cc:1514] Processing DeleteTablet for tablet 840faf042d8749bd958ce1de0207a734 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:11 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:11.443851 10817 tablet_service.cc:1514] Processing DeleteTablet for tablet d8be238061c146eb96d17a22c1b2104a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:08 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:11.445281 10815 tablet_service.cc:1514] Processing DeleteTablet for tablet 931769e78f32467b8f27bacb286a7a60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:09 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:11.445947 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd73d0d52350444499abae34e8c02fab" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:11.447594 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c07c6205d1284a6fbb369c010a226da7" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:11.448976 10918 leader_election.cc:341] T fd73d0d52350444499abae34e8c02fab P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:11.449419 10918 leader_election.cc:302] T fd73d0d52350444499abae34e8c02fab P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
W20260925 09:57:11.450628 10918 leader_election.cc:341] T c07c6205d1284a6fbb369c010a226da7 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:11.451184 10813 tablet_service.cc:1514] Processing DeleteTablet for tablet d48d4517c13049389c5b2ba1ab7b14bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:07 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:11.440683 11084 raft_consensus.cc:348] T 833fac37485141f6a18104eaea907b0e P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:11.456780 11084 raft_consensus.cc:374] T 833fac37485141f6a18104eaea907b0e P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:11.457199 11084 raft_consensus.cc:729] T 833fac37485141f6a18104eaea907b0e P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:11.458123 11084 consensus_queue.cc:260] T 833fac37485141f6a18104eaea907b0e P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:11.459813 11206 raft_consensus.cc:2726] T fd73d0d52350444499abae34e8c02fab P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:11.398926 11298 log.cc:1196] T 931769e78f32467b8f27bacb286a7a60 P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/931769e78f32467b8f27bacb286a7a60
I20260925 09:57:11.467278 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26b7c37959d042b48234acc79290936f" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:11.471725 10838 raft_consensus.cc:2454] T 26b7c37959d042b48234acc79290936f P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:11.473451 11084 ts_tablet_manager.cc:1428] T 833fac37485141f6a18104eaea907b0e P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.084s user 0.007s sys 0.000s
I20260925 09:57:11.476763 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a2bbe141e564a96be05de86f2993234" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:11.477428 10838 raft_consensus.cc:3037] T 0a2bbe141e564a96be05de86f2993234 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:11.480476 11134 raft_consensus.cc:482] T 4c4e8790282a49fd849a34ca11bef5bf P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:11.481792 11134 raft_consensus.cc:504] T 4c4e8790282a49fd849a34ca11bef5bf P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:11.484181 11298 ts_tablet_manager.cc:1939] T 931769e78f32467b8f27bacb286a7a60 P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:11.485114 10836 raft_consensus.cc:3037] T e57065f2bb224a56ad1c3518a0c375da P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:11.485147 10619 leader_election.cc:302] T 26b7c37959d042b48234acc79290936f P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:11.486654 11272 raft_consensus.cc:2781] T 26b7c37959d042b48234acc79290936f P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
W20260925 09:57:11.309767 10384 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.3:41469 (request call id 11) took 2268 ms (client timeout 1000 ms). Trace:
I20260925 09:57:11.346153 10391 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet aad3cb5e33f64cb885d51f4ea089307b
I20260925 09:57:11.362607 10387 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_8f64c62cd63d43ce89d446956c0883e3" } new_table_name: "default.Table_8705f75ef7b2448ab645f1d25e3c2637" modify_external_catalogs: true
I20260925 09:57:11.489701 10392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_8dc254af52974ea2827405df2b0a5961"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:11.492193 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c07c6205d1284a6fbb369c010a226da7" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:11.492646 11134 leader_election.cc:288] T 4c4e8790282a49fd849a34ca11bef5bf P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:11.493471 10680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c4e8790282a49fd849a34ca11bef5bf" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:11.494088 10392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8dc254af52974ea2827405df2b0a5961 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 09:57:11.366909 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 8c13a2f288b14d03a3ca4fdd4889cbad because the tablet was not found. No further retry: Not found: Tablet not found: 8c13a2f288b14d03a3ca4fdd4889cbad
W20260925 09:57:11.495733 10377 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 128864us
W20260925 09:57:11.498183 10919 leader_election.cc:341] T c07c6205d1284a6fbb369c010a226da7 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:11.498576 10919 leader_election.cc:302] T c07c6205d1284a6fbb369c010a226da7 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:11.499372 11134 raft_consensus.cc:2726] T c07c6205d1284a6fbb369c010a226da7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:11.504182 10984 raft_consensus.cc:2454] T 0a2bbe141e564a96be05de86f2993234 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:11.509899 10391 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_0108f9fbea174357ae1e8d04519bbaf0" } modify_external_catalogs: true
W20260925 09:57:11.487748 10384 rpcz_store.cc:269] 0925 09:57:09.041124 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:09.047898 (+ 6774us) service_pool.cc:227] Handling call
0925 09:57:11.309695 (+2261797us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:09.546117 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:09.546585 (+ 468us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:09.546618 (+ 33us) write_op.cc:435] Acquired schema lock
0925 09:57:09.546637 (+ 19us) tablet.cc:660] Decoding operations
0925 09:57:09.546892 (+ 255us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:09.546928 (+ 36us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:09.546951 (+ 23us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:09.547089 (+ 138us) tablet.cc:699] Row locks acquired
0925 09:57:09.547102 (+ 13us) write_op.cc:260] PREPARE: finished
0925 09:57:09.547219 (+ 117us) write_op.cc:270] Start()
0925 09:57:09.547305 (+ 86us) write_op.cc:276] Timestamp: P: 1790330229547173 usec, L: 0
0925 09:57:09.547321 (+ 16us) op_driver.cc:331] REPLICATION: starting
0925 09:57:09.629758 (+ 82437us) log.cc:842] Serialized 523 byte log entry
0925 09:57:09.631342 (+ 1584us) spinlock_profiling.cc:243] Waited 81.9 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90471 00007fabf0b07198 00007fabf0f68ade 00007fabf0f682b8 00007fabf0f6cba1 00007fabf0f6cb59 00007fabf0f6cae9 00007fabf0f6cab1 00007fabf0f6bdad 0000000000580784 00000000005805b9
0925 09:57:09.642310 (+ 10968us) spinlock_profiling.cc:243] Waited 9.47 ms on lock 0x7b58000b03b0. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0a85834 00007fabf0a8a771 00007fabf0a8a6f9 00007fabf0a8a689 00007fabf0a8a651 00007fabf0a8981d 0000000000580784 00000000005805b9 00007fabe9bb66b6 00007fabe9bb9b11
0925 09:57:09.643528 (+ 1218us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b5400012398. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90471 00007fabf0ae0605 00007fabf0ae0752 00007fabf0a9cb94 00007fabf0a85545 00007fabf0a8a771 00007fabf0a8a6f9 00007fabf0a8a689 00007fabf0a8a651 00007fabf0a8981d 0000000000580784
0925 09:57:09.652527 (+ 8999us) op_driver.cc:444] REPLICATION: finished
0925 09:57:09.653730 (+ 1203us) write_op.cc:301] APPLY: starting
0925 09:57:09.653789 (+ 59us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:09.653884 (+ 95us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:09.653903 (+ 19us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:09.654065 (+ 162us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:09.654201 (+ 136us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:09.654247 (+ 46us) write_op.cc:312] APPLY: finished
0925 09:57:09.654454 (+ 207us) log.cc:842] Serialized 22 byte log entry
0925 09:57:09.654588 (+ 134us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:09.654650 (+ 62us) write_op.cc:454] Released schema lock
0925 09:57:09.654679 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:09.654852 (+ 173us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:09.654877 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":15,"child_traces":[["op",{"apply.queue_time_us":999,"apply.run_cpu_time_us":1319,"apply.run_wall_time_us":1319,"num_ops":1,"prepare.queue_time_us":91,"prepare.run_cpu_time_us":4398,"prepare.run_wall_time_us":85467,"raft.queue_time_us":818,"raft.run_cpu_time_us":3581,"raft.run_wall_time_us":22682,"replication_time_us":105106,"spinlock_wait_cycles":233478528}]]}
I20260925 09:57:11.510510 10391 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_0108f9fbea174357ae1e8d04519bbaf0" } modify_external_catalogs: true
I20260925 09:57:11.512151 10961 tablet_service.cc:1514] Processing DeleteTablet for tablet aad3cb5e33f64cb885d51f4ea089307b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:11 UTC) from {username='slave'} at 127.0.0.1:53560
W20260925 09:57:11.516911 10387 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0a2bbe141e564a96be05de86f2993234 consensus state has no leader
W20260925 09:57:11.373795 10390 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.3:41469 (request call id 12) took 1015 ms (client timeout 1000 ms). Trace:
W20260925 09:57:11.523537 10390 rpcz_store.cc:269] 0925 09:57:10.358146 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:10.393592 (+ 35446us) service_pool.cc:227] Handling call
0925 09:57:10.507897 (+114305us) spinlock_profiling.cc:243] Waited 72.1 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2ad9c3e 00007fabf0610fb4 00007fabf0610f42 00007fabf0610e71 00007fabf0610dec 00007fabf06100a2 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:11.373724 (+865827us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":123,"reported_tablets":22,"spinlock_wait_cycles":165768448}
W20260925 09:57:11.524899 10919 leader_election.cc:341] T 4c4e8790282a49fd849a34ca11bef5bf P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:11.529475 10619 leader_election.cc:302] T 0a2bbe141e564a96be05de86f2993234 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:11.529385 11109 consensus_queue.cc:1035] T 59f0c6489a33433899bb6e62088121b6 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:11.530634 11109 raft_consensus.cc:2781] T 0a2bbe141e564a96be05de86f2993234 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:11.531057 11109 raft_consensus.cc:686] T 0a2bbe141e564a96be05de86f2993234 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:11.532330 11084 tablet_bootstrap.cc:492] T 69f84509cafc44dd9444218738ebc2fe P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:11.531872 11109 consensus_queue.cc:237] T 0a2bbe141e564a96be05de86f2993234 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:11.533118 11298 tablet_replica.cc:330] T 6b03e5aad59e46a6b0e43b9cb36278d7 P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:11.533921 11298 raft_consensus.cc:2229] T 6b03e5aad59e46a6b0e43b9cb36278d7 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:11.537822 11084 tablet_bootstrap.cc:654] T 69f84509cafc44dd9444218738ebc2fe P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:11.490123 11272 raft_consensus.cc:686] T 26b7c37959d042b48234acc79290936f P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:11.534786 11298 raft_consensus.cc:2258] T 6b03e5aad59e46a6b0e43b9cb36278d7 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:11.547123 10658 tablet_service.cc:1514] Processing DeleteTablet for tablet aad3cb5e33f64cb885d51f4ea089307b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:11 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:11.545848 11272 consensus_queue.cc:237] T 26b7c37959d042b48234acc79290936f P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:11.607153 10391 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet ce12bb98c06c40f0b106e886c8c86682
I20260925 09:57:11.614009 11084 tablet_bootstrap.cc:492] T 69f84509cafc44dd9444218738ebc2fe P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:11.614485 11084 ts_tablet_manager.cc:1397] T 69f84509cafc44dd9444218738ebc2fe P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.140s user 0.012s sys 0.017s
I20260925 09:57:11.614606 11245 ts_tablet_manager.cc:1918] T 26b7c37959d042b48234acc79290936f P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:11.615005 11245 log.cc:1196] T 26b7c37959d042b48234acc79290936f P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/26b7c37959d042b48234acc79290936f
I20260925 09:57:11.617327 11084 raft_consensus.cc:348] T 69f84509cafc44dd9444218738ebc2fe P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:11.619973 11084 raft_consensus.cc:374] T 69f84509cafc44dd9444218738ebc2fe P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:11.620240 11084 raft_consensus.cc:729] T 69f84509cafc44dd9444218738ebc2fe P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:11.621069 11084 consensus_queue.cc:260] T 69f84509cafc44dd9444218738ebc2fe P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:11.623377 11084 ts_tablet_manager.cc:1428] T 69f84509cafc44dd9444218738ebc2fe P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.009s user 0.004s sys 0.000s
I20260925 09:57:11.615888 11245 ts_tablet_manager.cc:1939] T 26b7c37959d042b48234acc79290936f P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
W20260925 09:57:11.630651 10621 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 75688.6us
I20260925 09:57:11.625775 11084 tablet_bootstrap.cc:492] T 91c09118b43b41eca8f7040323ba76f7 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:11.631911 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 26b7c37959d042b48234acc79290936f (table default.Table_fe84e50a7371410d81b423c4324b17f2 [id=2b43210a3c1b4f29a1b0f5fdc7819ef2]) successfully deleted
I20260925 09:57:11.635902 11245 tablet_replica.cc:330] T d48d4517c13049389c5b2ba1ab7b14bc P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:11.637068 11245 raft_consensus.cc:2229] T d48d4517c13049389c5b2ba1ab7b14bc P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:11.637639 11245 raft_consensus.cc:2258] T d48d4517c13049389c5b2ba1ab7b14bc P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:11.639447 11245 ts_tablet_manager.cc:1905] T d48d4517c13049389c5b2ba1ab7b14bc P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:11.544226 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 931769e78f32467b8f27bacb286a7a60 (table default.Table_b9a40e813ba14ce89495f9edffeb4ee0 [id=68697340a8b046d79b795c15a68e1c6f]) successfully deleted
W20260925 09:57:11.642104 10380 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 97900.2us
W20260925 09:57:11.643638 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 0e882284a014455192375daad391efaf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0e882284a014455192375daad391efaf already in progress: creating tablet
I20260925 09:57:11.643966 10392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_bb742c618d96453aaabe897d54fc657b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:11.645586 10392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bb742c618d96453aaabe897d54fc657b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:11.651655 11084 tablet_bootstrap.cc:654] T 91c09118b43b41eca8f7040323ba76f7 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:11.651808 10379 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 0a2bbe141e564a96be05de86f2993234 (table default.Table_8705f75ef7b2448ab645f1d25e3c2637 [id=aed8a2ddc51545ad9cec4d272dcf3daf], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 0a2bbe141e564a96be05de86f2993234 consensus state has no leader
W20260925 09:57:11.656497 10393 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 28d60531970a4ab58ce5ef99cb3a96bf consensus state has no leader
W20260925 09:57:11.662791 10385 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.1:56563 (request call id 21) took 844 ms (client timeout 1000 ms). Trace:
W20260925 09:57:11.663247 10385 rpcz_store.cc:269] 0925 09:57:10.818505 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:10.843789 (+ 25284us) service_pool.cc:227] Handling call
0925 09:57:11.662715 (+818926us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:11.202754 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:11.203083 (+ 329us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:11.203106 (+ 23us) write_op.cc:435] Acquired schema lock
0925 09:57:11.203118 (+ 12us) tablet.cc:660] Decoding operations
0925 09:57:11.203360 (+ 242us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:11.203426 (+ 66us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:11.203447 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:11.203585 (+ 138us) tablet.cc:699] Row locks acquired
0925 09:57:11.203600 (+ 15us) write_op.cc:260] PREPARE: finished
0925 09:57:11.203703 (+ 103us) write_op.cc:270] Start()
0925 09:57:11.203790 (+ 87us) write_op.cc:276] Timestamp: P: 1790330231203655 usec, L: 0
0925 09:57:11.203806 (+ 16us) op_driver.cc:331] REPLICATION: starting
0925 09:57:11.204256 (+ 450us) log.cc:842] Serialized 523 byte log entry
0925 09:57:11.213176 (+ 8920us) op_driver.cc:444] REPLICATION: finished
0925 09:57:11.213557 (+ 381us) write_op.cc:301] APPLY: starting
0925 09:57:11.213603 (+ 46us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:11.213698 (+ 95us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:11.213786 (+ 88us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:11.213922 (+ 136us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:11.214039 (+ 117us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:11.214076 (+ 37us) write_op.cc:312] APPLY: finished
0925 09:57:11.214233 (+ 157us) log.cc:842] Serialized 22 byte log entry
0925 09:57:11.214361 (+ 128us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:11.214422 (+ 61us) write_op.cc:454] Released schema lock
0925 09:57:11.214450 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:11.214601 (+ 151us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:11.214624 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":17,"child_traces":[["op",{"apply.queue_time_us":234,"apply.run_cpu_time_us":1232,"apply.run_wall_time_us":1232,"num_ops":1,"prepare.queue_time_us":38889,"prepare.run_cpu_time_us":1805,"prepare.run_wall_time_us":1805,"raft.queue_time_us":2403,"raft.run_cpu_time_us":730,"raft.run_wall_time_us":731,"replication_time_us":9301}]]}
I20260925 09:57:11.682561 11245 ts_tablet_manager.cc:1918] T d48d4517c13049389c5b2ba1ab7b14bc P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:11.691262 11245 log.cc:1196] T d48d4517c13049389c5b2ba1ab7b14bc P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/d48d4517c13049389c5b2ba1ab7b14bc
I20260925 09:57:11.691717 11084 tablet_bootstrap.cc:492] T 91c09118b43b41eca8f7040323ba76f7 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:11.692206 11084 ts_tablet_manager.cc:1397] T 91c09118b43b41eca8f7040323ba76f7 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.067s user 0.011s sys 0.014s
I20260925 09:57:11.692382 11245 ts_tablet_manager.cc:1939] T d48d4517c13049389c5b2ba1ab7b14bc P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:11.697316 10389 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_3b9695593bd6426e8932f7b11f01d1ac" } modify_external_catalogs: true
I20260925 09:57:11.697971 10389 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_3b9695593bd6426e8932f7b11f01d1ac" } modify_external_catalogs: true
I20260925 09:57:11.706946 11245 tablet_replica.cc:330] T 840faf042d8749bd958ce1de0207a734 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:11.707321 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet d48d4517c13049389c5b2ba1ab7b14bc (table default.Table_699fb6630e69435d946304ff8473d1e8 [id=1b1ce9c0cada4beeae7e02eb3105a564]) successfully deleted
I20260925 09:57:11.708752 11245 raft_consensus.cc:2229] T 840faf042d8749bd958ce1de0207a734 P be97ab5d75934a648d4ab76341193ec0 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:11.708424 11084 raft_consensus.cc:348] T 91c09118b43b41eca8f7040323ba76f7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:11.709285 11084 raft_consensus.cc:374] T 91c09118b43b41eca8f7040323ba76f7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:11.709597 11084 raft_consensus.cc:729] T 91c09118b43b41eca8f7040323ba76f7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:11.709836 11245 raft_consensus.cc:2258] T 840faf042d8749bd958ce1de0207a734 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:11.710445 11084 consensus_queue.cc:260] T 91c09118b43b41eca8f7040323ba76f7 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:11.720626 10385 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_8dc254af52974ea2827405df2b0a5961" } new_table_name: "default.Table_650d171329b0468fbdf510ae8e4ce99f" modify_external_catalogs: true
I20260925 09:57:11.721372 11245 ts_tablet_manager.cc:1905] T 840faf042d8749bd958ce1de0207a734 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:11.723913 11082 tablet_bootstrap.cc:492] T 8b281fc20e6f42239fee80d05da88ab6 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:11.724299 11082 ts_tablet_manager.cc:1397] T 8b281fc20e6f42239fee80d05da88ab6 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.682s user 0.019s sys 0.024s
W20260925 09:57:11.746588 10385 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ef6fefaa76d44fe798201aab2f21b9c7 has no consensus state
I20260925 09:57:11.748713 11084 ts_tablet_manager.cc:1428] T 91c09118b43b41eca8f7040323ba76f7 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.056s user 0.006s sys 0.000s
I20260925 09:57:11.751657 11084 tablet_bootstrap.cc:492] T 47a9fb9392244e1d89f8af2bfb42edd6 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:11.658694 10816 tablet_service.cc:1514] Processing DeleteTablet for tablet aad3cb5e33f64cb885d51f4ea089307b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:11 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:11.784407 11085 ts_tablet_manager.cc:1428] T 8b281fc20e6f42239fee80d05da88ab6 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.747s user 0.010s sys 0.001s
W20260925 09:57:11.785023 11085 ts_tablet_manager.cc:1460] T 8b281fc20e6f42239fee80d05da88ab6 P c43461c4ee854ae1ac52b873ccada318: Tablet startup took 747ms
W20260925 09:57:11.785583 11085 ts_tablet_manager.cc:1462] T 8b281fc20e6f42239fee80d05da88ab6 P c43461c4ee854ae1ac52b873ccada318: Trace:
0925 09:57:09.225526 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:09.225661 (+ 135us) service_pool.cc:227] Handling call
0925 09:57:09.227812 (+ 2151us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0925 09:57:09.227894 (+ 82us) ts_tablet_manager.cc:676] Creating new metadata...
0925 09:57:09.236074 (+ 8180us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:09.258738 (+ 22664us) tablet_replica.cc:205] Creating consensus instance
0925 09:57:09.259144 (+ 406us) inbound_call.cc:173] Queueing success response
0925 09:57:10.742165 (+1483021us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0925 09:57:11.037510 (+295345us) ts_tablet_manager.cc:1429] Starting tablet replica
0925 09:57:11.037965 (+ 455us) tablet_replica.cc:259] Starting instrumentation
0925 09:57:11.038359 (+ 394us) tablet_replica.cc:278] Starting consensus
Metrics: {"raft.queue_time_us":310,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":216,"tablet-open.queue_time_us":1483100}
I20260925 09:57:11.788086 10838 raft_consensus.cc:2454] T 0a2bbe141e564a96be05de86f2993234 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:11.793946 11245 ts_tablet_manager.cc:1918] T 840faf042d8749bd958ce1de0207a734 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260925 09:57:11.794405 11245 log.cc:1196] T 840faf042d8749bd958ce1de0207a734 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/840faf042d8749bd958ce1de0207a734
I20260925 09:57:11.794482 11085 tablet_bootstrap.cc:492] T 2d661d5000894c45ac3d1c27f0e72889 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:11.796458 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c4e8790282a49fd849a34ca11bef5bf" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:11.799311 10775 leader_election.cc:302] T e57065f2bb224a56ad1c3518a0c375da P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:11.808298 11245 ts_tablet_manager.cc:1939] T 840faf042d8749bd958ce1de0207a734 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
W20260925 09:57:11.810761 11338 log.cc:924] Time spent T 4be3f6e81ddf418fabc7b694a2e62f35 P b8850818e2ee42a082802d0095c08d21: Append to log took a long time: real 0.175s user 0.000s sys 0.000s
I20260925 09:57:11.813540 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 840faf042d8749bd958ce1de0207a734 (table default.Table_faa21794a80c407f800038e583c08f16 [id=800eba46046443a7ad196808dec4cdc0]) successfully deleted
I20260925 09:57:11.814581 11245 tablet_replica.cc:330] T 6b03e5aad59e46a6b0e43b9cb36278d7 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:11.815979 10658 tablet_service.cc:1514] Processing DeleteTablet for tablet d48d4517c13049389c5b2ba1ab7b14bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:07 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:11.816847 10666 tablet_service.cc:1514] Processing DeleteTablet for tablet 931769e78f32467b8f27bacb286a7a60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:09 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:11.816912 10658 tablet_service.cc:1514] Processing DeleteTablet for tablet d8be238061c146eb96d17a22c1b2104a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:08 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:11.810168 11298 ts_tablet_manager.cc:1905] T 6b03e5aad59e46a6b0e43b9cb36278d7 P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:11.807510 11301 consensus_queue.cc:1035] T 59f0c6489a33433899bb6e62088121b6 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260925 09:57:11.806828 11299 consensus_queue.cc:1035] T 04fafb9dcf274806b11b2598e73853e9 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
I20260925 09:57:11.834951 11245 raft_consensus.cc:2229] T 6b03e5aad59e46a6b0e43b9cb36278d7 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:11.835615 11245 raft_consensus.cc:2258] T 6b03e5aad59e46a6b0e43b9cb36278d7 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:11.837102 11084 tablet_bootstrap.cc:654] T 47a9fb9392244e1d89f8af2bfb42edd6 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:11.838435 11245 ts_tablet_manager.cc:1905] T 6b03e5aad59e46a6b0e43b9cb36278d7 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:11.847049 10392 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_beb33698ec744e42b4f6ac2fdd8fe2b9" } modify_external_catalogs: true
I20260925 09:57:11.847637 10392 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_beb33698ec744e42b4f6ac2fdd8fe2b9" } modify_external_catalogs: true
I20260925 09:57:11.849308 10393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_3f0e11249b90447595581ed433d20c5b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:11.850207 11292 ts_tablet_manager.cc:1918] T 6b03e5aad59e46a6b0e43b9cb36278d7 P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:11.850695 11292 log.cc:1196] T 6b03e5aad59e46a6b0e43b9cb36278d7 P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/6b03e5aad59e46a6b0e43b9cb36278d7
W20260925 09:57:11.850734 10393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3f0e11249b90447595581ed433d20c5b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:11.852151 11292 ts_tablet_manager.cc:1939] T 6b03e5aad59e46a6b0e43b9cb36278d7 P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
W20260925 09:57:11.854417 10918 leader_election.cc:341] T 4c4e8790282a49fd849a34ca11bef5bf P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:11.854803 10918 leader_election.cc:302] T 4c4e8790282a49fd849a34ca11bef5bf P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:11.855214 11292 tablet_replica.cc:330] T 840faf042d8749bd958ce1de0207a734 P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:11.855931 11134 raft_consensus.cc:2726] T 4c4e8790282a49fd849a34ca11bef5bf P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:11.855994 11292 raft_consensus.cc:2229] T 840faf042d8749bd958ce1de0207a734 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:11.856477 11292 raft_consensus.cc:2258] T 840faf042d8749bd958ce1de0207a734 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:11.858443 11292 ts_tablet_manager.cc:1905] T 840faf042d8749bd958ce1de0207a734 P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:11.860788 10391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_0507f47042b9445abaf8671c979f8994"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:11.862489 10391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0507f47042b9445abaf8671c979f8994 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:11.867468 11245 ts_tablet_manager.cc:1918] T 6b03e5aad59e46a6b0e43b9cb36278d7 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:11.867885 11245 log.cc:1196] T 6b03e5aad59e46a6b0e43b9cb36278d7 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/6b03e5aad59e46a6b0e43b9cb36278d7
I20260925 09:57:11.868671 11245 ts_tablet_manager.cc:1939] T 6b03e5aad59e46a6b0e43b9cb36278d7 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:11.870384 11348 raft_consensus.cc:482] T 8b281fc20e6f42239fee80d05da88ab6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:11.870900 11348 raft_consensus.cc:504] T 8b281fc20e6f42239fee80d05da88ab6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:11.871783 11245 tablet_replica.cc:330] T d8be238061c146eb96d17a22c1b2104a P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:11.873073 11348 leader_election.cc:288] T 8b281fc20e6f42239fee80d05da88ab6 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:11.873559 11245 raft_consensus.cc:2229] T d8be238061c146eb96d17a22c1b2104a P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:11.873982 11245 raft_consensus.cc:2258] T d8be238061c146eb96d17a22c1b2104a P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:11.874228 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 6b03e5aad59e46a6b0e43b9cb36278d7 (table default.Table_534d354d5b4f4f90b8f80a76ddbc8214 [id=2c0582d5ad4b4eab8db802302c925de1]) successfully deleted
I20260925 09:57:11.874619 10962 tablet_service.cc:1467] Processing CreateTablet for tablet 2d55154ceb064f22bd6f37c8ae5dce74 (DEFAULT_TABLE table=default.Table_7eaf247fb74f497d80a88b67fa301cab [id=efe6d05ed9084a1c998e95e99672d4fc]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 09:57:11.874845 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 931769e78f32467b8f27bacb286a7a60 because the tablet was not found. No further retry: Not found: Tablet not found: 931769e78f32467b8f27bacb286a7a60
I20260925 09:57:11.875563 10962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d55154ceb064f22bd6f37c8ae5dce74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:11.875770 11245 ts_tablet_manager.cc:1905] T d8be238061c146eb96d17a22c1b2104a P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:11.877856 10386 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_6b025220a9aa49ecb47a7e17824f0a71" } new_table_name: "default.Table_33b40a0d6ca44f15be47ca34bab6a093" modify_external_catalogs: true
I20260925 09:57:11.895336 10388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_8c5b820f0db34e3bbea2d404afb1909a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:11.897032 10388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8c5b820f0db34e3bbea2d404afb1909a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:11.897290 10390 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 8aae639695f44d8c8a61b3ea42ab6ac8 (table default.Table_396cf11a26a04a17a2ad247fdf6a6c1b [id=e493befbf9d94366aa09d518014389eb]). Expected version 1 got 0
I20260925 09:57:11.898392 10390 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 0a2bbe141e564a96be05de86f2993234 (table default.Table_8705f75ef7b2448ab645f1d25e3c2637 [id=aed8a2ddc51545ad9cec4d272dcf3daf]). Expected version 1 got 0
I20260925 09:57:11.900691 10390 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet b7646a1cde1b45b9bf7928c3e13bcbd3 (table default.Table_9f64de2cbc4c4968bc0cfe51d0a9a487 [id=11e4b3a3251f4fa4a78de075f8ecb6f2]). Expected version 1 got 0
I20260925 09:57:11.902043 10389 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet e48d86c20e9e4db990c23359ff1d1a6d
I20260925 09:57:11.906001 10963 tablet_service.cc:1514] Processing DeleteTablet for tablet e48d86c20e9e4db990c23359ff1d1a6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:11 UTC) from {username='slave'} at 127.0.0.1:53560
W20260925 09:57:11.877656 10775 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 128546us
I20260925 09:57:11.830094 11346 raft_consensus.cc:2726] T e57065f2bb224a56ad1c3518a0c375da P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:11.881047 11082 raft_consensus.cc:348] T 8b281fc20e6f42239fee80d05da88ab6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:11.912787 11082 raft_consensus.cc:374] T 8b281fc20e6f42239fee80d05da88ab6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:11.913110 11082 raft_consensus.cc:729] T 8b281fc20e6f42239fee80d05da88ab6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:11.916019 11300 raft_consensus.cc:482] T 8c13a2f288b14d03a3ca4fdd4889cbad P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:11.918541 11300 raft_consensus.cc:504] T 8c13a2f288b14d03a3ca4fdd4889cbad P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:11.920487 11300 leader_election.cc:288] T 8c13a2f288b14d03a3ca4fdd4889cbad P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:11.922101 10666 tablet_service.cc:1514] Processing DeleteTablet for tablet e48d86c20e9e4db990c23359ff1d1a6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:11 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:11.922896 10385 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 8aae639695f44d8c8a61b3ea42ab6ac8 (table default.Table_396cf11a26a04a17a2ad247fdf6a6c1b [id=e493befbf9d94366aa09d518014389eb]). Expected version 1 got 0
I20260925 09:57:11.923095 10658 tablet_service.cc:1467] Processing CreateTablet for tablet 2d55154ceb064f22bd6f37c8ae5dce74 (DEFAULT_TABLE table=default.Table_7eaf247fb74f497d80a88b67fa301cab [id=efe6d05ed9084a1c998e95e99672d4fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:11.924379 10658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d55154ceb064f22bd6f37c8ae5dce74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:11.924340 10385 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 0a2bbe141e564a96be05de86f2993234 (table default.Table_8705f75ef7b2448ab645f1d25e3c2637 [id=aed8a2ddc51545ad9cec4d272dcf3daf]). Expected version 1 got 0
I20260925 09:57:11.928258 10661 tablet_service.cc:1467] Processing CreateTablet for tablet 3c31682bf400494f82ee3ce5e6eec441 (DEFAULT_TABLE table=default.Table_d3efd9bb6b174cdfbdbd1af72107b558 [id=0040224e69114e918cec485791935895]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:11.929432 10661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c31682bf400494f82ee3ce5e6eec441. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:11.942270 11351 tablet.cc:1722] T 8aae639695f44d8c8a61b3ea42ab6ac8 P be97ab5d75934a648d4ab76341193ec0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:11.916807 11082 consensus_queue.cc:260] T 8b281fc20e6f42239fee80d05da88ab6 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:11.946408 10393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_5157be3699be4fa4ac153a69f6010ecc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:11.947878 10393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5157be3699be4fa4ac153a69f6010ecc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:11.949743 11245 ts_tablet_manager.cc:1918] T d8be238061c146eb96d17a22c1b2104a P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:11.950120 11245 log.cc:1196] T d8be238061c146eb96d17a22c1b2104a P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/d8be238061c146eb96d17a22c1b2104a
I20260925 09:57:11.950780 11245 ts_tablet_manager.cc:1939] T d8be238061c146eb96d17a22c1b2104a P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:11.957464 11082 ts_tablet_manager.cc:1428] T 8b281fc20e6f42239fee80d05da88ab6 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.233s user 0.003s sys 0.002s
I20260925 09:57:11.958437 11082 tablet_bootstrap.cc:492] T b7646a1cde1b45b9bf7928c3e13bcbd3 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
W20260925 09:57:11.958540 10619 leader_election.cc:341] T 8c13a2f288b14d03a3ca4fdd4889cbad P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 8c13a2f288b14d03a3ca4fdd4889cbad
I20260925 09:57:11.958959 11245 tablet_replica.cc:330] T aad3cb5e33f64cb885d51f4ea089307b P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:11.960198 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet d8be238061c146eb96d17a22c1b2104a (table default.Table_2bc55a20487a41b182135205d1a4b65d [id=f97ee2a971ab46abb4cbb37169fb309d]) successfully deleted
I20260925 09:57:11.960610 11206 raft_consensus.cc:482] T 69f84509cafc44dd9444218738ebc2fe P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:11.961186 11206 raft_consensus.cc:504] T 69f84509cafc44dd9444218738ebc2fe P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:11.962561 10389 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_4e09a615fb82487baa4783a5f6e2181e" } new_table_name: "default.Table_7104d0f2dac14fa0a59685700d80c110" modify_external_catalogs: true
I20260925 09:57:11.963028 11206 leader_election.cc:288] T 69f84509cafc44dd9444218738ebc2fe P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
W20260925 09:57:11.963625 10386 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 91c09118b43b41eca8f7040323ba76f7 consensus state has no leader
I20260925 09:57:11.964855 10683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69f84509cafc44dd9444218738ebc2fe" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:11.965063 11245 raft_consensus.cc:2229] T aad3cb5e33f64cb885d51f4ea089307b P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:11.965462 11245 raft_consensus.cc:2258] T aad3cb5e33f64cb885d51f4ea089307b P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 09:57:11.966207 10919 leader_election.cc:341] T 69f84509cafc44dd9444218738ebc2fe P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:11.970407 11245 ts_tablet_manager.cc:1905] T aad3cb5e33f64cb885d51f4ea089307b P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:11.984889 10392 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 4c4e8790282a49fd849a34ca11bef5bf
I20260925 09:57:11.987685 10666 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c4e8790282a49fd849a34ca11bef5bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:11 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:11.988171 10963 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c4e8790282a49fd849a34ca11bef5bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:11 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:12.003736 11206 raft_consensus.cc:482] T 3fa527403d98425ea5cff9215d3a5e7f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:12.004280 11206 raft_consensus.cc:504] T 3fa527403d98425ea5cff9215d3a5e7f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:12.006495 11206 leader_election.cc:288] T 3fa527403d98425ea5cff9215d3a5e7f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:12.008174 10683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fa527403d98425ea5cff9215d3a5e7f" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:12.009629 10919 leader_election.cc:341] T 3fa527403d98425ea5cff9215d3a5e7f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:12.013947 11082 tablet_bootstrap.cc:654] T b7646a1cde1b45b9bf7928c3e13bcbd3 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:12.022339 11245 ts_tablet_manager.cc:1918] T aad3cb5e33f64cb885d51f4ea089307b P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:12.026743 11245 log.cc:1196] T aad3cb5e33f64cb885d51f4ea089307b P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/aad3cb5e33f64cb885d51f4ea089307b
I20260925 09:57:12.027863 11245 ts_tablet_manager.cc:1939] T aad3cb5e33f64cb885d51f4ea089307b P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:12.031205 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet aad3cb5e33f64cb885d51f4ea089307b (table default.Table_2d7acf0d8022443090ba12cd87956502 [id=9d2f4f9f46394332bec9f8beda0d9595]) successfully deleted
I20260925 09:57:12.034548 11245 tablet_replica.cc:330] T e48d86c20e9e4db990c23359ff1d1a6d P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:12.035915 11245 raft_consensus.cc:2229] T e48d86c20e9e4db990c23359ff1d1a6d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:12.037284 11245 raft_consensus.cc:2258] T e48d86c20e9e4db990c23359ff1d1a6d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:12.040870 11245 ts_tablet_manager.cc:1905] T e48d86c20e9e4db990c23359ff1d1a6d P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:12.067808 11245 ts_tablet_manager.cc:1918] T e48d86c20e9e4db990c23359ff1d1a6d P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:12.071781 11245 log.cc:1196] T e48d86c20e9e4db990c23359ff1d1a6d P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/e48d86c20e9e4db990c23359ff1d1a6d
I20260925 09:57:12.073104 11245 ts_tablet_manager.cc:1939] T e48d86c20e9e4db990c23359ff1d1a6d P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:12.078576 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet e48d86c20e9e4db990c23359ff1d1a6d (table default.Table_3b9695593bd6426e8932f7b11f01d1ac [id=c524d4b2bbb1495cbe85a6f1b3d595d2]) successfully deleted
I20260925 09:57:12.079895 11245 tablet_replica.cc:330] T 4c4e8790282a49fd849a34ca11bef5bf P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:12.081379 11245 raft_consensus.cc:2229] T 4c4e8790282a49fd849a34ca11bef5bf P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:12.082688 11245 raft_consensus.cc:2258] T 4c4e8790282a49fd849a34ca11bef5bf P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:12.084667 11245 ts_tablet_manager.cc:1905] T 4c4e8790282a49fd849a34ca11bef5bf P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:12.091003 10388 catalog_manager.cc:5495] T 59f0c6489a33433899bb6e62088121b6 P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: HEALTHY } } }
I20260925 09:57:12.096156 10390 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: default.Table_1e7379199917430eb9e027fc7bb2a4f9 [id=c2d0336fd84c4dbe95472f458b813fe6] alter complete (version 1)
I20260925 09:57:12.105526 10963 tablet_service.cc:1467] Processing CreateTablet for tablet 3c31682bf400494f82ee3ce5e6eec441 (DEFAULT_TABLE table=default.Table_d3efd9bb6b174cdfbdbd1af72107b558 [id=0040224e69114e918cec485791935895]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:12.106904 10963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c31682bf400494f82ee3ce5e6eec441. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:12.107023 10666 tablet_service.cc:1467] Processing CreateTablet for tablet ef22f64036684dce9d7c69c37722f468 (DEFAULT_TABLE table=default.Table_d586d012b3534ee79591128c9aac74ac [id=a20216b9b3074e408bee74e39c5819b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:12.108057 10666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef22f64036684dce9d7c69c37722f468. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:12.110579 10392 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_3bc500239afb4c5eae08ecf1401ae72f" } modify_external_catalogs: true
I20260925 09:57:12.111217 10392 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_3bc500239afb4c5eae08ecf1401ae72f" } modify_external_catalogs: true
I20260925 09:57:12.115716 11245 ts_tablet_manager.cc:1918] T 4c4e8790282a49fd849a34ca11bef5bf P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:12.116047 11245 log.cc:1196] T 4c4e8790282a49fd849a34ca11bef5bf P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/4c4e8790282a49fd849a34ca11bef5bf
I20260925 09:57:12.116644 11245 ts_tablet_manager.cc:1939] T 4c4e8790282a49fd849a34ca11bef5bf P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:12.119326 10388 catalog_manager.cc:5495] T 0a2bbe141e564a96be05de86f2993234 P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:12.121039 10388 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 0a2bbe141e564a96be05de86f2993234 (table default.Table_8705f75ef7b2448ab645f1d25e3c2637 [id=aed8a2ddc51545ad9cec4d272dcf3daf]). Expected version 1 got 0
I20260925 09:57:12.123509 11085 tablet_bootstrap.cc:654] T 2d661d5000894c45ac3d1c27f0e72889 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:12.124693 10391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_055cf78170f24a8e94074c1f02e2f5a3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:12.126772 10391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_055cf78170f24a8e94074c1f02e2f5a3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:12.023017 11298 ts_tablet_manager.cc:1918] T 6b03e5aad59e46a6b0e43b9cb36278d7 P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:12.128175 11298 log.cc:1196] T 6b03e5aad59e46a6b0e43b9cb36278d7 P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/6b03e5aad59e46a6b0e43b9cb36278d7
I20260925 09:57:12.129012 11298 ts_tablet_manager.cc:1939] T 6b03e5aad59e46a6b0e43b9cb36278d7 P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:12.129915 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b281fc20e6f42239fee80d05da88ab6" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:12.130499 10984 raft_consensus.cc:2454] T 8b281fc20e6f42239fee80d05da88ab6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
I20260925 09:57:12.136619 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 6b03e5aad59e46a6b0e43b9cb36278d7 (table default.Table_534d354d5b4f4f90b8f80a76ddbc8214 [id=2c0582d5ad4b4eab8db802302c925de1]) successfully deleted
I20260925 09:57:12.137621 11134 raft_consensus.cc:482] T 833fac37485141f6a18104eaea907b0e P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:12.138113 11134 raft_consensus.cc:504] T 833fac37485141f6a18104eaea907b0e P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:12.133210 10388 catalog_manager.cc:5495] T 04fafb9dcf274806b11b2598e73853e9 P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: HEALTHY } } }
I20260925 09:57:12.143311 10683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "833fac37485141f6a18104eaea907b0e" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:12.144735 10919 leader_election.cc:341] T 833fac37485141f6a18104eaea907b0e P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:12.146054 11134 leader_election.cc:288] T 833fac37485141f6a18104eaea907b0e P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:12.155210 11082 tablet_bootstrap.cc:492] T b7646a1cde1b45b9bf7928c3e13bcbd3 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:12.155617 11082 ts_tablet_manager.cc:1397] T b7646a1cde1b45b9bf7928c3e13bcbd3 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.197s user 0.007s sys 0.004s
I20260925 09:57:12.157968 11082 raft_consensus.cc:348] T b7646a1cde1b45b9bf7928c3e13bcbd3 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:12.158636 11082 raft_consensus.cc:374] T b7646a1cde1b45b9bf7928c3e13bcbd3 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:12.158901 11082 raft_consensus.cc:729] T b7646a1cde1b45b9bf7928c3e13bcbd3 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:12.171674 10389 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_7d0a974dfa9a48c496f3128d28fe1d15" } new_table_name: "default.Table_0011387b45014de4b24b466f4cb907f7" modify_external_catalogs: true
I20260925 09:57:12.172710 11211 consensus_queue.cc:1035] T e57065f2bb224a56ad1c3518a0c375da P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:12.176898 10683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b281fc20e6f42239fee80d05da88ab6" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:12.177166 10619 consensus_peers.cc:487] T d48d4517c13049389c5b2ba1ab7b14bc P b8850818e2ee42a082802d0095c08d21 -> Peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Couldn't send request to peer be97ab5d75934a648d4ab76341193ec0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d48d4517c13049389c5b2ba1ab7b14bc. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:12.177485 10683 raft_consensus.cc:2454] T 8b281fc20e6f42239fee80d05da88ab6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
I20260925 09:57:12.187855 10386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_1299538965b945cf859dbbd0cb7003f1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:12.189996 10386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1299538965b945cf859dbbd0cb7003f1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:12.193575 11357 tablet.cc:1722] T 8aae639695f44d8c8a61b3ea42ab6ac8 P b8850818e2ee42a082802d0095c08d21: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260925 09:57:12.196099 10772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 56641.8us
I20260925 09:57:12.197002 10772 leader_election.cc:302] T 8b281fc20e6f42239fee80d05da88ab6 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:12.198155 10384 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_ae6bc3dfc98244fda3409f53da244f7a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:12.199730 10387 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_9f64de2cbc4c4968bc0cfe51d0a9a487" } modify_external_catalogs: true
I20260925 09:57:12.200176 11249 raft_consensus.cc:2781] T 8b281fc20e6f42239fee80d05da88ab6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:12.200453 11249 raft_consensus.cc:482] T 8b281fc20e6f42239fee80d05da88ab6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:12.200736 11249 raft_consensus.cc:3037] T 8b281fc20e6f42239fee80d05da88ab6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:57:12.201115 10384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ae6bc3dfc98244fda3409f53da244f7a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:12.204978 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69f84509cafc44dd9444218738ebc2fe" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:12.206347 11298 tablet_replica.cc:330] T 840faf042d8749bd958ce1de0207a734 P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:12.207228 11298 raft_consensus.cc:2229] T 840faf042d8749bd958ce1de0207a734 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:12.205685 10962 tablet_service.cc:1467] Processing CreateTablet for tablet ef22f64036684dce9d7c69c37722f468 (DEFAULT_TABLE table=default.Table_d586d012b3534ee79591128c9aac74ac [id=a20216b9b3074e408bee74e39c5819b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:12.209422 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "833fac37485141f6a18104eaea907b0e" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:12.210747 10918 leader_election.cc:341] T 69f84509cafc44dd9444218738ebc2fe P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:12.211099 10918 leader_election.cc:302] T 69f84509cafc44dd9444218738ebc2fe P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:12.214558 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fa527403d98425ea5cff9215d3a5e7f" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:12.216077 10918 leader_election.cc:341] T 833fac37485141f6a18104eaea907b0e P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:12.216414 10918 leader_election.cc:302] T 833fac37485141f6a18104eaea907b0e P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:12.217289 11206 raft_consensus.cc:2726] T 69f84509cafc44dd9444218738ebc2fe P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:12.217792 11206 raft_consensus.cc:2726] T 833fac37485141f6a18104eaea907b0e P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:12.221758 10918 leader_election.cc:341] T 3fa527403d98425ea5cff9215d3a5e7f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:12.222117 10918 leader_election.cc:302] T 3fa527403d98425ea5cff9215d3a5e7f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:12.223057 11206 raft_consensus.cc:2726] T 3fa527403d98425ea5cff9215d3a5e7f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:12.228804 10387 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_9f64de2cbc4c4968bc0cfe51d0a9a487" } modify_external_catalogs: true
I20260925 09:57:12.234051 11249 raft_consensus.cc:504] T 8b281fc20e6f42239fee80d05da88ab6 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:12.241503 10619 consensus_peers.cc:487] T d8be238061c146eb96d17a22c1b2104a P b8850818e2ee42a082802d0095c08d21 -> Peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Couldn't send request to peer be97ab5d75934a648d4ab76341193ec0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d8be238061c146eb96d17a22c1b2104a. This is attempt 1: this message will repeat every 5th retry.
W20260925 09:57:12.243201 10619 consensus_peers.cc:487] T aad3cb5e33f64cb885d51f4ea089307b P b8850818e2ee42a082802d0095c08d21 -> Peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Couldn't send request to peer be97ab5d75934a648d4ab76341193ec0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: aad3cb5e33f64cb885d51f4ea089307b. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:12.244701 10962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef22f64036684dce9d7c69c37722f468. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:12.245379 11084 tablet_bootstrap.cc:492] T 47a9fb9392244e1d89f8af2bfb42edd6 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:12.245841 11084 ts_tablet_manager.cc:1397] T 47a9fb9392244e1d89f8af2bfb42edd6 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.495s user 0.011s sys 0.000s
I20260925 09:57:12.247677 11298 raft_consensus.cc:2258] T 840faf042d8749bd958ce1de0207a734 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:12.250011 11298 ts_tablet_manager.cc:1905] T 840faf042d8749bd958ce1de0207a734 P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:12.252514 11084 raft_consensus.cc:348] T 47a9fb9392244e1d89f8af2bfb42edd6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:12.253435 11084 raft_consensus.cc:374] T 47a9fb9392244e1d89f8af2bfb42edd6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:12.253741 11084 raft_consensus.cc:729] T 47a9fb9392244e1d89f8af2bfb42edd6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:12.254469 11084 consensus_queue.cc:260] T 47a9fb9392244e1d89f8af2bfb42edd6 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:12.265254 10379 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.60:36985, user_credentials={real_user=slave}} blocked reactor thread for 262846us
I20260925 09:57:12.268501 11249 leader_election.cc:288] T 8b281fc20e6f42239fee80d05da88ab6 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:12.270215 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b281fc20e6f42239fee80d05da88ab6" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:12.270803 10984 raft_consensus.cc:3037] T 8b281fc20e6f42239fee80d05da88ab6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:12.272805 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 4c4e8790282a49fd849a34ca11bef5bf (table default.Table_beb33698ec744e42b4f6ac2fdd8fe2b9 [id=9c778da25cde488a953f712100b2fbb2]) successfully deleted
I20260925 09:57:12.273582 11082 consensus_queue.cc:260] T b7646a1cde1b45b9bf7928c3e13bcbd3 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:12.273801 11362 tablet.cc:1722] T 8aae639695f44d8c8a61b3ea42ab6ac8 P c43461c4ee854ae1ac52b873ccada318: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:12.277175 11082 ts_tablet_manager.cc:1428] T b7646a1cde1b45b9bf7928c3e13bcbd3 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.121s user 0.004s sys 0.002s
I20260925 09:57:12.277944 10984 raft_consensus.cc:2454] T 8b281fc20e6f42239fee80d05da88ab6 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:12.279201 10772 leader_election.cc:302] T 8b281fc20e6f42239fee80d05da88ab6 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:12.280071 11249 raft_consensus.cc:2781] T 8b281fc20e6f42239fee80d05da88ab6 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:12.283017 11082 tablet_bootstrap.cc:492] T 2d661d5000894c45ac3d1c27f0e72889 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:12.285270 11249 raft_consensus.cc:686] T 8b281fc20e6f42239fee80d05da88ab6 P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Becoming Leader. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Running, Role: LEADER
I20260925 09:57:12.286099 11249 consensus_queue.cc:237] T 8b281fc20e6f42239fee80d05da88ab6 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:12.288753 11082 tablet_bootstrap.cc:654] T 2d661d5000894c45ac3d1c27f0e72889 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:12.294489 10814 tablet_service.cc:1467] Processing CreateTablet for tablet 2d55154ceb064f22bd6f37c8ae5dce74 (DEFAULT_TABLE table=default.Table_7eaf247fb74f497d80a88b67fa301cab [id=efe6d05ed9084a1c998e95e99672d4fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:12.296164 10814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d55154ceb064f22bd6f37c8ae5dce74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:12.304796 11085 tablet_bootstrap.cc:492] T 2d661d5000894c45ac3d1c27f0e72889 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:12.305294 11085 ts_tablet_manager.cc:1397] T 2d661d5000894c45ac3d1c27f0e72889 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.511s user 0.013s sys 0.004s
I20260925 09:57:12.307417 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 6b03e5aad59e46a6b0e43b9cb36278d7 (table default.Table_534d354d5b4f4f90b8f80a76ddbc8214 [id=2c0582d5ad4b4eab8db802302c925de1]) successfully deleted
I20260925 09:57:12.313146 11206 raft_consensus.cc:482] T 47a9fb9392244e1d89f8af2bfb42edd6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:12.313575 11206 raft_consensus.cc:504] T 47a9fb9392244e1d89f8af2bfb42edd6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:12.316819 10819 tablet_service.cc:1467] Processing CreateTablet for tablet 3c31682bf400494f82ee3ce5e6eec441 (DEFAULT_TABLE table=default.Table_d3efd9bb6b174cdfbdbd1af72107b558 [id=0040224e69114e918cec485791935895]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:12.317202 11206 leader_election.cc:288] T 47a9fb9392244e1d89f8af2bfb42edd6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:12.317888 10819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c31682bf400494f82ee3ce5e6eec441. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:12.321509 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47a9fb9392244e1d89f8af2bfb42edd6" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:12.322680 10918 leader_election.cc:341] T 47a9fb9392244e1d89f8af2bfb42edd6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:12.324493 11084 ts_tablet_manager.cc:1428] T 47a9fb9392244e1d89f8af2bfb42edd6 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.078s user 0.007s sys 0.000s
I20260925 09:57:12.325328 11084 tablet_bootstrap.cc:492] T 28d60531970a4ab58ce5ef99cb3a96bf P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:12.326238 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47a9fb9392244e1d89f8af2bfb42edd6" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:12.327355 10919 leader_election.cc:341] T 47a9fb9392244e1d89f8af2bfb42edd6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:12.327751 10919 leader_election.cc:302] T 47a9fb9392244e1d89f8af2bfb42edd6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:12.328892 11134 raft_consensus.cc:2726] T 47a9fb9392244e1d89f8af2bfb42edd6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:12.336138 11292 ts_tablet_manager.cc:1918] T 840faf042d8749bd958ce1de0207a734 P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260925 09:57:12.344302 11292 log.cc:1196] T 840faf042d8749bd958ce1de0207a734 P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/840faf042d8749bd958ce1de0207a734
W20260925 09:57:12.336664 10775 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 129264us
I20260925 09:57:12.337363 10816 tablet_service.cc:1514] Processing DeleteTablet for tablet e48d86c20e9e4db990c23359ff1d1a6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:11 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:12.347585 11292 ts_tablet_manager.cc:1939] T 840faf042d8749bd958ce1de0207a734 P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:12.338403 10813 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c4e8790282a49fd849a34ca11bef5bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:11 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:12.340148 10815 tablet_service.cc:1467] Processing CreateTablet for tablet ef22f64036684dce9d7c69c37722f468 (DEFAULT_TABLE table=default.Table_d586d012b3534ee79591128c9aac74ac [id=a20216b9b3074e408bee74e39c5819b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:12.349050 10815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef22f64036684dce9d7c69c37722f468. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:12.350338 11292 tablet_replica.cc:330] T d8be238061c146eb96d17a22c1b2104a P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:12.351171 11292 raft_consensus.cc:2229] T d8be238061c146eb96d17a22c1b2104a P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:12.351701 11292 raft_consensus.cc:2258] T d8be238061c146eb96d17a22c1b2104a P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:12.355679 10392 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 25cf462a99f644c8a890aa53af38dbcd
I20260925 09:57:12.355824 10385 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: default.Table_44abad8e8def4bf7be1d01c50a435298 [id=5e5f72cecad44b5e9bfd60320ee42243] alter complete (version 2)
I20260925 09:57:12.374625 11292 ts_tablet_manager.cc:1905] T d8be238061c146eb96d17a22c1b2104a P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:12.388185 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b281fc20e6f42239fee80d05da88ab6" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:12.388787 10686 raft_consensus.cc:3037] T 8b281fc20e6f42239fee80d05da88ab6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:12.395591 10686 raft_consensus.cc:2454] T 8b281fc20e6f42239fee80d05da88ab6 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:12.399443 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26b7c37959d042b48234acc79290936f" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:12.402969 11368 tablet.cc:1722] T 4be3f6e81ddf418fabc7b694a2e62f35 P b8850818e2ee42a082802d0095c08d21: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:12.407704 11134 raft_consensus.cc:482] T 91c09118b43b41eca8f7040323ba76f7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:12.408349 11134 raft_consensus.cc:504] T 91c09118b43b41eca8f7040323ba76f7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:12.410118 11134 leader_election.cc:288] T 91c09118b43b41eca8f7040323ba76f7 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:12.411533 10837 raft_consensus.cc:3037] T 26b7c37959d042b48234acc79290936f P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:12.412501 10683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91c09118b43b41eca8f7040323ba76f7" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:12.413852 10919 leader_election.cc:341] T 91c09118b43b41eca8f7040323ba76f7 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:12.423784 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91c09118b43b41eca8f7040323ba76f7" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:12.427098 11084 tablet_bootstrap.cc:654] T 28d60531970a4ab58ce5ef99cb3a96bf P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:12.439493 11292 ts_tablet_manager.cc:1918] T d8be238061c146eb96d17a22c1b2104a P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:12.439927 11292 log.cc:1196] T d8be238061c146eb96d17a22c1b2104a P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/d8be238061c146eb96d17a22c1b2104a
I20260925 09:57:12.440580 11292 ts_tablet_manager.cc:1939] T d8be238061c146eb96d17a22c1b2104a P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
W20260925 09:57:12.446110 10918 leader_election.cc:341] T 91c09118b43b41eca8f7040323ba76f7 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:12.446507 10918 leader_election.cc:302] T 91c09118b43b41eca8f7040323ba76f7 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:12.447432 11134 raft_consensus.cc:2726] T 91c09118b43b41eca8f7040323ba76f7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:12.458554 11085 raft_consensus.cc:348] T 2d661d5000894c45ac3d1c27f0e72889 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:12.462229 11085 raft_consensus.cc:374] T 2d661d5000894c45ac3d1c27f0e72889 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:12.466835 11085 raft_consensus.cc:729] T 2d661d5000894c45ac3d1c27f0e72889 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:12.470436 11292 tablet_replica.cc:330] T d48d4517c13049389c5b2ba1ab7b14bc P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:12.471235 11292 raft_consensus.cc:2229] T d48d4517c13049389c5b2ba1ab7b14bc P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:12.474291 10834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c13a2f288b14d03a3ca4fdd4889cbad" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:12.476552 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 840faf042d8749bd958ce1de0207a734 (table default.Table_faa21794a80c407f800038e583c08f16 [id=800eba46046443a7ad196808dec4cdc0]) successfully deleted
I20260925 09:57:12.479182 11211 raft_consensus.cc:482] T b7646a1cde1b45b9bf7928c3e13bcbd3 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:12.479630 11211 raft_consensus.cc:504] T b7646a1cde1b45b9bf7928c3e13bcbd3 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:12.481432 11211 leader_election.cc:288] T b7646a1cde1b45b9bf7928c3e13bcbd3 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:12.481882 11082 tablet_bootstrap.cc:492] T 2d661d5000894c45ac3d1c27f0e72889 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:12.482299 11082 ts_tablet_manager.cc:1397] T 2d661d5000894c45ac3d1c27f0e72889 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.199s user 0.011s sys 0.006s
I20260925 09:57:12.482949 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7646a1cde1b45b9bf7928c3e13bcbd3" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:12.483594 10984 raft_consensus.cc:2454] T b7646a1cde1b45b9bf7928c3e13bcbd3 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:12.484479 11082 raft_consensus.cc:348] T 2d661d5000894c45ac3d1c27f0e72889 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:12.484735 10619 leader_election.cc:302] T b7646a1cde1b45b9bf7928c3e13bcbd3 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:12.485096 11082 raft_consensus.cc:374] T 2d661d5000894c45ac3d1c27f0e72889 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:12.485347 11082 raft_consensus.cc:729] T 2d661d5000894c45ac3d1c27f0e72889 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:12.485491 11211 raft_consensus.cc:2781] T b7646a1cde1b45b9bf7928c3e13bcbd3 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:12.485790 11211 raft_consensus.cc:482] T b7646a1cde1b45b9bf7928c3e13bcbd3 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:12.486088 11211 raft_consensus.cc:3037] T b7646a1cde1b45b9bf7928c3e13bcbd3 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:12.485918 11082 consensus_queue.cc:260] T 2d661d5000894c45ac3d1c27f0e72889 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:12.497275 11211 raft_consensus.cc:504] T b7646a1cde1b45b9bf7928c3e13bcbd3 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:12.500502 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7646a1cde1b45b9bf7928c3e13bcbd3" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:12.501149 10984 raft_consensus.cc:3037] T b7646a1cde1b45b9bf7928c3e13bcbd3 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:12.502488 10658 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b03e5aad59e46a6b0e43b9cb36278d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:07 UTC) from {username='slave'} at 127.0.0.1:37876
W20260925 09:57:12.505276 10389 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 26fae3fbfe374f348a1729c21ec751b9 has no consensus state
I20260925 09:57:12.511561 10984 raft_consensus.cc:2454] T b7646a1cde1b45b9bf7928c3e13bcbd3 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:12.475950 11085 consensus_queue.cc:260] T 2d661d5000894c45ac3d1c27f0e72889 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:12.514176 10658 tablet_service.cc:1514] Processing DeleteTablet for tablet 26b7c37959d042b48234acc79290936f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:08 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:12.514348 10619 leader_election.cc:302] T b7646a1cde1b45b9bf7928c3e13bcbd3 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:12.515242 10387 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet b7646a1cde1b45b9bf7928c3e13bcbd3
I20260925 09:57:12.475015 10834 raft_consensus.cc:2454] T 8c13a2f288b14d03a3ca4fdd4889cbad P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:12.518106 10658 tablet_service.cc:1514] Processing DeleteTablet for tablet b7646a1cde1b45b9bf7928c3e13bcbd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:12 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:12.518802 10963 tablet_service.cc:1514] Processing DeleteTablet for tablet b7646a1cde1b45b9bf7928c3e13bcbd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:12 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:12.519454 11245 tablet_replica.cc:330] T b7646a1cde1b45b9bf7928c3e13bcbd3 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:12.520216 11245 raft_consensus.cc:2229] T b7646a1cde1b45b9bf7928c3e13bcbd3 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:12.520661 11245 raft_consensus.cc:2258] T b7646a1cde1b45b9bf7928c3e13bcbd3 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:12.521292 10963 tablet_service.cc:1514] Processing DeleteTablet for tablet e48d86c20e9e4db990c23359ff1d1a6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:11 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:12.522318 11245 ts_tablet_manager.cc:1905] T b7646a1cde1b45b9bf7928c3e13bcbd3 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:12.523581 10962 tablet_service.cc:1514] Processing DeleteTablet for tablet aad3cb5e33f64cb885d51f4ea089307b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:11 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:12.523844 10963 tablet_service.cc:1514] Processing DeleteTablet for tablet 26b7c37959d042b48234acc79290936f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:08 UTC) from {username='slave'} at 127.0.0.1:53560
W20260925 09:57:12.524375 10390 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.3:41469 (request call id 13) took 993 ms (client timeout 1000 ms). Trace:
I20260925 09:57:12.524507 10963 tablet_service.cc:1514] Processing DeleteTablet for tablet d8be238061c146eb96d17a22c1b2104a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:08 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:12.525573 10963 tablet_service.cc:1514] Processing DeleteTablet for tablet 840faf042d8749bd958ce1de0207a734 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:11 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:12.526168 10962 tablet_service.cc:1514] Processing DeleteTablet for tablet d48d4517c13049389c5b2ba1ab7b14bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:07 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:12.526187 10963 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b03e5aad59e46a6b0e43b9cb36278d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:07 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:12.528225 10817 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b03e5aad59e46a6b0e43b9cb36278d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:07 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:12.528903 11292 raft_consensus.cc:2258] T d48d4517c13049389c5b2ba1ab7b14bc P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:12.536909 10658 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c13a2f288b14d03a3ca4fdd4889cbad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:09 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:12.539913 11245 ts_tablet_manager.cc:1918] T b7646a1cde1b45b9bf7928c3e13bcbd3 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:12.540457 11245 log.cc:1196] T b7646a1cde1b45b9bf7928c3e13bcbd3 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/b7646a1cde1b45b9bf7928c3e13bcbd3
I20260925 09:57:12.541142 11245 ts_tablet_manager.cc:1939] T b7646a1cde1b45b9bf7928c3e13bcbd3 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:12.541213 11084 tablet_bootstrap.cc:492] T 28d60531970a4ab58ce5ef99cb3a96bf P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:12.548110 11084 ts_tablet_manager.cc:1397] T 28d60531970a4ab58ce5ef99cb3a96bf P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.223s user 0.011s sys 0.000s
I20260925 09:57:12.528347 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet d8be238061c146eb96d17a22c1b2104a (table default.Table_2bc55a20487a41b182135205d1a4b65d [id=f97ee2a971ab46abb4cbb37169fb309d]) successfully deleted
W20260925 09:57:12.549664 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 931769e78f32467b8f27bacb286a7a60 because the tablet was not found. No further retry: Not found: Tablet not found: 931769e78f32467b8f27bacb286a7a60
I20260925 09:57:12.531576 10393 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_650d171329b0468fbdf510ae8e4ce99f" } new_table_name: "default.Table_46e4c497898b454c85b7c2a299ebf3d9" modify_external_catalogs: true
I20260925 09:57:12.551488 11211 leader_election.cc:288] T b7646a1cde1b45b9bf7928c3e13bcbd3 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:12.552678 10814 tablet_service.cc:1514] Processing DeleteTablet for tablet 26b7c37959d042b48234acc79290936f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:08 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:12.553422 10817 tablet_service.cc:1514] Processing DeleteTablet for tablet b7646a1cde1b45b9bf7928c3e13bcbd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:12 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:12.554426 10837 raft_consensus.cc:2454] T 26b7c37959d042b48234acc79290936f P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
W20260925 09:57:12.555768 10621 consensus_peers.cc:487] T d48d4517c13049389c5b2ba1ab7b14bc P b8850818e2ee42a082802d0095c08d21 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
W20260925 09:57:12.556349 10621 consensus_peers.cc:487] T d8be238061c146eb96d17a22c1b2104a P b8850818e2ee42a082802d0095c08d21 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d8be238061c146eb96d17a22c1b2104a. This is attempt 1: this message will repeat every 5th retry.
W20260925 09:57:12.560204 10619 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 45959.8us
I20260925 09:57:12.560479 11320 raft_consensus.cc:2781] T b7646a1cde1b45b9bf7928c3e13bcbd3 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:12.560925 11320 raft_consensus.cc:686] T b7646a1cde1b45b9bf7928c3e13bcbd3 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:12.560828 10386 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_7eaf247fb74f497d80a88b67fa301cab" } modify_external_catalogs: true
I20260925 09:57:12.561484 10386 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_7eaf247fb74f497d80a88b67fa301cab" } modify_external_catalogs: true
I20260925 09:57:12.561731 11320 consensus_queue.cc:237] T b7646a1cde1b45b9bf7928c3e13bcbd3 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:12.564124 10387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_fea25102a7f84d58927645a0feae51e3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:12.565905 10387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fea25102a7f84d58927645a0feae51e3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:12.568212 11082 ts_tablet_manager.cc:1428] T 2d661d5000894c45ac3d1c27f0e72889 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.086s user 0.005s sys 0.001s
W20260925 09:57:12.524906 10390 rpcz_store.cc:269] 0925 09:57:11.530610 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:11.531714 (+ 1104us) service_pool.cc:227] Handling call
0925 09:57:12.524310 (+992596us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:11.953257 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:11.953717 (+ 460us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:11.953739 (+ 22us) write_op.cc:435] Acquired schema lock
0925 09:57:11.953822 (+ 83us) tablet.cc:660] Decoding operations
0925 09:57:11.954055 (+ 233us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:11.954088 (+ 33us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:11.954111 (+ 23us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:11.954309 (+ 198us) tablet.cc:699] Row locks acquired
0925 09:57:11.954324 (+ 15us) write_op.cc:260] PREPARE: finished
0925 09:57:11.954420 (+ 96us) write_op.cc:270] Start()
0925 09:57:11.954493 (+ 73us) write_op.cc:276] Timestamp: P: 1790330231954349 usec, L: 0
0925 09:57:11.954509 (+ 16us) op_driver.cc:331] REPLICATION: starting
0925 09:57:11.965835 (+ 11326us) spinlock_profiling.cc:243] Waited 9.93 ms on lock 0x7b5400012320. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90471 00007fabf0aa0dc0 00007fabf0b11bd2 00007fabf0b072d7 00007fabf0b07114 00007fabf0f68ade 00007fabf0f682b8 00007fabf0f6cba1 00007fabf0f6cb59 00007fabf0f6cae9 00007fabf0f6cab1
0925 09:57:11.966289 (+ 454us) log.cc:842] Serialized 356 byte log entry
0925 09:57:11.971351 (+ 5062us) spinlock_profiling.cc:243] Waited 3.65 ms on lock 0x7b58000b03b0. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0a85834 00007fabf0a8a771 00007fabf0a8a6f9 00007fabf0a8a689 00007fabf0a8a651 00007fabf0a8981d 0000000000580784 00000000005805b9 00007fabe9bb66b6 00007fabe9bb9b11
0925 09:57:12.087981 (+116630us) op_driver.cc:444] REPLICATION: finished
0925 09:57:12.092580 (+ 4599us) write_op.cc:301] APPLY: starting
0925 09:57:12.092660 (+ 80us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:12.092778 (+ 118us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:12.092811 (+ 33us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:12.093010 (+ 199us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:12.093194 (+ 184us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:12.093256 (+ 62us) write_op.cc:312] APPLY: finished
0925 09:57:12.093486 (+ 230us) log.cc:842] Serialized 23 byte log entry
0925 09:57:12.093670 (+ 184us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:12.093745 (+ 75us) write_op.cc:454] Released schema lock
0925 09:57:12.093786 (+ 41us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:12.093999 (+ 213us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:12.094033 (+ 34us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":27,"child_traces":[["op",{"apply.queue_time_us":4388,"apply.run_cpu_time_us":1660,"apply.run_wall_time_us":1661,"num_ops":1,"prepare.queue_time_us":20672,"prepare.run_cpu_time_us":3371,"prepare.run_wall_time_us":13262,"raft.queue_time_us":826,"raft.run_cpu_time_us":573,"raft.run_wall_time_us":4115,"replication_time_us":133372,"spinlock_wait_cycles":31225856}]]}
I20260925 09:57:12.546926 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet b7646a1cde1b45b9bf7928c3e13bcbd3 (table default.Table_9f64de2cbc4c4968bc0cfe51d0a9a487 [id=11e4b3a3251f4fa4a78de075f8ecb6f2]) successfully deleted
W20260925 09:57:12.570228 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet e48d86c20e9e4db990c23359ff1d1a6d because the tablet was not found. No further retry: Not found: Tablet not found: e48d86c20e9e4db990c23359ff1d1a6d
W20260925 09:57:12.570919 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet aad3cb5e33f64cb885d51f4ea089307b because the tablet was not found. No further retry: Not found: Tablet not found: aad3cb5e33f64cb885d51f4ea089307b
I20260925 09:57:12.550917 11084 raft_consensus.cc:348] T 28d60531970a4ab58ce5ef99cb3a96bf P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:12.571777 11084 raft_consensus.cc:374] T 28d60531970a4ab58ce5ef99cb3a96bf P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:12.572049 11084 raft_consensus.cc:729] T 28d60531970a4ab58ce5ef99cb3a96bf P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:12.572676 11084 consensus_queue.cc:260] T 28d60531970a4ab58ce5ef99cb3a96bf P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:12.572369 11082 tablet_bootstrap.cc:492] T e48d86c20e9e4db990c23359ff1d1a6d P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:12.574786 11084 ts_tablet_manager.cc:1428] T 28d60531970a4ab58ce5ef99cb3a96bf P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.026s user 0.003s sys 0.000s
W20260925 09:57:12.571580 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 26b7c37959d042b48234acc79290936f because the tablet was not found. No further retry: Not found: Tablet not found: 26b7c37959d042b48234acc79290936f
W20260925 09:57:12.578600 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet d8be238061c146eb96d17a22c1b2104a because the tablet was not found. No further retry: Not found: Tablet not found: d8be238061c146eb96d17a22c1b2104a
I20260925 09:57:12.579046 11085 ts_tablet_manager.cc:1428] T 2d661d5000894c45ac3d1c27f0e72889 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.273s user 0.008s sys 0.002s
W20260925 09:57:12.579236 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 840faf042d8749bd958ce1de0207a734 because the tablet was not found. No further retry: Not found: Tablet not found: 840faf042d8749bd958ce1de0207a734
I20260925 09:57:12.579948 11085 tablet_bootstrap.cc:492] T e48d86c20e9e4db990c23359ff1d1a6d P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:12.581192 11292 ts_tablet_manager.cc:1905] T d48d4517c13049389c5b2ba1ab7b14bc P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:12.581638 11211 consensus_queue.cc:1035] T aad3cb5e33f64cb885d51f4ea089307b P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 09:57:12.583003 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet d48d4517c13049389c5b2ba1ab7b14bc because the tablet was not found. No further retry: Not found: Tablet not found: d48d4517c13049389c5b2ba1ab7b14bc
W20260925 09:57:12.584249 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 6b03e5aad59e46a6b0e43b9cb36278d7 because the tablet was not found. No further retry: Not found: Tablet not found: 6b03e5aad59e46a6b0e43b9cb36278d7
I20260925 09:57:12.584611 10819 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c13a2f288b14d03a3ca4fdd4889cbad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:09 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:12.586081 11085 tablet_bootstrap.cc:654] T e48d86c20e9e4db990c23359ff1d1a6d P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:12.587595 10834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7646a1cde1b45b9bf7928c3e13bcbd3" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:12.587733 10390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_7e0155a70cc54485b8fdd4d936460d1c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:12.591567 10390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7e0155a70cc54485b8fdd4d936460d1c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:12.595798 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7646a1cde1b45b9bf7928c3e13bcbd3" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:12.607630 10621 leader_election.cc:302] T 8c13a2f288b14d03a3ca4fdd4889cbad P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters: be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:12.608004 11084 tablet_bootstrap.cc:492] T c16e096038084b2b9148390b6e773294 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:12.608827 11211 raft_consensus.cc:2781] T 8c13a2f288b14d03a3ca4fdd4889cbad P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:12.609143 11211 raft_consensus.cc:482] T 8c13a2f288b14d03a3ca4fdd4889cbad P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:12.609432 11211 raft_consensus.cc:3037] T 8c13a2f288b14d03a3ca4fdd4889cbad P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:12.613723 11084 tablet_bootstrap.cc:654] T c16e096038084b2b9148390b6e773294 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:12.614687 10621 leader_election.cc:341] T b7646a1cde1b45b9bf7928c3e13bcbd3 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:12.620036 10621 leader_election.cc:341] T b7646a1cde1b45b9bf7928c3e13bcbd3 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:12.622383 11211 raft_consensus.cc:504] T 8c13a2f288b14d03a3ca4fdd4889cbad P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:12.625489 11211 leader_election.cc:288] T 8c13a2f288b14d03a3ca4fdd4889cbad P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
W20260925 09:57:12.626441 10619 leader_election.cc:341] T 8c13a2f288b14d03a3ca4fdd4889cbad P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 8c13a2f288b14d03a3ca4fdd4889cbad
W20260925 09:57:12.634078 10393 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ef6fefaa76d44fe798201aab2f21b9c7 has no consensus state
I20260925 09:57:12.650415 11298 ts_tablet_manager.cc:1918] T 840faf042d8749bd958ce1de0207a734 P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260925 09:57:12.650903 11298 log.cc:1196] T 840faf042d8749bd958ce1de0207a734 P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/840faf042d8749bd958ce1de0207a734
W20260925 09:57:12.650835 10619 consensus_peers.cc:487] T 26b7c37959d042b48234acc79290936f P b8850818e2ee42a082802d0095c08d21 -> Peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Couldn't send request to peer be97ab5d75934a648d4ab76341193ec0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 26b7c37959d042b48234acc79290936f. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:12.652011 11298 ts_tablet_manager.cc:1939] T 840faf042d8749bd958ce1de0207a734 P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
W20260925 09:57:12.659016 10379 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet ef6fefaa76d44fe798201aab2f21b9c7 (table default.Table_46e4c497898b454c85b7c2a299ebf3d9 [id=80b48dd413e3446eb7ac65e5f957b5dc], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet ef6fefaa76d44fe798201aab2f21b9c7 has no consensus state
I20260925 09:57:12.667219 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c13a2f288b14d03a3ca4fdd4889cbad" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:12.667806 10836 raft_consensus.cc:3037] T 8c13a2f288b14d03a3ca4fdd4889cbad P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:12.671026 11082 tablet_bootstrap.cc:654] T e48d86c20e9e4db990c23359ff1d1a6d P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:12.673676 11298 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 65) took 1349 ms. Trace:
I20260925 09:57:12.674135 11298 rpcz_store.cc:276] 0925 09:57:11.323929 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:11.326088 (+ 2159us) service_pool.cc:227] Handling call
0925 09:57:12.206167 (+880079us) ts_tablet_manager.cc:1145] Deleting tablet 840faf042d8749bd958ce1de0207a734
0925 09:57:12.206225 (+ 58us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:12.444362 (+238137us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:12.650164 (+205802us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:12.673611 (+ 23447us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":879358}
I20260925 09:57:12.675925 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 840faf042d8749bd958ce1de0207a734 (table default.Table_faa21794a80c407f800038e583c08f16 [id=800eba46046443a7ad196808dec4cdc0]) successfully deleted
I20260925 09:57:12.678762 10836 raft_consensus.cc:2454] T 8c13a2f288b14d03a3ca4fdd4889cbad P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:12.678819 10389 catalog_manager.cc:5495] T 8b281fc20e6f42239fee80d05da88ab6 P c43461c4ee854ae1ac52b873ccada318 reported cstate change: term changed from 0 to 1, leader changed from <none> to c43461c4ee854ae1ac52b873ccada318 (127.10.9.2). New cstate: current_term: 1 leader_uuid: "c43461c4ee854ae1ac52b873ccada318" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:12.679842 11298 tablet_replica.cc:330] T aad3cb5e33f64cb885d51f4ea089307b P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:12.680400 10621 leader_election.cc:302] T 8c13a2f288b14d03a3ca4fdd4889cbad P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters: be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:12.681334 11320 raft_consensus.cc:2781] T 8c13a2f288b14d03a3ca4fdd4889cbad P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:12.681740 11320 raft_consensus.cc:686] T 8c13a2f288b14d03a3ca4fdd4889cbad P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:12.682463 11320 consensus_queue.cc:237] T 8c13a2f288b14d03a3ca4fdd4889cbad P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:12.689445 10386 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 2d55154ceb064f22bd6f37c8ae5dce74
I20260925 09:57:12.691617 10658 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d55154ceb064f22bd6f37c8ae5dce74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:12 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:12.694427 10963 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d55154ceb064f22bd6f37c8ae5dce74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:12 UTC) from {username='slave'} at 127.0.0.1:53560
W20260925 09:57:12.696347 10377 catalog_manager.cc:4821] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 2d55154ceb064f22bd6f37c8ae5dce74 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2d55154ceb064f22bd6f37c8ae5dce74 already in progress: creating tablet
I20260925 09:57:12.705928 11195 consensus_queue.cc:1035] T 0a2bbe141e564a96be05de86f2993234 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:12.707515 11249 raft_consensus.cc:482] T 2d661d5000894c45ac3d1c27f0e72889 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:12.707943 11249 raft_consensus.cc:504] T 2d661d5000894c45ac3d1c27f0e72889 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:12.710377 11249 leader_election.cc:288] T 2d661d5000894c45ac3d1c27f0e72889 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:12.710274 10386 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_861e5470a3b548419a9d96e59a1f9b0e" } new_table_name: "default.Table_5a6ecccec7f44c31bbfe8d64580ec33b" modify_external_catalogs: true
I20260925 09:57:12.712186 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d661d5000894c45ac3d1c27f0e72889" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:12.712745 10686 raft_consensus.cc:2454] T 2d661d5000894c45ac3d1c27f0e72889 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
I20260925 09:57:12.712610 11362 tablet.cc:1722] T 1969ac471b6f471ebe8e048a96c22837 P c43461c4ee854ae1ac52b873ccada318: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260925 09:57:12.713886 10775 leader_election.cc:302] T 2d661d5000894c45ac3d1c27f0e72889 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:12.714226 10390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_8cde0139b88d439eaee476479a937bf3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:12.714938 11361 raft_consensus.cc:2781] T 2d661d5000894c45ac3d1c27f0e72889 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:12.715261 11361 raft_consensus.cc:482] T 2d661d5000894c45ac3d1c27f0e72889 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:12.715595 11361 raft_consensus.cc:3037] T 2d661d5000894c45ac3d1c27f0e72889 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:57:12.715857 10390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8cde0139b88d439eaee476479a937bf3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:12.716269 10983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d661d5000894c45ac3d1c27f0e72889" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:12.716957 10983 raft_consensus.cc:2454] T 2d661d5000894c45ac3d1c27f0e72889 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
I20260925 09:57:12.720919 10819 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d55154ceb064f22bd6f37c8ae5dce74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:12 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:12.724378 10387 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_c6db4ef50ad442fcb28e2646fd6b7571" } modify_external_catalogs: true
I20260925 09:57:12.725037 10387 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_c6db4ef50ad442fcb28e2646fd6b7571" } modify_external_catalogs: true
I20260925 09:57:12.727977 10393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_a36cbf13cf074bf29cd3fee057b97bb7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:12.729524 10393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a36cbf13cf074bf29cd3fee057b97bb7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:12.733716 11084 tablet_bootstrap.cc:492] T c16e096038084b2b9148390b6e773294 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:12.734221 11084 ts_tablet_manager.cc:1397] T c16e096038084b2b9148390b6e773294 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.126s user 0.009s sys 0.000s
I20260925 09:57:12.737351 11084 raft_consensus.cc:348] T c16e096038084b2b9148390b6e773294 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:12.738165 11084 raft_consensus.cc:374] T c16e096038084b2b9148390b6e773294 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:12.738533 11084 raft_consensus.cc:729] T c16e096038084b2b9148390b6e773294 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:12.756131 11361 raft_consensus.cc:504] T 2d661d5000894c45ac3d1c27f0e72889 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:12.766676 10384 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 0a2bbe141e564a96be05de86f2993234 (table default.Table_8705f75ef7b2448ab645f1d25e3c2637 [id=aed8a2ddc51545ad9cec4d272dcf3daf]). Expected version 1 got 0
I20260925 09:57:12.782866 10963 tablet_service.cc:1467] Processing CreateTablet for tablet 752105c1514f4c1db2a11b95b413b4f6 (DEFAULT_TABLE table=default.Table_8c5b820f0db34e3bbea2d404afb1909a [id=fc979ab5541c41ef82a95e11f5a455ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:12.783902 10963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 752105c1514f4c1db2a11b95b413b4f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:12.739447 11084 consensus_queue.cc:260] T c16e096038084b2b9148390b6e773294 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:12.796610 11084 ts_tablet_manager.cc:1428] T c16e096038084b2b9148390b6e773294 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.062s user 0.008s sys 0.000s
I20260925 09:57:12.798616 11084 tablet_bootstrap.cc:492] T 3be0eccc65564bfba3d6e5c398c61fb6 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:12.820267 11298 raft_consensus.cc:2229] T aad3cb5e33f64cb885d51f4ea089307b P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:12.821185 11298 raft_consensus.cc:2258] T aad3cb5e33f64cb885d51f4ea089307b P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:12.838299 10393 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_c3a609c3cdbf4990910b8784a7412d34" } new_table_name: "default.Table_ce09d3a8d8e84973894f33b18bbfdb22" modify_external_catalogs: true
I20260925 09:57:12.840322 11298 ts_tablet_manager.cc:1905] T aad3cb5e33f64cb885d51f4ea089307b P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:12.840318 10390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_bbc3c50c06634d119ae96dcf649f44e7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:12.840528 10385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_5641cc23eb8b41959d9656f65afb9fe8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:12.841758 10390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bbc3c50c06634d119ae96dcf649f44e7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 09:57:12.841766 10385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5641cc23eb8b41959d9656f65afb9fe8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 09:57:12.847352 10621 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 106877us
I20260925 09:57:12.848067 10661 tablet_service.cc:1467] Processing CreateTablet for tablet 41df75fe4b8e4856bdbb5ead3cf4fd24 (DEFAULT_TABLE table=default.Table_0507f47042b9445abaf8671c979f8994 [id=55860c5dba9f4c818cfa6d523041ec5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:12.849118 10661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41df75fe4b8e4856bdbb5ead3cf4fd24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:12.847033 10658 tablet_service.cc:1467] Processing CreateTablet for tablet 752105c1514f4c1db2a11b95b413b4f6 (DEFAULT_TABLE table=default.Table_8c5b820f0db34e3bbea2d404afb1909a [id=fc979ab5541c41ef82a95e11f5a455ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:12.852121 10658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 752105c1514f4c1db2a11b95b413b4f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:12.853012 11084 tablet_bootstrap.cc:654] T 3be0eccc65564bfba3d6e5c398c61fb6 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:12.852115 11368 tablet.cc:1722] T 1969ac471b6f471ebe8e048a96c22837 P b8850818e2ee42a082802d0095c08d21: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
W20260925 09:57:12.855317 10619 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 79808.4us
I20260925 09:57:12.860944 11082 tablet_bootstrap.cc:492] T e48d86c20e9e4db990c23359ff1d1a6d P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:12.861334 11082 ts_tablet_manager.cc:1397] T e48d86c20e9e4db990c23359ff1d1a6d P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.289s user 0.006s sys 0.005s
I20260925 09:57:12.878551 11082 raft_consensus.cc:348] T e48d86c20e9e4db990c23359ff1d1a6d P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:12.885211 11082 raft_consensus.cc:374] T e48d86c20e9e4db990c23359ff1d1a6d P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:12.885556 11082 raft_consensus.cc:729] T e48d86c20e9e4db990c23359ff1d1a6d P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:12.890321 11082 consensus_queue.cc:260] T e48d86c20e9e4db990c23359ff1d1a6d P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:12.892462 10388 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 0a2bbe141e564a96be05de86f2993234 (table default.Table_8705f75ef7b2448ab645f1d25e3c2637 [id=aed8a2ddc51545ad9cec4d272dcf3daf]). Expected version 1 got 0
I20260925 09:57:12.893880 10388 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 91c09118b43b41eca8f7040323ba76f7 (table default.Table_33b40a0d6ca44f15be47ca34bab6a093 [id=453beec106a64428b163c7db8132fc21]). Expected version 1 got 0
I20260925 09:57:12.897333 10391 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_a36cbf13cf074bf29cd3fee057b97bb7" } modify_external_catalogs: true
I20260925 09:57:12.897604 11084 tablet_bootstrap.cc:492] T 3be0eccc65564bfba3d6e5c398c61fb6 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:12.897981 11084 ts_tablet_manager.cc:1397] T 3be0eccc65564bfba3d6e5c398c61fb6 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.100s user 0.008s sys 0.000s
I20260925 09:57:12.897891 10391 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_a36cbf13cf074bf29cd3fee057b97bb7" } modify_external_catalogs: true
W20260925 09:57:12.898044 10379 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.62:34455, user_credentials={real_user=slave}} blocked reactor thread for 62296.9us
I20260925 09:57:12.901191 11082 ts_tablet_manager.cc:1428] T e48d86c20e9e4db990c23359ff1d1a6d P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.040s user 0.005s sys 0.001s
I20260925 09:57:12.901242 10962 tablet_service.cc:1467] Processing CreateTablet for tablet 41df75fe4b8e4856bdbb5ead3cf4fd24 (DEFAULT_TABLE table=default.Table_0507f47042b9445abaf8671c979f8994 [id=55860c5dba9f4c818cfa6d523041ec5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:12.901804 11084 raft_consensus.cc:348] T 3be0eccc65564bfba3d6e5c398c61fb6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:12.902369 10962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41df75fe4b8e4856bdbb5ead3cf4fd24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:12.902426 11084 raft_consensus.cc:374] T 3be0eccc65564bfba3d6e5c398c61fb6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:12.902721 11084 raft_consensus.cc:729] T 3be0eccc65564bfba3d6e5c398c61fb6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:12.907053 11082 tablet_bootstrap.cc:492] T 0e882284a014455192375daad391efaf P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:12.912400 11084 consensus_queue.cc:260] T 3be0eccc65564bfba3d6e5c398c61fb6 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:12.764456 11085 tablet_bootstrap.cc:492] T e48d86c20e9e4db990c23359ff1d1a6d P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:12.913533 11085 ts_tablet_manager.cc:1397] T e48d86c20e9e4db990c23359ff1d1a6d P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.334s user 0.012s sys 0.004s
I20260925 09:57:12.915522 11085 raft_consensus.cc:348] T e48d86c20e9e4db990c23359ff1d1a6d P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:12.916115 11085 raft_consensus.cc:374] T e48d86c20e9e4db990c23359ff1d1a6d P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:12.916339 11085 raft_consensus.cc:729] T e48d86c20e9e4db990c23359ff1d1a6d P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:12.916178 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d661d5000894c45ac3d1c27f0e72889" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:12.785913 11292 ts_tablet_manager.cc:1918] T d48d4517c13049389c5b2ba1ab7b14bc P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:12.916738 10984 raft_consensus.cc:3037] T 2d661d5000894c45ac3d1c27f0e72889 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:12.916910 11292 log.cc:1196] T d48d4517c13049389c5b2ba1ab7b14bc P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/d48d4517c13049389c5b2ba1ab7b14bc
I20260925 09:57:12.916711 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d661d5000894c45ac3d1c27f0e72889" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:12.917277 10686 raft_consensus.cc:3037] T 2d661d5000894c45ac3d1c27f0e72889 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:12.917552 11292 ts_tablet_manager.cc:1939] T d48d4517c13049389c5b2ba1ab7b14bc P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:12.921167 11292 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 73) took 1477 ms. Trace:
I20260925 09:57:12.921489 11292 rpcz_store.cc:276] 0925 09:57:11.442270 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:11.451065 (+ 8795us) service_pool.cc:227] Handling call
0925 09:57:12.443323 (+992258us) ts_tablet_manager.cc:1145] Deleting tablet d48d4517c13049389c5b2ba1ab7b14bc
0925 09:57:12.443366 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:12.698751 (+255385us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:12.785653 (+ 86902us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:12.919344 (+133691us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":41089,"tablet-delete.queue_time_us":991760}
I20260925 09:57:12.922668 11292 tablet_replica.cc:330] T aad3cb5e33f64cb885d51f4ea089307b P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:12.923491 11085 consensus_queue.cc:260] T e48d86c20e9e4db990c23359ff1d1a6d P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:12.924849 10686 raft_consensus.cc:2454] T 2d661d5000894c45ac3d1c27f0e72889 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:12.926191 10775 leader_election.cc:302] T 2d661d5000894c45ac3d1c27f0e72889 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:12.926944 11345 raft_consensus.cc:2781] T 2d661d5000894c45ac3d1c27f0e72889 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:12.927309 11345 raft_consensus.cc:686] T 2d661d5000894c45ac3d1c27f0e72889 P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Becoming Leader. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Running, Role: LEADER
I20260925 09:57:12.928284 11345 consensus_queue.cc:237] T 2d661d5000894c45ac3d1c27f0e72889 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:12.930729 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet d48d4517c13049389c5b2ba1ab7b14bc (table default.Table_699fb6630e69435d946304ff8473d1e8 [id=1b1ce9c0cada4beeae7e02eb3105a564]) successfully deleted
I20260925 09:57:12.931237 11292 raft_consensus.cc:2229] T aad3cb5e33f64cb885d51f4ea089307b P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:12.931775 11084 ts_tablet_manager.cc:1428] T 3be0eccc65564bfba3d6e5c398c61fb6 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.034s user 0.006s sys 0.001s
I20260925 09:57:12.926178 11085 ts_tablet_manager.cc:1428] T e48d86c20e9e4db990c23359ff1d1a6d P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.012s user 0.004s sys 0.000s
I20260925 09:57:12.932685 11085 tablet_bootstrap.cc:492] T b7646a1cde1b45b9bf7928c3e13bcbd3 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:12.931895 11292 pending_rounds.cc:62] T aad3cb5e33f64cb885d51f4ea089307b P c43461c4ee854ae1ac52b873ccada318: Trying to abort 1 pending ops.
I20260925 09:57:12.933974 11292 pending_rounds.cc:68] T aad3cb5e33f64cb885d51f4ea089307b P c43461c4ee854ae1ac52b873ccada318: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192627076841472 op_type: NO_OP noop_request { }
I20260925 09:57:12.934523 11292 raft_consensus.cc:2866] T aad3cb5e33f64cb885d51f4ea089307b P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:57:12.935827 10984 raft_consensus.cc:2454] T 2d661d5000894c45ac3d1c27f0e72889 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:12.940464 10666 tablet_service.cc:1467] Processing CreateTablet for tablet 89a143a5623e4ef38027381a56c95227 (DEFAULT_TABLE table=default.Table_bb742c618d96453aaabe897d54fc657b [id=68683893b91c4f0190a0050ecdc6d396]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:12.941469 10666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89a143a5623e4ef38027381a56c95227. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:12.942233 11368 tablet.cc:1722] T 0a2bbe141e564a96be05de86f2993234 P b8850818e2ee42a082802d0095c08d21: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:12.943590 11084 tablet_bootstrap.cc:492] T 2d55154ceb064f22bd6f37c8ae5dce74 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:12.947975 11292 raft_consensus.cc:2258] T aad3cb5e33f64cb885d51f4ea089307b P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:12.950340 11292 ts_tablet_manager.cc:1905] T aad3cb5e33f64cb885d51f4ea089307b P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:12.953960 11386 tablet.cc:1722] T 1969ac471b6f471ebe8e048a96c22837 P be97ab5d75934a648d4ab76341193ec0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260925 09:57:12.960131 11361 leader_election.cc:288] T 2d661d5000894c45ac3d1c27f0e72889 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:12.969355 11085 tablet_bootstrap.cc:654] T b7646a1cde1b45b9bf7928c3e13bcbd3 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:12.977645 11082 tablet_bootstrap.cc:654] T 0e882284a014455192375daad391efaf P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:12.991003 11298 ts_tablet_manager.cc:1918] T aad3cb5e33f64cb885d51f4ea089307b P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:12.991415 11298 log.cc:1196] T aad3cb5e33f64cb885d51f4ea089307b P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/aad3cb5e33f64cb885d51f4ea089307b
I20260925 09:57:12.992200 11298 ts_tablet_manager.cc:1939] T aad3cb5e33f64cb885d51f4ea089307b P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:12.994536 11298 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 66) took 1470 ms. Trace:
I20260925 09:57:12.967696 10958 tablet_service.cc:1467] Processing CreateTablet for tablet 89a143a5623e4ef38027381a56c95227 (DEFAULT_TABLE table=default.Table_bb742c618d96453aaabe897d54fc657b [id=68683893b91c4f0190a0050ecdc6d396]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:12.968133 11084 tablet_bootstrap.cc:654] T 2d55154ceb064f22bd6f37c8ae5dce74 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:13.004172 10958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89a143a5623e4ef38027381a56c95227. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:12.994889 11298 rpcz_store.cc:276] 0925 09:57:11.524282 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:11.534959 (+ 10677us) service_pool.cc:227] Handling call
0925 09:57:12.679682 (+1144723us) ts_tablet_manager.cc:1145] Deleting tablet aad3cb5e33f64cb885d51f4ea089307b
0925 09:57:12.679729 (+ 47us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:12.894862 (+215133us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:12.990767 (+ 95905us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:12.994477 (+ 3710us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1127963}
I20260925 09:57:13.012693 11085 tablet_bootstrap.cc:492] T b7646a1cde1b45b9bf7928c3e13bcbd3 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:13.013007 11298 tablet_replica.cc:330] T d48d4517c13049389c5b2ba1ab7b14bc P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:13.013074 11085 ts_tablet_manager.cc:1397] T b7646a1cde1b45b9bf7928c3e13bcbd3 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.081s user 0.009s sys 0.000s
I20260925 09:57:13.013262 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet aad3cb5e33f64cb885d51f4ea089307b (table default.Table_2d7acf0d8022443090ba12cd87956502 [id=9d2f4f9f46394332bec9f8beda0d9595]) successfully deleted
I20260925 09:57:13.013809 11298 raft_consensus.cc:2229] T d48d4517c13049389c5b2ba1ab7b14bc P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:13.014598 11298 pending_rounds.cc:62] T d48d4517c13049389c5b2ba1ab7b14bc P b8850818e2ee42a082802d0095c08d21: Trying to abort 1 pending ops.
I20260925 09:57:13.014844 11298 pending_rounds.cc:68] T d48d4517c13049389c5b2ba1ab7b14bc P b8850818e2ee42a082802d0095c08d21: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192626059612160 op_type: NO_OP noop_request { }
I20260925 09:57:13.015046 11085 raft_consensus.cc:348] T b7646a1cde1b45b9bf7928c3e13bcbd3 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:13.015754 11085 raft_consensus.cc:374] T b7646a1cde1b45b9bf7928c3e13bcbd3 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:13.016008 11085 raft_consensus.cc:729] T b7646a1cde1b45b9bf7928c3e13bcbd3 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:13.019079 11085 consensus_queue.cc:260] T b7646a1cde1b45b9bf7928c3e13bcbd3 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:13.021785 11085 ts_tablet_manager.cc:1428] T b7646a1cde1b45b9bf7928c3e13bcbd3 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260925 09:57:13.015211 11298 raft_consensus.cc:2866] T d48d4517c13049389c5b2ba1ab7b14bc P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:57:13.023821 11298 raft_consensus.cc:2258] T d48d4517c13049389c5b2ba1ab7b14bc P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:13.025992 11298 ts_tablet_manager.cc:1905] T d48d4517c13049389c5b2ba1ab7b14bc P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:13.029131 11082 tablet_bootstrap.cc:492] T 0e882284a014455192375daad391efaf P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:13.029526 11082 ts_tablet_manager.cc:1397] T 0e882284a014455192375daad391efaf P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.123s user 0.013s sys 0.001s
I20260925 09:57:13.029831 11292 ts_tablet_manager.cc:1918] T aad3cb5e33f64cb885d51f4ea089307b P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:13.030190 11292 log.cc:1196] T aad3cb5e33f64cb885d51f4ea089307b P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/aad3cb5e33f64cb885d51f4ea089307b
I20260925 09:57:13.032114 11082 raft_consensus.cc:348] T 0e882284a014455192375daad391efaf P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:13.032763 11082 raft_consensus.cc:374] T 0e882284a014455192375daad391efaf P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:13.032840 11084 tablet_bootstrap.cc:492] T 2d55154ceb064f22bd6f37c8ae5dce74 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:13.033027 11082 raft_consensus.cc:729] T 0e882284a014455192375daad391efaf P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:13.033209 11084 ts_tablet_manager.cc:1397] T 2d55154ceb064f22bd6f37c8ae5dce74 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.101s user 0.004s sys 0.006s
W20260925 09:57:13.034921 10379 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.62:34455, user_credentials={real_user=slave}} blocked reactor thread for 58870.6us
I20260925 09:57:13.035111 11292 ts_tablet_manager.cc:1939] T aad3cb5e33f64cb885d51f4ea089307b P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:13.039721 11082 consensus_queue.cc:260] T 0e882284a014455192375daad391efaf P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:13.047173 11082 ts_tablet_manager.cc:1428] T 0e882284a014455192375daad391efaf P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.017s user 0.005s sys 0.001s
I20260925 09:57:13.035274 11084 raft_consensus.cc:348] T 2d55154ceb064f22bd6f37c8ae5dce74 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:13.051961 11084 raft_consensus.cc:374] T 2d55154ceb064f22bd6f37c8ae5dce74 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:13.052202 11084 raft_consensus.cc:729] T 2d55154ceb064f22bd6f37c8ae5dce74 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:13.052853 11084 consensus_queue.cc:260] T 2d55154ceb064f22bd6f37c8ae5dce74 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:13.055052 11084 ts_tablet_manager.cc:1428] T 2d55154ceb064f22bd6f37c8ae5dce74 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.022s user 0.005s sys 0.001s
I20260925 09:57:13.059763 10961 tablet_service.cc:1467] Processing CreateTablet for tablet ef6fefaa76d44fe798201aab2f21b9c7 (DEFAULT_TABLE table=default.Table_46e4c497898b454c85b7c2a299ebf3d9 [id=80b48dd413e3446eb7ac65e5f957b5dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:13.060802 10961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef6fefaa76d44fe798201aab2f21b9c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:13.065016 10819 tablet_service.cc:1467] Processing CreateTablet for tablet 752105c1514f4c1db2a11b95b413b4f6 (DEFAULT_TABLE table=default.Table_8c5b820f0db34e3bbea2d404afb1909a [id=fc979ab5541c41ef82a95e11f5a455ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:13.066869 10817 tablet_service.cc:1467] Processing CreateTablet for tablet 41df75fe4b8e4856bdbb5ead3cf4fd24 (DEFAULT_TABLE table=default.Table_0507f47042b9445abaf8671c979f8994 [id=55860c5dba9f4c818cfa6d523041ec5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:13.066954 10815 tablet_service.cc:1467] Processing CreateTablet for tablet 89a143a5623e4ef38027381a56c95227 (DEFAULT_TABLE table=default.Table_bb742c618d96453aaabe897d54fc657b [id=68683893b91c4f0190a0050ecdc6d396]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:13.070461 10388 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 1969ac471b6f471ebe8e048a96c22837 (table default.Table_7104d0f2dac14fa0a59685700d80c110 [id=f17a11d223204599afb24e2e9d1d9d9d]). Expected version 4 got 3
I20260925 09:57:13.071831 10387 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 2d661d5000894c45ac3d1c27f0e72889
I20260925 09:57:13.074774 10953 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d661d5000894c45ac3d1c27f0e72889 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:12 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:13.054533 10659 tablet_service.cc:1467] Processing CreateTablet for tablet ef6fefaa76d44fe798201aab2f21b9c7 (DEFAULT_TABLE table=default.Table_46e4c497898b454c85b7c2a299ebf3d9 [id=80b48dd413e3446eb7ac65e5f957b5dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:13.075496 11245 tablet_replica.cc:330] T 2d661d5000894c45ac3d1c27f0e72889 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:13.076380 10659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef6fefaa76d44fe798201aab2f21b9c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:13.076329 11245 raft_consensus.cc:2229] T 2d661d5000894c45ac3d1c27f0e72889 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:13.076802 11245 raft_consensus.cc:2258] T 2d661d5000894c45ac3d1c27f0e72889 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:13.078408 11245 ts_tablet_manager.cc:1905] T 2d661d5000894c45ac3d1c27f0e72889 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:13.061013 10668 tablet_service.cc:1467] Processing CreateTablet for tablet 42ec275c905343c4be1b72dc2bd0f6e3 (DEFAULT_TABLE table=default.Table_c2ce8fea3dc743379c1912445278a487 [id=cf07ed8756904011ab3dac900ea6705a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:13.083243 10668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42ec275c905343c4be1b72dc2bd0f6e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:13.069388 11320 consensus_queue.cc:1035] T 0a2bbe141e564a96be05de86f2993234 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:13.090270 11082 tablet_bootstrap.cc:492] T 3fa527403d98425ea5cff9215d3a5e7f P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:13.090443 10664 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d661d5000894c45ac3d1c27f0e72889 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:12 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:13.095172 11082 tablet_bootstrap.cc:654] T 3fa527403d98425ea5cff9215d3a5e7f P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:13.098482 11345 consensus_queue.cc:1035] T 8b281fc20e6f42239fee80d05da88ab6 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:13.124771 11085 tablet_bootstrap.cc:492] T 0e882284a014455192375daad391efaf P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:13.127769 11298 ts_tablet_manager.cc:1918] T d48d4517c13049389c5b2ba1ab7b14bc P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:13.128156 11298 log.cc:1196] T d48d4517c13049389c5b2ba1ab7b14bc P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/d48d4517c13049389c5b2ba1ab7b14bc
I20260925 09:57:13.128899 11298 ts_tablet_manager.cc:1939] T d48d4517c13049389c5b2ba1ab7b14bc P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
W20260925 09:57:13.132874 10386 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c07c6205d1284a6fbb369c010a226da7 consensus state has no leader
I20260925 09:57:13.130954 11298 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 67) took 1382 ms. Trace:
I20260925 09:57:13.136826 11298 rpcz_store.cc:276] 0925 09:57:11.815724 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:11.815829 (+ 105us) service_pool.cc:227] Handling call
0925 09:57:13.012850 (+1197021us) ts_tablet_manager.cc:1145] Deleting tablet d48d4517c13049389c5b2ba1ab7b14bc
0925 09:57:13.012895 (+ 45us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:13.060407 (+ 47512us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:13.127536 (+ 67129us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:13.130901 (+ 3365us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1196462}
I20260925 09:57:13.138195 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet d48d4517c13049389c5b2ba1ab7b14bc (table default.Table_699fb6630e69435d946304ff8473d1e8 [id=1b1ce9c0cada4beeae7e02eb3105a564]) successfully deleted
I20260925 09:57:13.138123 11298 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 68) took 1322 ms. Trace:
I20260925 09:57:13.140161 11298 rpcz_store.cc:276] 0925 09:57:11.816190 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:11.816689 (+ 499us) service_pool.cc:227] Handling call
0925 09:57:13.137864 (+1321175us) ts_tablet_manager.cc:1145] Deleting tablet 931769e78f32467b8f27bacb286a7a60
0925 09:57:13.137907 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:13.138074 (+ 167us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1320587}
I20260925 09:57:13.139253 11085 tablet_bootstrap.cc:654] T 0e882284a014455192375daad391efaf P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:13.141377 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 931769e78f32467b8f27bacb286a7a60 because the tablet was not found. No further retry: Not found: Tablet not found: 931769e78f32467b8f27bacb286a7a60
I20260925 09:57:13.143728 11298 tablet_replica.cc:330] T d8be238061c146eb96d17a22c1b2104a P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:13.145561 11298 raft_consensus.cc:2229] T d8be238061c146eb96d17a22c1b2104a P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:13.147688 11298 pending_rounds.cc:62] T d8be238061c146eb96d17a22c1b2104a P b8850818e2ee42a082802d0095c08d21: Trying to abort 1 pending ops.
I20260925 09:57:13.148773 11298 pending_rounds.cc:68] T d8be238061c146eb96d17a22c1b2104a P b8850818e2ee42a082802d0095c08d21: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192626223239168 op_type: NO_OP noop_request { }
I20260925 09:57:13.149745 11298 raft_consensus.cc:2866] T d8be238061c146eb96d17a22c1b2104a P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:57:13.159880 11298 raft_consensus.cc:2258] T d8be238061c146eb96d17a22c1b2104a P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:13.164398 10389 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: default.Table_396cf11a26a04a17a2ad247fdf6a6c1b [id=e493befbf9d94366aa09d518014389eb] alter complete (version 1)
I20260925 09:57:13.169826 11298 ts_tablet_manager.cc:1905] T d8be238061c146eb96d17a22c1b2104a P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:13.170706 10819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 752105c1514f4c1db2a11b95b413b4f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:13.171340 11361 consensus_queue.cc:1035] T 8b281fc20e6f42239fee80d05da88ab6 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:13.177210 10659 tablet_service.cc:1514] Processing DeleteTablet for tablet aad3cb5e33f64cb885d51f4ea089307b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:11 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:13.180820 10817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41df75fe4b8e4856bdbb5ead3cf4fd24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:13.179371 11292 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 77) took 1682 ms. Trace:
I20260925 09:57:13.182618 11292 rpcz_store.cc:276] 0925 09:57:11.497422 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:11.649332 (+151910us) service_pool.cc:227] Handling call
0925 09:57:12.922520 (+1273188us) ts_tablet_manager.cc:1145] Deleting tablet aad3cb5e33f64cb885d51f4ea089307b
0925 09:57:12.922568 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:12.987204 (+ 64636us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:13.029597 (+ 42393us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:13.140562 (+110965us) spinlock_profiling.cc:243] Waited 84.1 ms on lock 0x7b5800001898. stack: 00007f0270186033 00007f0270185f30 00007f026fd07534 00000000004db2b4 00000000004db249 00007f026fffcf11 00007f0270180851 00007f0270180e82 00007f02790408b9 00007f02780745d8 00007f0278138e42 00007f0278168e10 00007f027814fe0d 00007f027814fdb9 00007f027814fd49 00007f027814fd11
0925 09:57:13.179309 (+ 38747us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":193533312,"tablet-delete.queue_time_us":1154563}
I20260925 09:57:13.184501 11292 tablet_replica.cc:330] T e48d86c20e9e4db990c23359ff1d1a6d P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:13.185564 11292 raft_consensus.cc:2229] T e48d86c20e9e4db990c23359ff1d1a6d P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:13.186085 11292 raft_consensus.cc:2258] T e48d86c20e9e4db990c23359ff1d1a6d P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:13.187194 10815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89a143a5623e4ef38027381a56c95227. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:13.188524 11292 ts_tablet_manager.cc:1905] T e48d86c20e9e4db990c23359ff1d1a6d P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:12.980865 11394 tablet.cc:1722] T 0a2bbe141e564a96be05de86f2993234 P be97ab5d75934a648d4ab76341193ec0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:13.221822 11085 tablet_bootstrap.cc:492] T 0e882284a014455192375daad391efaf P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:13.222259 11085 ts_tablet_manager.cc:1397] T 0e882284a014455192375daad391efaf P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.098s user 0.010s sys 0.000s
W20260925 09:57:13.224478 10379 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.62:34455, user_credentials={real_user=slave}} blocked reactor thread for 95076.4us
I20260925 09:57:13.241071 10664 tablet_service.cc:1514] Processing DeleteTablet for tablet d48d4517c13049389c5b2ba1ab7b14bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:07 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:13.243069 11134 raft_consensus.cc:482] T 3be0eccc65564bfba3d6e5c398c61fb6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:13.243590 11134 raft_consensus.cc:504] T 3be0eccc65564bfba3d6e5c398c61fb6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:13.246217 11085 raft_consensus.cc:348] T 0e882284a014455192375daad391efaf P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:13.246980 11085 raft_consensus.cc:374] T 0e882284a014455192375daad391efaf P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:13.247231 11085 raft_consensus.cc:729] T 0e882284a014455192375daad391efaf P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:13.246408 11134 leader_election.cc:288] T 3be0eccc65564bfba3d6e5c398c61fb6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:13.247982 11085 consensus_queue.cc:260] T 0e882284a014455192375daad391efaf P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:13.249574 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3be0eccc65564bfba3d6e5c398c61fb6" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:13.251190 10918 leader_election.cc:341] T 3be0eccc65564bfba3d6e5c398c61fb6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:13.251108 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3be0eccc65564bfba3d6e5c398c61fb6" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:13.256006 10919 leader_election.cc:341] T 3be0eccc65564bfba3d6e5c398c61fb6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:13.256506 10919 leader_election.cc:302] T 3be0eccc65564bfba3d6e5c398c61fb6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:13.257685 11134 raft_consensus.cc:2726] T 3be0eccc65564bfba3d6e5c398c61fb6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:13.258121 11320 consensus_queue.cc:1035] T 26b7c37959d042b48234acc79290936f P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:13.268417 11084 tablet_bootstrap.cc:492] T ef22f64036684dce9d7c69c37722f468 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:13.284636 11084 tablet_bootstrap.cc:654] T ef22f64036684dce9d7c69c37722f468 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:13.300324 11292 ts_tablet_manager.cc:1918] T e48d86c20e9e4db990c23359ff1d1a6d P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:13.302408 11292 log.cc:1196] T e48d86c20e9e4db990c23359ff1d1a6d P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/e48d86c20e9e4db990c23359ff1d1a6d
I20260925 09:57:13.308957 11292 ts_tablet_manager.cc:1939] T e48d86c20e9e4db990c23359ff1d1a6d P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:13.309522 11084 tablet_bootstrap.cc:492] T ef22f64036684dce9d7c69c37722f468 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:13.310045 11084 ts_tablet_manager.cc:1397] T ef22f64036684dce9d7c69c37722f468 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.042s user 0.012s sys 0.001s
I20260925 09:57:13.312462 11084 raft_consensus.cc:348] T ef22f64036684dce9d7c69c37722f468 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:13.313269 11084 raft_consensus.cc:374] T ef22f64036684dce9d7c69c37722f468 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:13.322031 11292 tablet_replica.cc:330] T 26b7c37959d042b48234acc79290936f P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:13.323292 11292 raft_consensus.cc:2229] T 26b7c37959d042b48234acc79290936f P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:13.324703 11292 raft_consensus.cc:2258] T 26b7c37959d042b48234acc79290936f P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:13.249966 11085 ts_tablet_manager.cc:1428] T 0e882284a014455192375daad391efaf P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.027s user 0.005s sys 0.000s
I20260925 09:57:13.327757 11085 tablet_bootstrap.cc:492] T 83b0cf21f2a144719a2e13990e1921c0 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:13.328557 11292 ts_tablet_manager.cc:1905] T 26b7c37959d042b48234acc79290936f P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:13.340585 11298 ts_tablet_manager.cc:1918] T d8be238061c146eb96d17a22c1b2104a P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:13.342710 11298 log.cc:1196] T d8be238061c146eb96d17a22c1b2104a P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/d8be238061c146eb96d17a22c1b2104a
I20260925 09:57:13.343688 11298 ts_tablet_manager.cc:1939] T d8be238061c146eb96d17a22c1b2104a P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:13.343297 10392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_5adf3f6c195e490fa4c978d7e1b4c187"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:13.345047 10392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5adf3f6c195e490fa4c978d7e1b4c187 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:13.348044 11082 tablet_bootstrap.cc:492] T 3fa527403d98425ea5cff9215d3a5e7f P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:13.348460 11082 ts_tablet_manager.cc:1397] T 3fa527403d98425ea5cff9215d3a5e7f P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.258s user 0.018s sys 0.026s
I20260925 09:57:13.350179 10389 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_8dfe0230298f4ff9b7e34161a9e90f53" } modify_external_catalogs: true
I20260925 09:57:13.350842 10389 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_8dfe0230298f4ff9b7e34161a9e90f53" } modify_external_catalogs: true
I20260925 09:57:13.350827 11082 raft_consensus.cc:348] T 3fa527403d98425ea5cff9215d3a5e7f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:13.351528 11082 raft_consensus.cc:374] T 3fa527403d98425ea5cff9215d3a5e7f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:13.352504 10387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_a45881b9149d451ea82bef40dc98e75c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:13.354518 10387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a45881b9149d451ea82bef40dc98e75c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:13.356876 11298 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 69) took 1540 ms. Trace:
I20260925 09:57:13.357230 11298 rpcz_store.cc:276] 0925 09:57:11.816500 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:11.816785 (+ 285us) service_pool.cc:227] Handling call
0925 09:57:13.143514 (+1326729us) ts_tablet_manager.cc:1145] Deleting tablet d8be238061c146eb96d17a22c1b2104a
0925 09:57:13.143568 (+ 54us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:13.282543 (+138975us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:13.340272 (+ 57729us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:13.356813 (+ 16541us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":60,"tablet-delete.queue_time_us":1326231}
I20260925 09:57:13.358826 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet d8be238061c146eb96d17a22c1b2104a (table default.Table_2bc55a20487a41b182135205d1a4b65d [id=f97ee2a971ab46abb4cbb37169fb309d]) successfully deleted
I20260925 09:57:13.351817 11082 raft_consensus.cc:729] T 3fa527403d98425ea5cff9215d3a5e7f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:13.363049 11082 consensus_queue.cc:260] T 3fa527403d98425ea5cff9215d3a5e7f P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:13.364362 11298 tablet_replica.cc:330] T e48d86c20e9e4db990c23359ff1d1a6d P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
W20260925 09:57:13.375414 10384 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.1:56563 (request call id 23) took 788 ms (client timeout 1000 ms). Trace:
W20260925 09:57:13.375874 10384 rpcz_store.cc:269] 0925 09:57:12.587167 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:12.588780 (+ 1613us) service_pool.cc:227] Handling call
0925 09:57:13.375324 (+786544us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":11}
I20260925 09:57:13.379422 11298 raft_consensus.cc:2229] T e48d86c20e9e4db990c23359ff1d1a6d P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:13.379953 11298 raft_consensus.cc:2258] T e48d86c20e9e4db990c23359ff1d1a6d P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:13.381964 11298 ts_tablet_manager.cc:1905] T e48d86c20e9e4db990c23359ff1d1a6d P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:13.390202 10661 tablet_service.cc:1467] Processing CreateTablet for tablet 689ab2b6cf1640129f6db8b113cf2fc6 (DEFAULT_TABLE table=default.Table_077f38b3329348828ef80c0f4e0c972f [id=51887beca2a54a4bb1e8f4262bc4fb4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:13.391521 10661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 689ab2b6cf1640129f6db8b113cf2fc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:13.396965 10819 tablet_service.cc:1467] Processing CreateTablet for tablet ef6fefaa76d44fe798201aab2f21b9c7 (DEFAULT_TABLE table=default.Table_46e4c497898b454c85b7c2a299ebf3d9 [id=80b48dd413e3446eb7ac65e5f957b5dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:13.397253 10817 tablet_service.cc:1467] Processing CreateTablet for tablet 42ec275c905343c4be1b72dc2bd0f6e3 (DEFAULT_TABLE table=default.Table_c2ce8fea3dc743379c1912445278a487 [id=cf07ed8756904011ab3dac900ea6705a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:13.398053 10819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef6fefaa76d44fe798201aab2f21b9c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:13.398429 10814 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d661d5000894c45ac3d1c27f0e72889 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:12 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:13.399168 10817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42ec275c905343c4be1b72dc2bd0f6e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:13.404156 11082 ts_tablet_manager.cc:1428] T 3fa527403d98425ea5cff9215d3a5e7f P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.055s user 0.005s sys 0.001s
I20260925 09:57:13.416334 11082 tablet_bootstrap.cc:492] T 25db96e00e6646eaa1190b85c989958f P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:13.404825 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet aad3cb5e33f64cb885d51f4ea089307b (table default.Table_2d7acf0d8022443090ba12cd87956502 [id=9d2f4f9f46394332bec9f8beda0d9595]) successfully deleted
I20260925 09:57:13.430517 10658 tablet_service.cc:1467] Processing CreateTablet for tablet 359090244c4748e8a2d45e3c77e89498 (DEFAULT_TABLE table=default.Table_3f0e11249b90447595581ed433d20c5b [id=d0cfc580b2c64772b90d93e44f1fdb85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:13.432124 10658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 359090244c4748e8a2d45e3c77e89498. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:13.429261 11082 tablet_bootstrap.cc:654] T 25db96e00e6646eaa1190b85c989958f P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:13.443492 11292 ts_tablet_manager.cc:1918] T 26b7c37959d042b48234acc79290936f P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:13.448916 11292 log.cc:1196] T 26b7c37959d042b48234acc79290936f P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/26b7c37959d042b48234acc79290936f
I20260925 09:57:13.449707 11292 ts_tablet_manager.cc:1939] T 26b7c37959d042b48234acc79290936f P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:13.450055 11082 tablet_bootstrap.cc:492] T 25db96e00e6646eaa1190b85c989958f P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:13.450443 11082 ts_tablet_manager.cc:1397] T 25db96e00e6646eaa1190b85c989958f P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.034s user 0.012s sys 0.000s
I20260925 09:57:13.451409 11085 tablet_bootstrap.cc:654] T 83b0cf21f2a144719a2e13990e1921c0 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:13.453609 11320 raft_consensus.cc:482] T 3fa527403d98425ea5cff9215d3a5e7f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:13.454068 11320 raft_consensus.cc:504] T 3fa527403d98425ea5cff9215d3a5e7f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:13.455458 11292 tablet_replica.cc:330] T b7646a1cde1b45b9bf7928c3e13bcbd3 P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:13.457818 11320 leader_election.cc:288] T 3fa527403d98425ea5cff9215d3a5e7f P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:13.460117 11292 raft_consensus.cc:2229] T b7646a1cde1b45b9bf7928c3e13bcbd3 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:13.460587 11292 raft_consensus.cc:2258] T b7646a1cde1b45b9bf7928c3e13bcbd3 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:13.462285 11298 ts_tablet_manager.cc:1918] T e48d86c20e9e4db990c23359ff1d1a6d P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:13.462514 11292 ts_tablet_manager.cc:1905] T b7646a1cde1b45b9bf7928c3e13bcbd3 P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:13.462872 11298 log.cc:1196] T e48d86c20e9e4db990c23359ff1d1a6d P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/e48d86c20e9e4db990c23359ff1d1a6d
I20260925 09:57:13.463567 11298 ts_tablet_manager.cc:1939] T e48d86c20e9e4db990c23359ff1d1a6d P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:13.462369 11082 raft_consensus.cc:348] T 25db96e00e6646eaa1190b85c989958f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:13.465461 11298 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 71) took 1543 ms. Trace:
W20260925 09:57:13.466161 10621 consensus_peers.cc:487] T 26b7c37959d042b48234acc79290936f P b8850818e2ee42a082802d0095c08d21 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 26b7c37959d042b48234acc79290936f. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:13.465780 11298 rpcz_store.cc:276] 0925 09:57:11.921819 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:11.921944 (+ 125us) service_pool.cc:227] Handling call
0925 09:57:13.364184 (+1442240us) ts_tablet_manager.cc:1145] Deleting tablet e48d86c20e9e4db990c23359ff1d1a6d
0925 09:57:13.364232 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:13.445197 (+ 80965us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:13.462067 (+ 16870us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:13.465406 (+ 3339us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1441670}
I20260925 09:57:13.467327 11298 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 73) took 1480 ms. Trace:
I20260925 09:57:13.465619 11082 raft_consensus.cc:374] T 25db96e00e6646eaa1190b85c989958f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:13.467779 11082 raft_consensus.cc:729] T 25db96e00e6646eaa1190b85c989958f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:13.467744 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fa527403d98425ea5cff9215d3a5e7f" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:13.467653 11298 rpcz_store.cc:276] 0925 09:57:11.987301 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:11.987457 (+ 156us) service_pool.cc:227] Handling call
0925 09:57:13.466989 (+1479532us) ts_tablet_manager.cc:1145] Deleting tablet 4c4e8790282a49fd849a34ca11bef5bf
0925 09:57:13.467031 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:13.467284 (+ 253us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1478876}
I20260925 09:57:13.469237 11298 tablet_replica.cc:330] T 26b7c37959d042b48234acc79290936f P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:13.469405 10817 tablet_service.cc:1467] Processing CreateTablet for tablet 689ab2b6cf1640129f6db8b113cf2fc6 (DEFAULT_TABLE table=default.Table_077f38b3329348828ef80c0f4e0c972f [id=51887beca2a54a4bb1e8f4262bc4fb4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:13.469955 11298 raft_consensus.cc:2229] T 26b7c37959d042b48234acc79290936f P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:13.470515 10817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 689ab2b6cf1640129f6db8b113cf2fc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:13.469120 11082 consensus_queue.cc:260] T 25db96e00e6646eaa1190b85c989958f P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:13.470814 11298 pending_rounds.cc:62] T 26b7c37959d042b48234acc79290936f P b8850818e2ee42a082802d0095c08d21: Trying to abort 1 pending ops.
I20260925 09:57:13.472326 11298 pending_rounds.cc:68] T 26b7c37959d042b48234acc79290936f P b8850818e2ee42a082802d0095c08d21: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192628754423808 op_type: NO_OP noop_request { }
I20260925 09:57:13.484719 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet e48d86c20e9e4db990c23359ff1d1a6d (table default.Table_3b9695593bd6426e8932f7b11f01d1ac [id=c524d4b2bbb1495cbe85a6f1b3d595d2]) successfully deleted
I20260925 09:57:13.489305 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet e48d86c20e9e4db990c23359ff1d1a6d (table default.Table_3b9695593bd6426e8932f7b11f01d1ac [id=c524d4b2bbb1495cbe85a6f1b3d595d2]) successfully deleted
W20260925 09:57:13.490118 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 4c4e8790282a49fd849a34ca11bef5bf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4c4e8790282a49fd849a34ca11bef5bf already in progress: creating tablet
I20260925 09:57:13.490690 10819 tablet_service.cc:1467] Processing CreateTablet for tablet 26fae3fbfe374f348a1729c21ec751b9 (DEFAULT_TABLE table=default.Table_0011387b45014de4b24b466f4cb907f7 [id=e3f7aa5e988c4b6c98be5cd3d465245d]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 09:57:13.490909 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 6b03e5aad59e46a6b0e43b9cb36278d7 because the tablet was not found. No further retry: Not found: Tablet not found: 6b03e5aad59e46a6b0e43b9cb36278d7
I20260925 09:57:13.491765 10819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26fae3fbfe374f348a1729c21ec751b9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:13.487251 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 4c4e8790282a49fd849a34ca11bef5bf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4c4e8790282a49fd849a34ca11bef5bf already in progress: creating tablet
W20260925 09:57:13.492471 10621 leader_election.cc:341] T 3fa527403d98425ea5cff9215d3a5e7f P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:13.492583 10661 tablet_service.cc:1467] Processing CreateTablet for tablet 26fae3fbfe374f348a1729c21ec751b9 (DEFAULT_TABLE table=default.Table_0011387b45014de4b24b466f4cb907f7 [id=e3f7aa5e988c4b6c98be5cd3d465245d]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 09:57:13.496948 10621 consensus_peers.cc:487] T b7646a1cde1b45b9bf7928c3e13bcbd3 P b8850818e2ee42a082802d0095c08d21 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:13.497413 10661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26fae3fbfe374f348a1729c21ec751b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:13.498245 11320 consensus_queue.cc:1035] T 8c13a2f288b14d03a3ca4fdd4889cbad P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:13.504207 10817 tablet_service.cc:1467] Processing CreateTablet for tablet 359090244c4748e8a2d45e3c77e89498 (DEFAULT_TABLE table=default.Table_3f0e11249b90447595581ed433d20c5b [id=d0cfc580b2c64772b90d93e44f1fdb85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:13.505278 11298 raft_consensus.cc:2866] T 26b7c37959d042b48234acc79290936f P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:57:13.505399 10817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 359090244c4748e8a2d45e3c77e89498. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:13.505645 11298 raft_consensus.cc:2258] T 26b7c37959d042b48234acc79290936f P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:13.508941 11082 ts_tablet_manager.cc:1428] T 25db96e00e6646eaa1190b85c989958f P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.058s user 0.005s sys 0.001s
I20260925 09:57:13.517381 11082 tablet_bootstrap.cc:492] T 83b0cf21f2a144719a2e13990e1921c0 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:13.518510 11409 tablet.cc:1722] T 0a2bbe141e564a96be05de86f2993234 P c43461c4ee854ae1ac52b873ccada318: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:13.522678 11082 tablet_bootstrap.cc:654] T 83b0cf21f2a144719a2e13990e1921c0 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:13.531766 11298 ts_tablet_manager.cc:1905] T 26b7c37959d042b48234acc79290936f P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:13.543215 11085 tablet_bootstrap.cc:492] T 83b0cf21f2a144719a2e13990e1921c0 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:13.543756 11085 ts_tablet_manager.cc:1397] T 83b0cf21f2a144719a2e13990e1921c0 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.216s user 0.010s sys 0.003s
I20260925 09:57:13.546700 11085 raft_consensus.cc:348] T 83b0cf21f2a144719a2e13990e1921c0 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:13.547528 11085 raft_consensus.cc:374] T 83b0cf21f2a144719a2e13990e1921c0 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:13.551079 11085 raft_consensus.cc:729] T 83b0cf21f2a144719a2e13990e1921c0 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:13.574393 11292 ts_tablet_manager.cc:1918] T b7646a1cde1b45b9bf7928c3e13bcbd3 P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:13.574862 11292 log.cc:1196] T b7646a1cde1b45b9bf7928c3e13bcbd3 P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/b7646a1cde1b45b9bf7928c3e13bcbd3
I20260925 09:57:13.575629 11292 ts_tablet_manager.cc:1939] T b7646a1cde1b45b9bf7928c3e13bcbd3 P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:13.577615 11292 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 89) took 1025 ms. Trace:
I20260925 09:57:13.587471 11292 rpcz_store.cc:276] 0925 09:57:12.552682 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:12.553268 (+ 586us) service_pool.cc:227] Handling call
0925 09:57:13.455285 (+902017us) ts_tablet_manager.cc:1145] Deleting tablet b7646a1cde1b45b9bf7928c3e13bcbd3
0925 09:57:13.455329 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:13.539310 (+ 83981us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:13.574163 (+ 34853us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:13.577560 (+ 3397us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":3848,"tablet-delete.queue_time_us":901497}
I20260925 09:57:13.575033 11085 consensus_queue.cc:260] T 83b0cf21f2a144719a2e13990e1921c0 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:13.604653 11292 tablet_replica.cc:330] T 8c13a2f288b14d03a3ca4fdd4889cbad P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
W20260925 09:57:13.605806 10621 consensus_peers.cc:487] T 8c13a2f288b14d03a3ca4fdd4889cbad P b8850818e2ee42a082802d0095c08d21 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:13.608415 11085 ts_tablet_manager.cc:1428] T 83b0cf21f2a144719a2e13990e1921c0 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.064s user 0.008s sys 0.001s
W20260925 09:57:13.579512 10379 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 91653.7us
I20260925 09:57:13.610986 11298 ts_tablet_manager.cc:1918] T 26b7c37959d042b48234acc79290936f P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:13.611353 11298 log.cc:1196] T 26b7c37959d042b48234acc79290936f P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/26b7c37959d042b48234acc79290936f
I20260925 09:57:13.612100 11298 ts_tablet_manager.cc:1939] T 26b7c37959d042b48234acc79290936f P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:13.608851 11292 raft_consensus.cc:2229] T 8c13a2f288b14d03a3ca4fdd4889cbad P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:13.614046 11298 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 76) took 1100 ms. Trace:
I20260925 09:57:13.614627 11292 pending_rounds.cc:62] T 8c13a2f288b14d03a3ca4fdd4889cbad P c43461c4ee854ae1ac52b873ccada318: Trying to abort 1 pending ops.
I20260925 09:57:13.615450 11292 pending_rounds.cc:68] T 8c13a2f288b14d03a3ca4fdd4889cbad P c43461c4ee854ae1ac52b873ccada318: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192633076633600 op_type: NO_OP noop_request { }
I20260925 09:57:13.617532 11292 raft_consensus.cc:2866] T 8c13a2f288b14d03a3ca4fdd4889cbad P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:57:13.617151 11361 raft_consensus.cc:482] T 0e882284a014455192375daad391efaf P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260925 09:57:13.617918 10393 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 28d60531970a4ab58ce5ef99cb3a96bf consensus state has no leader
I20260925 09:57:13.617959 11292 raft_consensus.cc:2258] T 8c13a2f288b14d03a3ca4fdd4889cbad P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:13.618091 11361 raft_consensus.cc:504] T 0e882284a014455192375daad391efaf P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:13.622963 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e882284a014455192375daad391efaf" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:13.619190 10393 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_7104d0f2dac14fa0a59685700d80c110" } new_table_name: "default.Table_76a153b441c24e029c2997b568a50df0" modify_external_catalogs: true
I20260925 09:57:13.623711 10686 raft_consensus.cc:2454] T 0e882284a014455192375daad391efaf P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
I20260925 09:57:13.625054 10775 leader_election.cc:302] T 0e882284a014455192375daad391efaf P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:13.625926 11249 raft_consensus.cc:2781] T 0e882284a014455192375daad391efaf P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:13.626296 11249 raft_consensus.cc:482] T 0e882284a014455192375daad391efaf P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:13.626629 11249 raft_consensus.cc:3037] T 0e882284a014455192375daad391efaf P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:13.622278 11361 leader_election.cc:288] T 0e882284a014455192375daad391efaf P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:13.628770 11292 ts_tablet_manager.cc:1905] T 8c13a2f288b14d03a3ca4fdd4889cbad P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 09:57:13.631722 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 6b03e5aad59e46a6b0e43b9cb36278d7 because the tablet was not found. No further retry: Not found: Tablet not found: 6b03e5aad59e46a6b0e43b9cb36278d7
I20260925 09:57:13.632833 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 26b7c37959d042b48234acc79290936f (table default.Table_fe84e50a7371410d81b423c4324b17f2 [id=2b43210a3c1b4f29a1b0f5fdc7819ef2]) successfully deleted
I20260925 09:57:13.634824 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet b7646a1cde1b45b9bf7928c3e13bcbd3 (table default.Table_9f64de2cbc4c4968bc0cfe51d0a9a487 [id=11e4b3a3251f4fa4a78de075f8ecb6f2]) successfully deleted
I20260925 09:57:13.634433 11249 raft_consensus.cc:504] T 0e882284a014455192375daad391efaf P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:13.637383 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e882284a014455192375daad391efaf" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:13.637938 10686 raft_consensus.cc:3037] T 0e882284a014455192375daad391efaf P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:13.644707 10686 raft_consensus.cc:2454] T 0e882284a014455192375daad391efaf P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:13.645957 10775 leader_election.cc:302] T 0e882284a014455192375daad391efaf P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:13.646929 11361 raft_consensus.cc:2781] T 0e882284a014455192375daad391efaf P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:13.647730 11361 raft_consensus.cc:686] T 0e882284a014455192375daad391efaf P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Becoming Leader. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Running, Role: LEADER
I20260925 09:57:13.650135 11361 consensus_queue.cc:237] T 0e882284a014455192375daad391efaf P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:13.657193 11082 tablet_bootstrap.cc:492] T 83b0cf21f2a144719a2e13990e1921c0 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:13.657723 11082 ts_tablet_manager.cc:1397] T 83b0cf21f2a144719a2e13990e1921c0 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.141s user 0.008s sys 0.008s
I20260925 09:57:13.660436 11082 raft_consensus.cc:348] T 83b0cf21f2a144719a2e13990e1921c0 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:13.661820 11249 leader_election.cc:288] T 0e882284a014455192375daad391efaf P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:13.663790 11082 raft_consensus.cc:374] T 83b0cf21f2a144719a2e13990e1921c0 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:13.664095 11082 raft_consensus.cc:729] T 83b0cf21f2a144719a2e13990e1921c0 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:13.672914 11292 ts_tablet_manager.cc:1918] T 8c13a2f288b14d03a3ca4fdd4889cbad P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:13.614405 11298 rpcz_store.cc:276] 0925 09:57:12.513753 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:12.513925 (+ 172us) service_pool.cc:227] Handling call
0925 09:57:13.469023 (+955098us) ts_tablet_manager.cc:1145] Deleting tablet 26b7c37959d042b48234acc79290936f
0925 09:57:13.469059 (+ 36us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:13.554500 (+ 85441us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:13.610769 (+ 56269us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:13.613990 (+ 3221us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":954342}
I20260925 09:57:13.683589 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 26b7c37959d042b48234acc79290936f (table default.Table_fe84e50a7371410d81b423c4324b17f2 [id=2b43210a3c1b4f29a1b0f5fdc7819ef2]) successfully deleted
I20260925 09:57:13.684269 11085 tablet_bootstrap.cc:492] T 3fa527403d98425ea5cff9215d3a5e7f P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:13.685698 11298 tablet_replica.cc:330] T b7646a1cde1b45b9bf7928c3e13bcbd3 P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:13.687243 11298 raft_consensus.cc:2229] T b7646a1cde1b45b9bf7928c3e13bcbd3 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:13.688891 11298 pending_rounds.cc:62] T b7646a1cde1b45b9bf7928c3e13bcbd3 P b8850818e2ee42a082802d0095c08d21: Trying to abort 1 pending ops.
I20260925 09:57:13.690032 11298 pending_rounds.cc:68] T b7646a1cde1b45b9bf7928c3e13bcbd3 P b8850818e2ee42a082802d0095c08d21: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192632584847360 op_type: NO_OP noop_request { }
I20260925 09:57:13.690987 11298 raft_consensus.cc:2866] T b7646a1cde1b45b9bf7928c3e13bcbd3 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:57:13.691692 11298 raft_consensus.cc:2258] T b7646a1cde1b45b9bf7928c3e13bcbd3 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 09:57:13.691684 10379 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet ef6fefaa76d44fe798201aab2f21b9c7 (table default.Table_46e4c497898b454c85b7c2a299ebf3d9 [id=80b48dd413e3446eb7ac65e5f957b5dc], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet ef6fefaa76d44fe798201aab2f21b9c7 consensus state has no leader
I20260925 09:57:13.693812 11298 ts_tablet_manager.cc:1905] T b7646a1cde1b45b9bf7928c3e13bcbd3 P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:13.695482 11292 log.cc:1196] T 8c13a2f288b14d03a3ca4fdd4889cbad P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/8c13a2f288b14d03a3ca4fdd4889cbad
I20260925 09:57:13.672252 11082 consensus_queue.cc:260] T 83b0cf21f2a144719a2e13990e1921c0 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:13.696424 11292 ts_tablet_manager.cc:1939] T 8c13a2f288b14d03a3ca4fdd4889cbad P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:13.698356 11292 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 87) took 1146 ms. Trace:
I20260925 09:57:13.703464 11292 rpcz_store.cc:276] 0925 09:57:12.552066 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:12.555175 (+ 3109us) service_pool.cc:227] Handling call
0925 09:57:13.604488 (+1049313us) ts_tablet_manager.cc:1145] Deleting tablet 8c13a2f288b14d03a3ca4fdd4889cbad
0925 09:57:13.604537 (+ 49us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:13.653824 (+ 49287us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:13.672703 (+ 18879us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:13.698305 (+ 25602us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1005619}
I20260925 09:57:13.705050 11292 tablet_replica.cc:330] T 2d661d5000894c45ac3d1c27f0e72889 P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:13.708835 11082 ts_tablet_manager.cc:1428] T 83b0cf21f2a144719a2e13990e1921c0 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.051s user 0.000s sys 0.004s
I20260925 09:57:13.712049 11082 tablet_bootstrap.cc:492] T 4c4e8790282a49fd849a34ca11bef5bf P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:13.719280 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 8c13a2f288b14d03a3ca4fdd4889cbad (table default.Table_ecec3b643d1c4b76b700090f3a3cab06 [id=52fd37f9b470404ebea6e54191436b12]) successfully deleted
W20260925 09:57:13.719980 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 2d55154ceb064f22bd6f37c8ae5dce74 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2d55154ceb064f22bd6f37c8ae5dce74 already in progress: creating tablet
I20260925 09:57:13.724541 10958 tablet_service.cc:1467] Processing CreateTablet for tablet 359090244c4748e8a2d45e3c77e89498 (DEFAULT_TABLE table=default.Table_3f0e11249b90447595581ed433d20c5b [id=d0cfc580b2c64772b90d93e44f1fdb85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:13.726706 10958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 359090244c4748e8a2d45e3c77e89498. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:13.728412 10961 tablet_service.cc:1467] Processing CreateTablet for tablet 689ab2b6cf1640129f6db8b113cf2fc6 (DEFAULT_TABLE table=default.Table_077f38b3329348828ef80c0f4e0c972f [id=51887beca2a54a4bb1e8f4262bc4fb4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:13.729318 10963 tablet_service.cc:1467] Processing CreateTablet for tablet 42ec275c905343c4be1b72dc2bd0f6e3 (DEFAULT_TABLE table=default.Table_c2ce8fea3dc743379c1912445278a487 [id=cf07ed8756904011ab3dac900ea6705a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:13.723003 10953 tablet_service.cc:1467] Processing CreateTablet for tablet 26fae3fbfe374f348a1729c21ec751b9 (DEFAULT_TABLE table=default.Table_0011387b45014de4b24b466f4cb907f7 [id=e3f7aa5e988c4b6c98be5cd3d465245d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:13.727334 11417 raft_consensus.cc:482] T 28d60531970a4ab58ce5ef99cb3a96bf P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:13.732043 11292 raft_consensus.cc:2229] T 2d661d5000894c45ac3d1c27f0e72889 P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:13.732648 11292 pending_rounds.cc:62] T 2d661d5000894c45ac3d1c27f0e72889 P c43461c4ee854ae1ac52b873ccada318: Trying to abort 1 pending ops.
I20260925 09:57:13.732869 11292 pending_rounds.cc:68] T 2d661d5000894c45ac3d1c27f0e72889 P c43461c4ee854ae1ac52b873ccada318: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192634082512896 op_type: NO_OP noop_request { }
I20260925 09:57:13.733203 11292 raft_consensus.cc:2866] T 2d661d5000894c45ac3d1c27f0e72889 P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:57:13.733503 11292 raft_consensus.cc:2258] T 2d661d5000894c45ac3d1c27f0e72889 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:13.735296 11082 tablet_bootstrap.cc:654] T 4c4e8790282a49fd849a34ca11bef5bf P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:13.736337 10963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42ec275c905343c4be1b72dc2bd0f6e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:13.738101 10961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 689ab2b6cf1640129f6db8b113cf2fc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:13.737838 10983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fa527403d98425ea5cff9215d3a5e7f" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:13.739051 10983 raft_consensus.cc:2454] T 3fa527403d98425ea5cff9215d3a5e7f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:13.739768 10953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26fae3fbfe374f348a1729c21ec751b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:13.313606 11084 raft_consensus.cc:729] T ef22f64036684dce9d7c69c37722f468 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:13.744576 11084 consensus_queue.cc:260] T ef22f64036684dce9d7c69c37722f468 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:13.752177 11292 ts_tablet_manager.cc:1905] T 2d661d5000894c45ac3d1c27f0e72889 P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:13.764918 11418 raft_consensus.cc:482] T c16e096038084b2b9148390b6e773294 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:13.731109 11417 raft_consensus.cc:504] T 28d60531970a4ab58ce5ef99cb3a96bf P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:13.768123 11134 raft_consensus.cc:482] T 2d55154ceb064f22bd6f37c8ae5dce74 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:13.770077 11418 raft_consensus.cc:504] T c16e096038084b2b9148390b6e773294 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:13.770212 11134 raft_consensus.cc:504] T 2d55154ceb064f22bd6f37c8ae5dce74 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:13.772099 11134 leader_election.cc:288] T 2d55154ceb064f22bd6f37c8ae5dce74 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:13.773012 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c16e096038084b2b9148390b6e773294" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:13.773537 10391 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 296f59c215104f2394e7c8e70f0f4987
I20260925 09:57:13.768553 11245 ts_tablet_manager.cc:1918] T 2d661d5000894c45ac3d1c27f0e72889 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:13.775157 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d55154ceb064f22bd6f37c8ae5dce74" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:13.775518 11245 log.cc:1196] T 2d661d5000894c45ac3d1c27f0e72889 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/2d661d5000894c45ac3d1c27f0e72889
I20260925 09:57:13.776384 11245 ts_tablet_manager.cc:1939] T 2d661d5000894c45ac3d1c27f0e72889 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
W20260925 09:57:13.776516 10919 leader_election.cc:341] T c16e096038084b2b9148390b6e773294 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:13.777212 10919 leader_election.cc:341] T 2d55154ceb064f22bd6f37c8ae5dce74 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:13.778316 10772 leader_election.cc:341] T 0e882284a014455192375daad391efaf P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 0e882284a014455192375daad391efaf
W20260925 09:57:13.779594 10772 leader_election.cc:341] T 0e882284a014455192375daad391efaf P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 0e882284a014455192375daad391efaf
I20260925 09:57:13.785389 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28d60531970a4ab58ce5ef99cb3a96bf" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:13.785637 11082 tablet_bootstrap.cc:492] T 4c4e8790282a49fd849a34ca11bef5bf P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:13.786068 11082 ts_tablet_manager.cc:1397] T 4c4e8790282a49fd849a34ca11bef5bf P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.074s user 0.008s sys 0.001s
W20260925 09:57:13.786636 10919 leader_election.cc:341] T 28d60531970a4ab58ce5ef99cb3a96bf P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:13.787802 11417 leader_election.cc:288] T 28d60531970a4ab58ce5ef99cb3a96bf P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
W20260925 09:57:13.790167 10619 consensus_peers.cc:487] T 8c13a2f288b14d03a3ca4fdd4889cbad P b8850818e2ee42a082802d0095c08d21 -> Peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Couldn't send request to peer be97ab5d75934a648d4ab76341193ec0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8c13a2f288b14d03a3ca4fdd4889cbad. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:13.791771 11418 leader_election.cc:288] T c16e096038084b2b9148390b6e773294 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:13.798269 11298 ts_tablet_manager.cc:1918] T b7646a1cde1b45b9bf7928c3e13bcbd3 P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:13.799652 11298 log.cc:1196] T b7646a1cde1b45b9bf7928c3e13bcbd3 P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/b7646a1cde1b45b9bf7928c3e13bcbd3
I20260925 09:57:13.790413 11082 raft_consensus.cc:348] T 4c4e8790282a49fd849a34ca11bef5bf P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:13.800130 11082 raft_consensus.cc:374] T 4c4e8790282a49fd849a34ca11bef5bf P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:13.800454 11298 ts_tablet_manager.cc:1939] T b7646a1cde1b45b9bf7928c3e13bcbd3 P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:13.802526 11298 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 77) took 1285 ms. Trace:
I20260925 09:57:13.802910 11298 rpcz_store.cc:276] 0925 09:57:12.516860 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:12.517430 (+ 570us) service_pool.cc:227] Handling call
0925 09:57:13.685563 (+1168133us) ts_tablet_manager.cc:1145] Deleting tablet b7646a1cde1b45b9bf7928c3e13bcbd3
0925 09:57:13.685604 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:13.782558 (+ 96954us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:13.798034 (+ 15476us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:13.802469 (+ 4435us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1167037}
I20260925 09:57:13.804454 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet b7646a1cde1b45b9bf7928c3e13bcbd3 (table default.Table_9f64de2cbc4c4968bc0cfe51d0a9a487 [id=11e4b3a3251f4fa4a78de075f8ecb6f2]) successfully deleted
I20260925 09:57:13.805410 11298 tablet_replica.cc:330] T 8c13a2f288b14d03a3ca4fdd4889cbad P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:13.806213 11298 raft_consensus.cc:2229] T 8c13a2f288b14d03a3ca4fdd4889cbad P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:13.807111 11298 raft_consensus.cc:2258] T 8c13a2f288b14d03a3ca4fdd4889cbad P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:13.800450 11082 raft_consensus.cc:729] T 4c4e8790282a49fd849a34ca11bef5bf P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:13.808202 11082 consensus_queue.cc:260] T 4c4e8790282a49fd849a34ca11bef5bf P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:13.809284 11298 ts_tablet_manager.cc:1905] T 8c13a2f288b14d03a3ca4fdd4889cbad P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:13.819000 11292 ts_tablet_manager.cc:1918] T 2d661d5000894c45ac3d1c27f0e72889 P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:13.823005 11082 ts_tablet_manager.cc:1428] T 4c4e8790282a49fd849a34ca11bef5bf P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.035s user 0.005s sys 0.001s
I20260925 09:57:13.819412 11292 log.cc:1196] T 2d661d5000894c45ac3d1c27f0e72889 P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/2d661d5000894c45ac3d1c27f0e72889
I20260925 09:57:13.824249 11292 ts_tablet_manager.cc:1939] T 2d661d5000894c45ac3d1c27f0e72889 P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:13.825134 11388 raft_consensus.cc:482] T 25db96e00e6646eaa1190b85c989958f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:13.825547 11388 raft_consensus.cc:504] T 25db96e00e6646eaa1190b85c989958f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:13.827917 11388 leader_election.cc:288] T 25db96e00e6646eaa1190b85c989958f P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:13.832301 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25db96e00e6646eaa1190b85c989958f" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:13.835026 10621 leader_election.cc:341] T 25db96e00e6646eaa1190b85c989958f P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:13.830771 11082 tablet_bootstrap.cc:492] T 833fac37485141f6a18104eaea907b0e P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:13.836715 10387 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_ce8527f03cfc4ae29d76f64597e0b075" } new_table_name: "default.Table_3a00773e32a34a41bead0b4116e4c6df" modify_external_catalogs: true
I20260925 09:57:13.841114 11082 tablet_bootstrap.cc:654] T 833fac37485141f6a18104eaea907b0e P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:13.840349 10392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_09fe8f366734488c93a79eeeec2c96f9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:13.842339 10392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_09fe8f366734488c93a79eeeec2c96f9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:13.858448 10389 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 8b281fc20e6f42239fee80d05da88ab6
I20260925 09:57:13.858477 10391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_f8d796315be4442394ff13a053dd3294"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:13.860246 10391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f8d796315be4442394ff13a053dd3294 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:13.861584 11082 tablet_bootstrap.cc:492] T 833fac37485141f6a18104eaea907b0e P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:13.862097 11082 ts_tablet_manager.cc:1397] T 833fac37485141f6a18104eaea907b0e P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.032s user 0.010s sys 0.003s
I20260925 09:57:13.865528 11298 ts_tablet_manager.cc:1918] T 8c13a2f288b14d03a3ca4fdd4889cbad P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:13.865857 11298 log.cc:1196] T 8c13a2f288b14d03a3ca4fdd4889cbad P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/8c13a2f288b14d03a3ca4fdd4889cbad
I20260925 09:57:13.865656 11082 raft_consensus.cc:348] T 833fac37485141f6a18104eaea907b0e P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:13.866384 11082 raft_consensus.cc:374] T 833fac37485141f6a18104eaea907b0e P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:13.866560 11298 ts_tablet_manager.cc:1939] T 8c13a2f288b14d03a3ca4fdd4889cbad P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:13.867147 10661 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b281fc20e6f42239fee80d05da88ab6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:13 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:13.871711 11298 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 78) took 1335 ms. Trace:
I20260925 09:57:13.872035 11298 rpcz_store.cc:276] 0925 09:57:12.536548 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:12.536703 (+ 155us) service_pool.cc:227] Handling call
0925 09:57:13.805264 (+1268561us) ts_tablet_manager.cc:1145] Deleting tablet 8c13a2f288b14d03a3ca4fdd4889cbad
0925 09:57:13.805308 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:13.856275 (+ 50967us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:13.865320 (+ 9045us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:13.871656 (+ 6336us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1267908}
I20260925 09:57:13.873306 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 8c13a2f288b14d03a3ca4fdd4889cbad (table default.Table_ecec3b643d1c4b76b700090f3a3cab06 [id=52fd37f9b470404ebea6e54191436b12]) successfully deleted
I20260925 09:57:13.874315 11298 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 79) took 1183 ms. Trace:
I20260925 09:57:13.874680 11298 rpcz_store.cc:276] 0925 09:57:12.691248 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:12.691428 (+ 180us) service_pool.cc:227] Handling call
0925 09:57:13.874003 (+1182575us) ts_tablet_manager.cc:1145] Deleting tablet 2d55154ceb064f22bd6f37c8ae5dce74
0925 09:57:13.874041 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:13.874272 (+ 231us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1181860}
W20260925 09:57:13.875896 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 2d55154ceb064f22bd6f37c8ae5dce74 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2d55154ceb064f22bd6f37c8ae5dce74 already in progress: creating tablet
I20260925 09:57:13.876717 11298 tablet_replica.cc:330] T 2d661d5000894c45ac3d1c27f0e72889 P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:13.877489 11298 raft_consensus.cc:2229] T 2d661d5000894c45ac3d1c27f0e72889 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:13.866655 11082 raft_consensus.cc:729] T 833fac37485141f6a18104eaea907b0e P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:13.878095 11298 raft_consensus.cc:2258] T 2d661d5000894c45ac3d1c27f0e72889 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:13.878342 11082 consensus_queue.cc:260] T 833fac37485141f6a18104eaea907b0e P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:13.880553 11298 ts_tablet_manager.cc:1905] T 2d661d5000894c45ac3d1c27f0e72889 P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:13.888854 11084 ts_tablet_manager.cc:1428] T ef22f64036684dce9d7c69c37722f468 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.578s user 0.012s sys 0.000s
W20260925 09:57:13.889206 11084 ts_tablet_manager.cc:1460] T ef22f64036684dce9d7c69c37722f468 P be97ab5d75934a648d4ab76341193ec0: Tablet startup took 578ms
W20260925 09:57:13.889477 11084 ts_tablet_manager.cc:1462] T ef22f64036684dce9d7c69c37722f468 P be97ab5d75934a648d4ab76341193ec0: Trace:
0925 09:57:12.192305 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:12.202168 (+ 9863us) service_pool.cc:227] Handling call
0925 09:57:12.209641 (+ 7473us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0925 09:57:12.210224 (+ 583us) ts_tablet_manager.cc:676] Creating new metadata...
0925 09:57:12.254411 (+ 44187us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:12.265625 (+ 11214us) tablet_replica.cc:205] Creating consensus instance
0925 09:57:12.266006 (+ 381us) inbound_call.cc:173] Queueing success response
0925 09:57:13.055610 (+789604us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0925 09:57:13.310354 (+254744us) ts_tablet_manager.cc:1429] Starting tablet replica
0925 09:57:13.310925 (+ 571us) tablet_replica.cc:259] Starting instrumentation
0925 09:57:13.311380 (+ 455us) tablet_replica.cc:278] Starting consensus
Metrics: {"mutex_wait_us":212449,"raft.queue_time_us":26407,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":194,"tablet-open.queue_time_us":789677,"thread_start_us":66409,"threads_started":1}
I20260925 09:57:13.891232 11082 ts_tablet_manager.cc:1428] T 833fac37485141f6a18104eaea907b0e P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.028s user 0.001s sys 0.005s
I20260925 09:57:13.894556 11298 ts_tablet_manager.cc:1918] T 2d661d5000894c45ac3d1c27f0e72889 P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:13.894956 11298 log.cc:1196] T 2d661d5000894c45ac3d1c27f0e72889 P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/2d661d5000894c45ac3d1c27f0e72889
I20260925 09:57:13.895623 11298 ts_tablet_manager.cc:1939] T 2d661d5000894c45ac3d1c27f0e72889 P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:13.896524 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d55154ceb064f22bd6f37c8ae5dce74" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:13.897864 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 2d661d5000894c45ac3d1c27f0e72889 (table default.Table_c6db4ef50ad442fcb28e2646fd6b7571 [id=756dfeff38d64529944858afa495ab49]) successfully deleted
W20260925 09:57:13.899377 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet aad3cb5e33f64cb885d51f4ea089307b because the tablet was not found. No further retry: Not found: Tablet not found: aad3cb5e33f64cb885d51f4ea089307b
W20260925 09:57:13.900822 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet d48d4517c13049389c5b2ba1ab7b14bc because the tablet was not found. No further retry: Not found: Tablet not found: d48d4517c13049389c5b2ba1ab7b14bc
I20260925 09:57:13.902314 11082 tablet_bootstrap.cc:492] T c07c6205d1284a6fbb369c010a226da7 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:13.902721 11298 tablet_replica.cc:330] T 8b281fc20e6f42239fee80d05da88ab6 P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:13.903585 11298 raft_consensus.cc:2229] T 8b281fc20e6f42239fee80d05da88ab6 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:13.904104 11298 raft_consensus.cc:2258] T 8b281fc20e6f42239fee80d05da88ab6 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:13.907895 11298 ts_tablet_manager.cc:1905] T 8b281fc20e6f42239fee80d05da88ab6 P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:13.911837 10619 leader_election.cc:302] T 3fa527403d98425ea5cff9215d3a5e7f P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:13.912984 11388 raft_consensus.cc:2781] T 3fa527403d98425ea5cff9215d3a5e7f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:13.913306 11388 raft_consensus.cc:482] T 3fa527403d98425ea5cff9215d3a5e7f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:13.913637 11388 raft_consensus.cc:3037] T 3fa527403d98425ea5cff9215d3a5e7f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:13.918052 11082 tablet_bootstrap.cc:654] T c07c6205d1284a6fbb369c010a226da7 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:13.922197 11298 ts_tablet_manager.cc:1918] T 8b281fc20e6f42239fee80d05da88ab6 P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:13.922554 11298 log.cc:1196] T 8b281fc20e6f42239fee80d05da88ab6 P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/8b281fc20e6f42239fee80d05da88ab6
I20260925 09:57:13.923229 11298 ts_tablet_manager.cc:1939] T 8b281fc20e6f42239fee80d05da88ab6 P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:13.925711 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 8b281fc20e6f42239fee80d05da88ab6 (table default.Table_8dfe0230298f4ff9b7e34161a9e90f53 [id=50788524d4b24e429c978c44428dc892]) successfully deleted
I20260925 09:57:13.926893 11388 raft_consensus.cc:504] T 3fa527403d98425ea5cff9215d3a5e7f P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:13.929581 11388 leader_election.cc:288] T 3fa527403d98425ea5cff9215d3a5e7f P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:13.929502 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fa527403d98425ea5cff9215d3a5e7f" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
W20260925 09:57:13.930832 10621 leader_election.cc:341] T 3fa527403d98425ea5cff9215d3a5e7f P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:13.951525 11082 tablet_bootstrap.cc:492] T c07c6205d1284a6fbb369c010a226da7 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:13.952023 11082 ts_tablet_manager.cc:1397] T c07c6205d1284a6fbb369c010a226da7 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.050s user 0.006s sys 0.003s
I20260925 09:57:13.954351 11082 raft_consensus.cc:348] T c07c6205d1284a6fbb369c010a226da7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:13.955072 11082 raft_consensus.cc:374] T c07c6205d1284a6fbb369c010a226da7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:13.955368 11082 raft_consensus.cc:729] T c07c6205d1284a6fbb369c010a226da7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:13.956154 11082 consensus_queue.cc:260] T c07c6205d1284a6fbb369c010a226da7 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:13.959939 11082 ts_tablet_manager.cc:1428] T c07c6205d1284a6fbb369c010a226da7 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.008s user 0.002s sys 0.004s
I20260925 09:57:13.961261 10817 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b281fc20e6f42239fee80d05da88ab6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:13 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:13.961978 11292 tablet_replica.cc:330] T 8b281fc20e6f42239fee80d05da88ab6 P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:13.962659 11292 raft_consensus.cc:2229] T 8b281fc20e6f42239fee80d05da88ab6 P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:13.963554 11292 raft_consensus.cc:2258] T 8b281fc20e6f42239fee80d05da88ab6 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:13.966622 11292 ts_tablet_manager.cc:1905] T 8b281fc20e6f42239fee80d05da88ab6 P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:13.978194 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 2d661d5000894c45ac3d1c27f0e72889 (table default.Table_c6db4ef50ad442fcb28e2646fd6b7571 [id=756dfeff38d64529944858afa495ab49]) successfully deleted
I20260925 09:57:13.985684 11292 ts_tablet_manager.cc:1918] T 8b281fc20e6f42239fee80d05da88ab6 P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:13.986106 11292 log.cc:1196] T 8b281fc20e6f42239fee80d05da88ab6 P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/8b281fc20e6f42239fee80d05da88ab6
I20260925 09:57:13.986860 11292 ts_tablet_manager.cc:1939] T 8b281fc20e6f42239fee80d05da88ab6 P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:13.989542 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 8b281fc20e6f42239fee80d05da88ab6 (table default.Table_8dfe0230298f4ff9b7e34161a9e90f53 [id=50788524d4b24e429c978c44428dc892]) successfully deleted
I20260925 09:57:13.991617 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28d60531970a4ab58ce5ef99cb3a96bf" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:13.992349 11084 tablet_bootstrap.cc:492] T 3c31682bf400494f82ee3ce5e6eec441 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:13.993546 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c16e096038084b2b9148390b6e773294" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:13.995532 10917 outbound_call.cc:322] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.10.9.61:33047, user_credentials={real_user=slave}} blocked reactor thread for 64302.7us
W20260925 09:57:13.996585 10918 leader_election.cc:341] T 2d55154ceb064f22bd6f37c8ae5dce74 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:13.996932 10918 leader_election.cc:302] T 2d55154ceb064f22bd6f37c8ae5dce74 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
W20260925 09:57:13.998178 10918 leader_election.cc:341] T 28d60531970a4ab58ce5ef99cb3a96bf P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:13.998328 11420 raft_consensus.cc:2726] T 2d55154ceb064f22bd6f37c8ae5dce74 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:13.998658 10918 leader_election.cc:302] T 28d60531970a4ab58ce5ef99cb3a96bf P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:13.998721 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 2d661d5000894c45ac3d1c27f0e72889 (table default.Table_c6db4ef50ad442fcb28e2646fd6b7571 [id=756dfeff38d64529944858afa495ab49]) successfully deleted
I20260925 09:57:13.999465 11416 raft_consensus.cc:2726] T 28d60531970a4ab58ce5ef99cb3a96bf P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:14.000392 10918 leader_election.cc:341] T c16e096038084b2b9148390b6e773294 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:14.000777 10918 leader_election.cc:302] T c16e096038084b2b9148390b6e773294 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:14.001426 11416 raft_consensus.cc:2726] T c16e096038084b2b9148390b6e773294 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:13.809115 11085 tablet_bootstrap.cc:654] T 3fa527403d98425ea5cff9215d3a5e7f P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:14.023703 11082 tablet_bootstrap.cc:492] T fd73d0d52350444499abae34e8c02fab P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:14.031796 11085 tablet_bootstrap.cc:492] T 3fa527403d98425ea5cff9215d3a5e7f P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:14.032195 11085 ts_tablet_manager.cc:1397] T 3fa527403d98425ea5cff9215d3a5e7f P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.348s user 0.008s sys 0.008s
I20260925 09:57:14.034358 11085 raft_consensus.cc:348] T 3fa527403d98425ea5cff9215d3a5e7f P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:14.034996 11085 raft_consensus.cc:374] T 3fa527403d98425ea5cff9215d3a5e7f P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:14.035254 11085 raft_consensus.cc:729] T 3fa527403d98425ea5cff9215d3a5e7f P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:14.042462 10961 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b281fc20e6f42239fee80d05da88ab6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:13 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:14.043295 10983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25db96e00e6646eaa1190b85c989958f" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:14.043900 10983 raft_consensus.cc:2454] T 25db96e00e6646eaa1190b85c989958f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:14.044811 11245 tablet_replica.cc:330] T 8b281fc20e6f42239fee80d05da88ab6 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:14.044677 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fa527403d98425ea5cff9215d3a5e7f" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:14.045181 10984 raft_consensus.cc:3037] T 3fa527403d98425ea5cff9215d3a5e7f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:14.045555 11245 raft_consensus.cc:2229] T 8b281fc20e6f42239fee80d05da88ab6 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:14.046026 11245 raft_consensus.cc:2258] T 8b281fc20e6f42239fee80d05da88ab6 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:14.048976 11245 ts_tablet_manager.cc:1905] T 8b281fc20e6f42239fee80d05da88ab6 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:14.052341 10984 raft_consensus.cc:2454] T 3fa527403d98425ea5cff9215d3a5e7f P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:14.052202 10619 leader_election.cc:302] T 25db96e00e6646eaa1190b85c989958f P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:14.053681 11388 raft_consensus.cc:2781] T 25db96e00e6646eaa1190b85c989958f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:14.053989 11388 raft_consensus.cc:482] T 25db96e00e6646eaa1190b85c989958f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:14.054260 11388 raft_consensus.cc:3037] T 25db96e00e6646eaa1190b85c989958f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:14.055306 10389 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_8e0c5caa8df14066af83c49a50a172bf" } modify_external_catalogs: true
I20260925 09:57:14.055939 10389 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_8e0c5caa8df14066af83c49a50a172bf" } modify_external_catalogs: true
I20260925 09:57:14.059162 11084 tablet_bootstrap.cc:654] T 3c31682bf400494f82ee3ce5e6eec441 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:14.072335 10276 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu
/tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.9.60:36985
--webserver_interface=127.10.9.60
--webserver_port=41643
--builtin_ntp_servers=127.10.9.20:38279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.9.62:34455,127.10.9.61:33047,127.10.9.60:36985
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260925 09:57:14.080533 11082 tablet_bootstrap.cc:654] T fd73d0d52350444499abae34e8c02fab P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:14.086231 11084 tablet_bootstrap.cc:492] T 3c31682bf400494f82ee3ce5e6eec441 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:14.086606 11084 ts_tablet_manager.cc:1397] T 3c31682bf400494f82ee3ce5e6eec441 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.094s user 0.006s sys 0.002s
I20260925 09:57:14.088709 11084 raft_consensus.cc:348] T 3c31682bf400494f82ee3ce5e6eec441 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:14.089290 11084 raft_consensus.cc:374] T 3c31682bf400494f82ee3ce5e6eec441 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:14.089525 11084 raft_consensus.cc:729] T 3c31682bf400494f82ee3ce5e6eec441 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:14.091179 11361 raft_consensus.cc:482] T 83b0cf21f2a144719a2e13990e1921c0 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:14.091641 11361 raft_consensus.cc:504] T 83b0cf21f2a144719a2e13990e1921c0 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:14.093465 11361 leader_election.cc:288] T 83b0cf21f2a144719a2e13990e1921c0 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:14.094807 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83b0cf21f2a144719a2e13990e1921c0" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:14.095428 10686 raft_consensus.cc:2454] T 83b0cf21f2a144719a2e13990e1921c0 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
I20260925 09:57:14.096330 10775 leader_election.cc:302] T 83b0cf21f2a144719a2e13990e1921c0 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:14.096576 10392 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_8c5b820f0db34e3bbea2d404afb1909a" } modify_external_catalogs: true
I20260925 09:57:14.097079 10392 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_8c5b820f0db34e3bbea2d404afb1909a" } modify_external_catalogs: true
I20260925 09:57:14.099012 10385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_48f8cb39156845c8bffbb2f9f17a900b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:14.100482 10385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_48f8cb39156845c8bffbb2f9f17a900b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:14.102172 11361 raft_consensus.cc:2781] T 83b0cf21f2a144719a2e13990e1921c0 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:14.102506 11361 raft_consensus.cc:482] T 83b0cf21f2a144719a2e13990e1921c0 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:14.102802 11361 raft_consensus.cc:3037] T 83b0cf21f2a144719a2e13990e1921c0 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:14.108099 10619 leader_election.cc:302] T 3fa527403d98425ea5cff9215d3a5e7f P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:14.109217 11389 raft_consensus.cc:2781] T 3fa527403d98425ea5cff9215d3a5e7f P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:14.109643 11389 raft_consensus.cc:686] T 3fa527403d98425ea5cff9215d3a5e7f P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:14.110399 11389 consensus_queue.cc:237] T 3fa527403d98425ea5cff9215d3a5e7f P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:14.113591 10393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_136e1173ab5c41c5bc5610f6b0c259a0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:14.113894 11388 raft_consensus.cc:504] T 25db96e00e6646eaa1190b85c989958f P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:14.115051 10393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_136e1173ab5c41c5bc5610f6b0c259a0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:14.116384 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25db96e00e6646eaa1190b85c989958f" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
W20260925 09:57:14.118184 10621 leader_election.cc:341] T 25db96e00e6646eaa1190b85c989958f P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:14.119648 10379 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.60:36985, user_credentials={real_user=slave}} blocked reactor thread for 82046.6us
I20260925 09:57:14.120137 11361 raft_consensus.cc:504] T 83b0cf21f2a144719a2e13990e1921c0 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:14.124842 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83b0cf21f2a144719a2e13990e1921c0" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:14.125392 10686 raft_consensus.cc:3037] T 83b0cf21f2a144719a2e13990e1921c0 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:14.129292 11082 tablet_bootstrap.cc:492] T fd73d0d52350444499abae34e8c02fab P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:14.129642 11082 ts_tablet_manager.cc:1397] T fd73d0d52350444499abae34e8c02fab P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.120s user 0.006s sys 0.005s
I20260925 09:57:14.131697 11388 leader_election.cc:288] T 25db96e00e6646eaa1190b85c989958f P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:14.132414 11084 consensus_queue.cc:260] T 3c31682bf400494f82ee3ce5e6eec441 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:14.136170 10379 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 -> Peer b8be057e0b6d40a3a4b90f2b1f4a0fa2 (127.10.9.60:36985): Couldn't send request to peer b8be057e0b6d40a3a4b90f2b1f4a0fa2. Status: Network error: Client connection negotiation failed: client connection to 127.10.9.60:36985: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260925 09:57:14.138769 11361 leader_election.cc:288] T 83b0cf21f2a144719a2e13990e1921c0 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:14.139803 11082 raft_consensus.cc:348] T fd73d0d52350444499abae34e8c02fab P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:14.140424 11082 raft_consensus.cc:374] T fd73d0d52350444499abae34e8c02fab P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:14.140640 11082 raft_consensus.cc:729] T fd73d0d52350444499abae34e8c02fab P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:14.144066 11082 consensus_queue.cc:260] T fd73d0d52350444499abae34e8c02fab P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:14.148505 11084 ts_tablet_manager.cc:1428] T 3c31682bf400494f82ee3ce5e6eec441 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.062s user 0.005s sys 0.005s
I20260925 09:57:14.150581 11082 ts_tablet_manager.cc:1428] T fd73d0d52350444499abae34e8c02fab P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.021s user 0.005s sys 0.001s
I20260925 09:57:14.151360 11082 tablet_bootstrap.cc:492] T 69f84509cafc44dd9444218738ebc2fe P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:14.155268 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25db96e00e6646eaa1190b85c989958f" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:14.155951 10984 raft_consensus.cc:3037] T 25db96e00e6646eaa1190b85c989958f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:14.158680 11084 tablet_bootstrap.cc:492] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:14.162967 10984 raft_consensus.cc:2454] T 25db96e00e6646eaa1190b85c989958f P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:14.163554 10686 raft_consensus.cc:2454] T 83b0cf21f2a144719a2e13990e1921c0 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:14.164644 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83b0cf21f2a144719a2e13990e1921c0" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:14.164618 10775 leader_election.cc:302] T 83b0cf21f2a144719a2e13990e1921c0 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:14.165195 10984 raft_consensus.cc:2454] T 83b0cf21f2a144719a2e13990e1921c0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
I20260925 09:57:14.165372 11361 raft_consensus.cc:2781] T 83b0cf21f2a144719a2e13990e1921c0 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:14.166425 11361 raft_consensus.cc:686] T 83b0cf21f2a144719a2e13990e1921c0 P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Becoming Leader. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Running, Role: LEADER
I20260925 09:57:14.167176 11361 consensus_queue.cc:237] T 83b0cf21f2a144719a2e13990e1921c0 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:14.169958 10983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83b0cf21f2a144719a2e13990e1921c0" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:14.170477 10983 raft_consensus.cc:3037] T 83b0cf21f2a144719a2e13990e1921c0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:14.174232 11082 tablet_bootstrap.cc:654] T 69f84509cafc44dd9444218738ebc2fe P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:14.163197 11084 tablet_bootstrap.cc:654] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:14.181630 10983 raft_consensus.cc:2454] T 83b0cf21f2a144719a2e13990e1921c0 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:14.189182 10619 leader_election.cc:302] T 25db96e00e6646eaa1190b85c989958f P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:14.191154 11388 raft_consensus.cc:2781] T 25db96e00e6646eaa1190b85c989958f P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:14.191601 11388 raft_consensus.cc:686] T 25db96e00e6646eaa1190b85c989958f P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:14.192294 11388 consensus_queue.cc:237] T 25db96e00e6646eaa1190b85c989958f P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:14.207934 11082 tablet_bootstrap.cc:492] T 69f84509cafc44dd9444218738ebc2fe P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:14.210433 11082 ts_tablet_manager.cc:1397] T 69f84509cafc44dd9444218738ebc2fe P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.059s user 0.005s sys 0.004s
I20260925 09:57:14.212536 11082 raft_consensus.cc:348] T 69f84509cafc44dd9444218738ebc2fe P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:14.218673 11084 tablet_bootstrap.cc:492] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:14.219122 11084 ts_tablet_manager.cc:1397] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.061s user 0.007s sys 0.002s
I20260925 09:57:14.221175 11084 raft_consensus.cc:348] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:14.221961 11084 raft_consensus.cc:374] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:14.222285 11082 raft_consensus.cc:374] T 69f84509cafc44dd9444218738ebc2fe P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:14.222579 11084 raft_consensus.cc:729] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:14.222903 11082 raft_consensus.cc:729] T 69f84509cafc44dd9444218738ebc2fe P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:14.037021 11085 consensus_queue.cc:260] T 3fa527403d98425ea5cff9215d3a5e7f P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:14.225914 11085 ts_tablet_manager.cc:1428] T 3fa527403d98425ea5cff9215d3a5e7f P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.193s user 0.004s sys 0.002s
I20260925 09:57:14.228639 11082 consensus_queue.cc:260] T 69f84509cafc44dd9444218738ebc2fe P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:14.229650 11084 consensus_queue.cc:260] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:14.237660 11084 ts_tablet_manager.cc:1428] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.018s user 0.004s sys 0.000s
I20260925 09:57:14.239470 11082 ts_tablet_manager.cc:1428] T 69f84509cafc44dd9444218738ebc2fe P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.029s user 0.005s sys 0.000s
I20260925 09:57:14.247330 11082 tablet_bootstrap.cc:492] T 47a9fb9392244e1d89f8af2bfb42edd6 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:14.263108 11082 tablet_bootstrap.cc:654] T 47a9fb9392244e1d89f8af2bfb42edd6 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:14.270782 10391 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_76a153b441c24e029c2997b568a50df0" } new_table_name: "default.Table_ff4e7021d36b43f8b51f19960e707ac8" modify_external_catalogs: true
I20260925 09:57:14.276903 11085 tablet_bootstrap.cc:492] T 4c4e8790282a49fd849a34ca11bef5bf P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:14.289675 11084 tablet_bootstrap.cc:492] T ef6fefaa76d44fe798201aab2f21b9c7 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:14.299507 10661 tablet_service.cc:1467] Processing CreateTablet for tablet 229aaa1fc7b84fb887a8c0d8a53bf0ae (DEFAULT_TABLE table=default.Table_5641cc23eb8b41959d9656f65afb9fe8 [id=a999ca974fd84de9a56d7f2fbd5c3074]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:14.300616 10661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 229aaa1fc7b84fb887a8c0d8a53bf0ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:14.305152 10658 tablet_service.cc:1467] Processing CreateTablet for tablet c588b0c05a1a4f3ea545066183e76502 (DEFAULT_TABLE table=default.Table_8cde0139b88d439eaee476479a937bf3 [id=a2086b5e7b334d63972538ed53810c55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:14.306252 10658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c588b0c05a1a4f3ea545066183e76502. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:14.327726 11084 tablet_bootstrap.cc:654] T ef6fefaa76d44fe798201aab2f21b9c7 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:14.333041 10668 tablet_service.cc:1467] Processing CreateTablet for tablet cf35f39bcd6a41128acb2a4936fcc316 (DEFAULT_TABLE table=default.Table_fea25102a7f84d58927645a0feae51e3 [id=61a4c50b2c6e43c1a471f63b59d218cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:14.334020 10668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf35f39bcd6a41128acb2a4936fcc316. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:14.343094 10666 tablet_service.cc:1467] Processing CreateTablet for tablet 197f61525b0c4f9da441d98701d231a3 (DEFAULT_TABLE table=default.Table_1299538965b945cf859dbbd0cb7003f1 [id=cdc838b49cbb40778c9f440e9f83e96d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:14.344063 10666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 197f61525b0c4f9da441d98701d231a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:14.347847 11389 raft_consensus.cc:482] T 69f84509cafc44dd9444218738ebc2fe P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:14.348230 11389 raft_consensus.cc:504] T 69f84509cafc44dd9444218738ebc2fe P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:14.349996 11389 leader_election.cc:288] T 69f84509cafc44dd9444218738ebc2fe P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:14.352049 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69f84509cafc44dd9444218738ebc2fe" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:14.353358 10621 leader_election.cc:341] T 69f84509cafc44dd9444218738ebc2fe P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:14.353588 11085 tablet_bootstrap.cc:654] T 4c4e8790282a49fd849a34ca11bef5bf P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:14.360236 10961 tablet_service.cc:1467] Processing CreateTablet for tablet c588b0c05a1a4f3ea545066183e76502 (DEFAULT_TABLE table=default.Table_8cde0139b88d439eaee476479a937bf3 [id=a2086b5e7b334d63972538ed53810c55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:14.362036 10958 tablet_service.cc:1467] Processing CreateTablet for tablet cf35f39bcd6a41128acb2a4936fcc316 (DEFAULT_TABLE table=default.Table_fea25102a7f84d58927645a0feae51e3 [id=61a4c50b2c6e43c1a471f63b59d218cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:14.363782 10953 tablet_service.cc:1467] Processing CreateTablet for tablet 229aaa1fc7b84fb887a8c0d8a53bf0ae (DEFAULT_TABLE table=default.Table_5641cc23eb8b41959d9656f65afb9fe8 [id=a999ca974fd84de9a56d7f2fbd5c3074]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:14.364666 10983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69f84509cafc44dd9444218738ebc2fe" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:14.365223 10983 raft_consensus.cc:2454] T 69f84509cafc44dd9444218738ebc2fe P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:14.367132 10961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c588b0c05a1a4f3ea545066183e76502. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:14.372048 10958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf35f39bcd6a41128acb2a4936fcc316. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:14.377451 10953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 229aaa1fc7b84fb887a8c0d8a53bf0ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:14.380401 10817 tablet_service.cc:1467] Processing CreateTablet for tablet 229aaa1fc7b84fb887a8c0d8a53bf0ae (DEFAULT_TABLE table=default.Table_5641cc23eb8b41959d9656f65afb9fe8 [id=a999ca974fd84de9a56d7f2fbd5c3074]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:14.381438 10817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 229aaa1fc7b84fb887a8c0d8a53bf0ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:14.381685 10666 tablet_service.cc:1467] Processing CreateTablet for tablet c8522c08bdd94806b5c84f5a41ca5bf4 (DEFAULT_TABLE table=default.Table_ae6bc3dfc98244fda3409f53da244f7a [id=ac3973a2d08d45838cfa059e39fa5f8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:14.382689 10666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8522c08bdd94806b5c84f5a41ca5bf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:14.384661 10819 tablet_service.cc:1467] Processing CreateTablet for tablet c588b0c05a1a4f3ea545066183e76502 (DEFAULT_TABLE table=default.Table_8cde0139b88d439eaee476479a937bf3 [id=a2086b5e7b334d63972538ed53810c55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:14.385511 10819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c588b0c05a1a4f3ea545066183e76502. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:14.387257 11361 raft_consensus.cc:482] T 3fa527403d98425ea5cff9215d3a5e7f P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:14.387668 11361 raft_consensus.cc:504] T 3fa527403d98425ea5cff9215d3a5e7f P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:14.389573 11361 leader_election.cc:288] T 3fa527403d98425ea5cff9215d3a5e7f P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:14.396185 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fa527403d98425ea5cff9215d3a5e7f" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:14.399664 10385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_66b8a45b14bf441d80ab32ee756de608"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:14.401119 10385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_66b8a45b14bf441d80ab32ee756de608 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:14.408633 10815 tablet_service.cc:1467] Processing CreateTablet for tablet cf35f39bcd6a41128acb2a4936fcc316 (DEFAULT_TABLE table=default.Table_fea25102a7f84d58927645a0feae51e3 [id=61a4c50b2c6e43c1a471f63b59d218cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:14.409722 10815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf35f39bcd6a41128acb2a4936fcc316. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:14.412670 10814 tablet_service.cc:1467] Processing CreateTablet for tablet 197f61525b0c4f9da441d98701d231a3 (DEFAULT_TABLE table=default.Table_1299538965b945cf859dbbd0cb7003f1 [id=cdc838b49cbb40778c9f440e9f83e96d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:14.413667 10814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 197f61525b0c4f9da441d98701d231a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:14.414608 10664 tablet_service.cc:1467] Processing CreateTablet for tablet b555eeba505740988c704df5c734e7c0 (DEFAULT_TABLE table=default.Table_055cf78170f24a8e94074c1f02e2f5a3 [id=3e32026d175b4b92a3ee036f17e54b9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:14.415689 10664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b555eeba505740988c704df5c734e7c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:14.428908 10813 tablet_service.cc:1467] Processing CreateTablet for tablet c8522c08bdd94806b5c84f5a41ca5bf4 (DEFAULT_TABLE table=default.Table_ae6bc3dfc98244fda3409f53da244f7a [id=ac3973a2d08d45838cfa059e39fa5f8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:14.430368 10813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8522c08bdd94806b5c84f5a41ca5bf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:14.433983 10816 tablet_service.cc:1467] Processing CreateTablet for tablet b555eeba505740988c704df5c734e7c0 (DEFAULT_TABLE table=default.Table_055cf78170f24a8e94074c1f02e2f5a3 [id=3e32026d175b4b92a3ee036f17e54b9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:14.435066 10816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b555eeba505740988c704df5c734e7c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:14.446002 11082 tablet_bootstrap.cc:492] T 47a9fb9392244e1d89f8af2bfb42edd6 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:14.446412 11082 ts_tablet_manager.cc:1397] T 47a9fb9392244e1d89f8af2bfb42edd6 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.199s user 0.010s sys 0.000s
I20260925 09:57:14.448570 11082 raft_consensus.cc:348] T 47a9fb9392244e1d89f8af2bfb42edd6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:14.449189 11082 raft_consensus.cc:374] T 47a9fb9392244e1d89f8af2bfb42edd6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:14.449460 11082 raft_consensus.cc:729] T 47a9fb9392244e1d89f8af2bfb42edd6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:14.450055 11082 consensus_queue.cc:260] T 47a9fb9392244e1d89f8af2bfb42edd6 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:14.451962 11082 ts_tablet_manager.cc:1428] T 47a9fb9392244e1d89f8af2bfb42edd6 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260925 09:57:14.459194 10659 tablet_service.cc:1467] Processing CreateTablet for tablet cfa619b699004882a82976f0ef8204ad (DEFAULT_TABLE table=default.Table_7e0155a70cc54485b8fdd4d936460d1c [id=c90c671ed4f043c2b56a1f84ff9a42cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:14.461639 10818 tablet_service.cc:1467] Processing CreateTablet for tablet cfa619b699004882a82976f0ef8204ad (DEFAULT_TABLE table=default.Table_7e0155a70cc54485b8fdd4d936460d1c [id=c90c671ed4f043c2b56a1f84ff9a42cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:14.462771 10818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfa619b699004882a82976f0ef8204ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:14.480880 10812 tablet_service.cc:1467] Processing CreateTablet for tablet 67ce2e3325214aa6bdc3347e9e8f82b0 (DEFAULT_TABLE table=default.Table_5157be3699be4fa4ac153a69f6010ecc [id=9cd1024c9d784df9af5f8239e39d1be7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:14.482375 10812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67ce2e3325214aa6bdc3347e9e8f82b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:14.484876 10619 leader_election.cc:302] T 69f84509cafc44dd9444218738ebc2fe P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:14.485934 11084 tablet_bootstrap.cc:492] T ef6fefaa76d44fe798201aab2f21b9c7 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:14.486058 11389 raft_consensus.cc:2781] T 69f84509cafc44dd9444218738ebc2fe P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:14.486382 11084 ts_tablet_manager.cc:1397] T ef6fefaa76d44fe798201aab2f21b9c7 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.197s user 0.006s sys 0.003s
I20260925 09:57:14.486403 11389 raft_consensus.cc:482] T 69f84509cafc44dd9444218738ebc2fe P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:14.486694 11389 raft_consensus.cc:3037] T 69f84509cafc44dd9444218738ebc2fe P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:14.487200 10665 tablet_service.cc:1467] Processing CreateTablet for tablet 67ce2e3325214aa6bdc3347e9e8f82b0 (DEFAULT_TABLE table=default.Table_5157be3699be4fa4ac153a69f6010ecc [id=9cd1024c9d784df9af5f8239e39d1be7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:14.491844 11085 tablet_bootstrap.cc:492] T 4c4e8790282a49fd849a34ca11bef5bf P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:14.492236 11085 ts_tablet_manager.cc:1397] T 4c4e8790282a49fd849a34ca11bef5bf P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.216s user 0.036s sys 0.067s
I20260925 09:57:14.494550 11085 raft_consensus.cc:348] T 4c4e8790282a49fd849a34ca11bef5bf P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:14.495291 11085 raft_consensus.cc:374] T 4c4e8790282a49fd849a34ca11bef5bf P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:14.495558 11085 raft_consensus.cc:729] T 4c4e8790282a49fd849a34ca11bef5bf P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:14.496153 11085 consensus_queue.cc:260] T 4c4e8790282a49fd849a34ca11bef5bf P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:14.498061 11085 ts_tablet_manager.cc:1428] T 4c4e8790282a49fd849a34ca11bef5bf P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260925 09:57:14.496888 11389 raft_consensus.cc:504] T 69f84509cafc44dd9444218738ebc2fe P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:14.502110 11085 tablet_bootstrap.cc:492] T 25db96e00e6646eaa1190b85c989958f P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:14.505115 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69f84509cafc44dd9444218738ebc2fe" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:14.505976 10665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67ce2e3325214aa6bdc3347e9e8f82b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:14.509433 10659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfa619b699004882a82976f0ef8204ad. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:14.513154 10621 leader_election.cc:341] T 69f84509cafc44dd9444218738ebc2fe P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:14.514453 11389 leader_election.cc:288] T 69f84509cafc44dd9444218738ebc2fe P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:14.502220 10983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fa527403d98425ea5cff9215d3a5e7f" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:14.518196 10983 raft_consensus.cc:2379] T 3fa527403d98425ea5cff9215d3a5e7f P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c43461c4ee854ae1ac52b873ccada318 in current term 1: Already voted for candidate b8850818e2ee42a082802d0095c08d21 in this term.
I20260925 09:57:14.489298 11084 raft_consensus.cc:348] T ef6fefaa76d44fe798201aab2f21b9c7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:14.521646 11085 tablet_bootstrap.cc:654] T 25db96e00e6646eaa1190b85c989958f P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:14.525825 11082 tablet_bootstrap.cc:492] T c16e096038084b2b9148390b6e773294 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:14.540571 11085 tablet_bootstrap.cc:492] T 25db96e00e6646eaa1190b85c989958f P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:14.540999 11085 ts_tablet_manager.cc:1397] T 25db96e00e6646eaa1190b85c989958f P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.039s user 0.007s sys 0.007s
I20260925 09:57:14.541391 11084 raft_consensus.cc:374] T ef6fefaa76d44fe798201aab2f21b9c7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:14.543881 11085 raft_consensus.cc:348] T 25db96e00e6646eaa1190b85c989958f P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:14.544480 11085 raft_consensus.cc:374] T 25db96e00e6646eaa1190b85c989958f P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:14.544713 11085 raft_consensus.cc:729] T 25db96e00e6646eaa1190b85c989958f P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:14.543262 11389 raft_consensus.cc:482] T fd73d0d52350444499abae34e8c02fab P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:14.545398 11085 consensus_queue.cc:260] T 25db96e00e6646eaa1190b85c989958f P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:14.545305 11389 raft_consensus.cc:504] T fd73d0d52350444499abae34e8c02fab P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:14.547520 11389 leader_election.cc:288] T fd73d0d52350444499abae34e8c02fab P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:14.548731 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd73d0d52350444499abae34e8c02fab" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:14.550031 10621 leader_election.cc:341] T fd73d0d52350444499abae34e8c02fab P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:14.550354 11085 ts_tablet_manager.cc:1428] T 25db96e00e6646eaa1190b85c989958f P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.009s user 0.004s sys 0.001s
I20260925 09:57:14.542789 10963 tablet_service.cc:1467] Processing CreateTablet for tablet 197f61525b0c4f9da441d98701d231a3 (DEFAULT_TABLE table=default.Table_1299538965b945cf859dbbd0cb7003f1 [id=cdc838b49cbb40778c9f440e9f83e96d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:14.568415 10963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 197f61525b0c4f9da441d98701d231a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:14.569286 11085 tablet_bootstrap.cc:492] T fd73d0d52350444499abae34e8c02fab P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:14.574549 11085 tablet_bootstrap.cc:654] T fd73d0d52350444499abae34e8c02fab P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:14.580549 11084 raft_consensus.cc:729] T ef6fefaa76d44fe798201aab2f21b9c7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:14.592206 10962 tablet_service.cc:1467] Processing CreateTablet for tablet c8522c08bdd94806b5c84f5a41ca5bf4 (DEFAULT_TABLE table=default.Table_ae6bc3dfc98244fda3409f53da244f7a [id=ac3973a2d08d45838cfa059e39fa5f8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:14.594911 10962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8522c08bdd94806b5c84f5a41ca5bf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:14.594637 10959 tablet_service.cc:1467] Processing CreateTablet for tablet b555eeba505740988c704df5c734e7c0 (DEFAULT_TABLE table=default.Table_055cf78170f24a8e94074c1f02e2f5a3 [id=3e32026d175b4b92a3ee036f17e54b9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:14.596100 10959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b555eeba505740988c704df5c734e7c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:14.596698 11389 raft_consensus.cc:482] T 4c4e8790282a49fd849a34ca11bef5bf P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:14.597251 11389 raft_consensus.cc:504] T 4c4e8790282a49fd849a34ca11bef5bf P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:14.600548 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c4e8790282a49fd849a34ca11bef5bf" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:14.601503 10831 raft_consensus.cc:2454] T 4c4e8790282a49fd849a34ca11bef5bf P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:14.602667 10621 leader_election.cc:302] T 4c4e8790282a49fd849a34ca11bef5bf P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:14.603631 11320 raft_consensus.cc:2781] T 4c4e8790282a49fd849a34ca11bef5bf P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:14.604015 11320 raft_consensus.cc:482] T 4c4e8790282a49fd849a34ca11bef5bf P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:14.604346 11320 raft_consensus.cc:3037] T 4c4e8790282a49fd849a34ca11bef5bf P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:14.618674 11085 tablet_bootstrap.cc:492] T fd73d0d52350444499abae34e8c02fab P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:14.619094 11085 ts_tablet_manager.cc:1397] T fd73d0d52350444499abae34e8c02fab P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.050s user 0.015s sys 0.017s
I20260925 09:57:14.622010 11320 raft_consensus.cc:504] T 4c4e8790282a49fd849a34ca11bef5bf P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:14.624029 11085 raft_consensus.cc:348] T fd73d0d52350444499abae34e8c02fab P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:14.624966 11085 raft_consensus.cc:374] T fd73d0d52350444499abae34e8c02fab P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:14.625298 11085 raft_consensus.cc:729] T fd73d0d52350444499abae34e8c02fab P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:14.625524 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c4e8790282a49fd849a34ca11bef5bf" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:14.626077 10831 raft_consensus.cc:3037] T 4c4e8790282a49fd849a34ca11bef5bf P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:14.622918 11389 leader_election.cc:288] T 4c4e8790282a49fd849a34ca11bef5bf P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:14.631810 11085 consensus_queue.cc:260] T fd73d0d52350444499abae34e8c02fab P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:14.637456 11085 ts_tablet_manager.cc:1428] T fd73d0d52350444499abae34e8c02fab P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.018s user 0.007s sys 0.000s
I20260925 09:57:14.638316 11085 tablet_bootstrap.cc:492] T c07c6205d1284a6fbb369c010a226da7 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:14.633193 11084 consensus_queue.cc:260] T ef6fefaa76d44fe798201aab2f21b9c7 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:14.660224 11085 tablet_bootstrap.cc:654] T c07c6205d1284a6fbb369c010a226da7 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:14.660224 11320 leader_election.cc:288] T 4c4e8790282a49fd849a34ca11bef5bf P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:14.669198 10959 tablet_service.cc:1467] Processing CreateTablet for tablet cfa619b699004882a82976f0ef8204ad (DEFAULT_TABLE table=default.Table_7e0155a70cc54485b8fdd4d936460d1c [id=c90c671ed4f043c2b56a1f84ff9a42cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:14.669840 10772 leader_election.cc:302] T 3fa527403d98425ea5cff9215d3a5e7f P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:14.670550 10959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfa619b699004882a82976f0ef8204ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:14.671095 11361 raft_consensus.cc:3037] T 3fa527403d98425ea5cff9215d3a5e7f P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:14.672328 10960 tablet_service.cc:1467] Processing CreateTablet for tablet 67ce2e3325214aa6bdc3347e9e8f82b0 (DEFAULT_TABLE table=default.Table_5157be3699be4fa4ac153a69f6010ecc [id=9cd1024c9d784df9af5f8239e39d1be7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:14.675915 10960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67ce2e3325214aa6bdc3347e9e8f82b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:14.690793 11389 raft_consensus.cc:482] T c07c6205d1284a6fbb369c010a226da7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:14.691354 11389 raft_consensus.cc:504] T c07c6205d1284a6fbb369c010a226da7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:14.694391 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c07c6205d1284a6fbb369c010a226da7" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:14.695852 10621 leader_election.cc:341] T c07c6205d1284a6fbb369c010a226da7 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:14.694814 11389 leader_election.cc:288] T c07c6205d1284a6fbb369c010a226da7 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:14.704900 11245 ts_tablet_manager.cc:1918] T 8b281fc20e6f42239fee80d05da88ab6 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:14.705307 11245 log.cc:1196] T 8b281fc20e6f42239fee80d05da88ab6 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/8b281fc20e6f42239fee80d05da88ab6
I20260925 09:57:14.706132 11245 ts_tablet_manager.cc:1939] T 8b281fc20e6f42239fee80d05da88ab6 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:14.719449 10831 raft_consensus.cc:2454] T 4c4e8790282a49fd849a34ca11bef5bf P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:14.721180 10621 leader_election.cc:302] T 4c4e8790282a49fd849a34ca11bef5bf P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:14.722923 11416 raft_consensus.cc:482] T ef22f64036684dce9d7c69c37722f468 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:14.723610 11416 raft_consensus.cc:504] T ef22f64036684dce9d7c69c37722f468 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:14.728740 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef22f64036684dce9d7c69c37722f468" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:14.728843 11454 tablet.cc:1722] T 1969ac471b6f471ebe8e048a96c22837 P c43461c4ee854ae1ac52b873ccada318: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20260925 09:57:14.730129 11416 leader_election.cc:288] T ef22f64036684dce9d7c69c37722f468 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
W20260925 09:57:14.731307 10918 leader_election.cc:341] T ef22f64036684dce9d7c69c37722f468 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:14.722680 11320 raft_consensus.cc:2781] T 4c4e8790282a49fd849a34ca11bef5bf P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:14.735831 11320 raft_consensus.cc:686] T 4c4e8790282a49fd849a34ca11bef5bf P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:14.737042 11320 consensus_queue.cc:237] T 4c4e8790282a49fd849a34ca11bef5bf P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:14.755971 11455 tablet.cc:1722] T 1969ac471b6f471ebe8e048a96c22837 P b8850818e2ee42a082802d0095c08d21: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20260925 09:57:14.758805 11361 raft_consensus.cc:2726] T 3fa527403d98425ea5cff9215d3a5e7f P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:14.760663 11388 raft_consensus.cc:482] T 833fac37485141f6a18104eaea907b0e P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:14.773070 11388 raft_consensus.cc:504] T 833fac37485141f6a18104eaea907b0e P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:14.775194 11388 leader_election.cc:288] T 833fac37485141f6a18104eaea907b0e P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:14.742002 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef22f64036684dce9d7c69c37722f468" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:14.777132 10919 leader_election.cc:341] T ef22f64036684dce9d7c69c37722f468 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:14.777585 10919 leader_election.cc:302] T ef22f64036684dce9d7c69c37722f468 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:14.778841 11416 raft_consensus.cc:2726] T ef22f64036684dce9d7c69c37722f468 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:14.779951 10380 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 26fae3fbfe374f348a1729c21ec751b9 (table default.Table_0011387b45014de4b24b466f4cb907f7 [id=e3f7aa5e988c4b6c98be5cd3d465245d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 26fae3fbfe374f348a1729c21ec751b9 consensus state has no leader
I20260925 09:57:14.779954 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "833fac37485141f6a18104eaea907b0e" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:14.781523 10621 leader_election.cc:341] T 833fac37485141f6a18104eaea907b0e P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:14.783246 11084 ts_tablet_manager.cc:1428] T ef6fefaa76d44fe798201aab2f21b9c7 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.297s user 0.005s sys 0.003s
W20260925 09:57:14.795507 10379 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.60:36985, user_credentials={real_user=slave}} blocked reactor thread for 234049us
I20260925 09:57:14.818740 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69f84509cafc44dd9444218738ebc2fe" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:14.819284 10984 raft_consensus.cc:3037] T 69f84509cafc44dd9444218738ebc2fe P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:14.820202 10983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd73d0d52350444499abae34e8c02fab" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:14.820765 10983 raft_consensus.cc:2454] T fd73d0d52350444499abae34e8c02fab P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:14.828588 11388 raft_consensus.cc:482] T 47a9fb9392244e1d89f8af2bfb42edd6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:14.829043 11388 raft_consensus.cc:504] T 47a9fb9392244e1d89f8af2bfb42edd6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:14.832329 11388 leader_election.cc:288] T 47a9fb9392244e1d89f8af2bfb42edd6 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:14.834117 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47a9fb9392244e1d89f8af2bfb42edd6" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:14.835543 10621 leader_election.cc:341] T 47a9fb9392244e1d89f8af2bfb42edd6 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:14.838629 10984 raft_consensus.cc:2454] T 69f84509cafc44dd9444218738ebc2fe P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:14.846417 10392 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 752105c1514f4c1db2a11b95b413b4f6
I20260925 09:57:14.852813 10666 tablet_service.cc:1514] Processing DeleteTablet for tablet 752105c1514f4c1db2a11b95b413b4f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:14 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:14.862211 11416 raft_consensus.cc:482] T 3c31682bf400494f82ee3ce5e6eec441 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:14.862625 10385 master_service.cc:946] ReplaceTablet: received request to replace tablet a0976e0c01234b38a7ab934dc1b289c4 from {username='slave'} at 127.0.0.1:35140
I20260925 09:57:14.862957 10390 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 91c09118b43b41eca8f7040323ba76f7 (table default.Table_33b40a0d6ca44f15be47ca34bab6a093 [id=453beec106a64428b163c7db8132fc21]). Expected version 1 got 0
I20260925 09:57:14.862706 11416 raft_consensus.cc:504] T 3c31682bf400494f82ee3ce5e6eec441 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:14.865020 10390 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 1969ac471b6f471ebe8e048a96c22837 (table default.Table_76a153b441c24e029c2997b568a50df0 [id=f17a11d223204599afb24e2e9d1d9d9d]). Expected version 5 got 4
I20260925 09:57:14.866426 10390 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet c07c6205d1284a6fbb369c010a226da7 (table default.Table_5a6ecccec7f44c31bbfe8d64580ec33b [id=2d6bc0e2aaee4b1483b01fc678505bb0]). Expected version 1 got 0
I20260925 09:57:14.867116 10390 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 28d60531970a4ab58ce5ef99cb3a96bf (table default.Table_ce09d3a8d8e84973894f33b18bbfdb22 [id=6851500c7dba44f1bf353bb79bafd97a]). Expected version 2 got 0
I20260925 09:57:14.867465 11416 leader_election.cc:288] T 3c31682bf400494f82ee3ce5e6eec441 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:14.868775 10389 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 833fac37485141f6a18104eaea907b0e
I20260925 09:57:14.868765 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c31682bf400494f82ee3ce5e6eec441" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:14.870666 10659 tablet_service.cc:1514] Processing DeleteTablet for tablet 833fac37485141f6a18104eaea907b0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:14 UTC) from {username='slave'} at 127.0.0.1:37876
W20260925 09:57:14.873068 10919 leader_election.cc:341] T 3c31682bf400494f82ee3ce5e6eec441 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:14.879911 10393 catalog_manager.cc:5495] T 83b0cf21f2a144719a2e13990e1921c0 P c43461c4ee854ae1ac52b873ccada318 reported cstate change: term changed from 0 to 1, leader changed from <none> to c43461c4ee854ae1ac52b873ccada318 (127.10.9.2). New cstate: current_term: 1 leader_uuid: "c43461c4ee854ae1ac52b873ccada318" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:14.881688 11361 consensus_queue.cc:1035] T 0e882284a014455192375daad391efaf P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 09:57:14.884361 10379 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.62:34455, user_credentials={real_user=slave}} blocked reactor thread for 43906.9us
I20260925 09:57:14.884609 10393 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 1969ac471b6f471ebe8e048a96c22837 (table default.Table_76a153b441c24e029c2997b568a50df0 [id=f17a11d223204599afb24e2e9d1d9d9d]). Expected version 5 got 4
I20260925 09:57:14.885684 10818 tablet_service.cc:1514] Processing DeleteTablet for tablet 752105c1514f4c1db2a11b95b413b4f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:14 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:14.886214 10812 tablet_service.cc:1514] Processing DeleteTablet for tablet 833fac37485141f6a18104eaea907b0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:14 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:14.891079 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c31682bf400494f82ee3ce5e6eec441" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:14.892464 10918 leader_election.cc:341] T 3c31682bf400494f82ee3ce5e6eec441 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:14.892866 10918 leader_election.cc:302] T 3c31682bf400494f82ee3ce5e6eec441 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
W20260925 09:57:14.904139 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 752105c1514f4c1db2a11b95b413b4f6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 752105c1514f4c1db2a11b95b413b4f6 already in progress: creating tablet
I20260925 09:57:14.898162 11416 raft_consensus.cc:2726] T 3c31682bf400494f82ee3ce5e6eec441 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:14.904907 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 833fac37485141f6a18104eaea907b0e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 833fac37485141f6a18104eaea907b0e already in progress: creating tablet
I20260925 09:57:14.909955 11082 tablet_bootstrap.cc:654] T c16e096038084b2b9148390b6e773294 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:14.910013 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 752105c1514f4c1db2a11b95b413b4f6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 752105c1514f4c1db2a11b95b413b4f6 already in progress: creating tablet
I20260925 09:57:14.894716 11084 tablet_bootstrap.cc:492] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:14.914988 10389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_bc338e8ca7174647948301512850ce00"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:14.916646 10389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bc338e8ca7174647948301512850ce00 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:14.948485 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c07c6205d1284a6fbb369c010a226da7" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:14.949220 10984 raft_consensus.cc:2454] T c07c6205d1284a6fbb369c010a226da7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:14.910928 11458 tablet_replica.cc:330] T 833fac37485141f6a18104eaea907b0e P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:14.970297 10389 master_service.cc:946] ReplaceTablet: received request to replace tablet 0e882284a014455192375daad391efaf from {username='slave'} at 127.0.0.1:35140
I20260925 09:57:14.977361 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "833fac37485141f6a18104eaea907b0e" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:14.978083 10984 raft_consensus.cc:2454] T 833fac37485141f6a18104eaea907b0e P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:14.978058 10389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_8839d4ce47434a51a850fd37841e859a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:14.980723 10389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8839d4ce47434a51a850fd37841e859a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:14.992337 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47a9fb9392244e1d89f8af2bfb42edd6" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:14.993176 10984 raft_consensus.cc:2454] T 47a9fb9392244e1d89f8af2bfb42edd6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:14.998801 11085 tablet_bootstrap.cc:492] T c07c6205d1284a6fbb369c010a226da7 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:14.999337 11085 ts_tablet_manager.cc:1397] T c07c6205d1284a6fbb369c010a226da7 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.361s user 0.045s sys 0.045s
I20260925 09:57:15.001814 11458 raft_consensus.cc:2229] T 833fac37485141f6a18104eaea907b0e P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:15.003666 11463 raft_consensus.cc:482] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:15.002854 11458 raft_consensus.cc:2258] T 833fac37485141f6a18104eaea907b0e P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:15.004169 11463 raft_consensus.cc:504] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:15.006057 11458 ts_tablet_manager.cc:1905] T 833fac37485141f6a18104eaea907b0e P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:15.008692 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89a143a5623e4ef38027381a56c95227" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:15.009966 10918 leader_election.cc:341] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:15.015836 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89a143a5623e4ef38027381a56c95227" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:15.018072 10919 leader_election.cc:341] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:15.018477 10919 leader_election.cc:302] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:15.025775 11416 raft_consensus.cc:2726] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:15.027926 10389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_256aab1fb8b742d6b04709d930f59bce"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:15.029458 10389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_256aab1fb8b742d6b04709d930f59bce in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:15.044034 10385 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet a0976e0c01234b38a7ab934dc1b289c4
I20260925 09:57:15.045208 10385 catalog_manager.cc:6565] ReplaceTablet: tablet a0976e0c01234b38a7ab934dc1b289c4 deleted and replaced by tablet 2d087d3e35814c90a03eccd90eeea86c
I20260925 09:57:15.045868 11463 leader_election.cc:288] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:15.045934 10817 tablet_service.cc:1514] Processing DeleteTablet for tablet a0976e0c01234b38a7ab934dc1b289c4 with delete_type TABLET_DATA_DELETED (replaced by tablet 2d087d3e35814c90a03eccd90eeea86c) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:15.048534 11461 tablet_replica.cc:330] T a0976e0c01234b38a7ab934dc1b289c4 P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:15.049242 11461 raft_consensus.cc:2229] T a0976e0c01234b38a7ab934dc1b289c4 P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:15.049975 11461 raft_consensus.cc:2258] T a0976e0c01234b38a7ab934dc1b289c4 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:15.051975 11461 ts_tablet_manager.cc:1905] T a0976e0c01234b38a7ab934dc1b289c4 P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:15.053649 10385 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_33b40a0d6ca44f15be47ca34bab6a093" } modify_external_catalogs: true
I20260925 09:57:15.053483 11085 raft_consensus.cc:348] T c07c6205d1284a6fbb369c010a226da7 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:15.054201 11085 raft_consensus.cc:374] T c07c6205d1284a6fbb369c010a226da7 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:15.054116 10385 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_33b40a0d6ca44f15be47ca34bab6a093" } modify_external_catalogs: true
I20260925 09:57:15.054461 11085 raft_consensus.cc:729] T c07c6205d1284a6fbb369c010a226da7 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:15.058097 11361 consensus_queue.cc:1035] T 83b0cf21f2a144719a2e13990e1921c0 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 09:57:15.063318 11435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:57:15.064152 11435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:57:15.064839 11435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260925 09:57:15.067474 11084 tablet_bootstrap.cc:700] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P be97ab5d75934a648d4ab76341193ec0: Time spent opening tablet: real 0.112s user 0.004s sys 0.000s
I20260925 09:57:15.067634 10665 tablet_service.cc:1514] Processing DeleteTablet for tablet a0976e0c01234b38a7ab934dc1b289c4 with delete_type TABLET_DATA_DELETED (replaced by tablet 2d087d3e35814c90a03eccd90eeea86c) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:15.068099 11084 tablet_bootstrap.cc:654] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:15.072459 11471 tablet.cc:1722] T 1969ac471b6f471ebe8e048a96c22837 P be97ab5d75934a648d4ab76341193ec0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20260925 09:57:15.078019 10619 leader_election.cc:302] T fd73d0d52350444499abae34e8c02fab P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:15.079499 11320 raft_consensus.cc:2781] T fd73d0d52350444499abae34e8c02fab P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:15.079830 11320 raft_consensus.cc:482] T fd73d0d52350444499abae34e8c02fab P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:15.080139 11320 raft_consensus.cc:3037] T fd73d0d52350444499abae34e8c02fab P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:15.088907 11463 raft_consensus.cc:482] T ef6fefaa76d44fe798201aab2f21b9c7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:15.088843 10619 leader_election.cc:302] T 69f84509cafc44dd9444218738ebc2fe P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:15.089335 11463 raft_consensus.cc:504] T ef6fefaa76d44fe798201aab2f21b9c7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:15.089898 11388 raft_consensus.cc:2781] T 69f84509cafc44dd9444218738ebc2fe P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:15.090288 11388 raft_consensus.cc:686] T 69f84509cafc44dd9444218738ebc2fe P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:15.091193 11388 consensus_queue.cc:237] T 69f84509cafc44dd9444218738ebc2fe P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:15.092474 10683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef6fefaa76d44fe798201aab2f21b9c7" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:15.093940 10919 leader_election.cc:341] T ef6fefaa76d44fe798201aab2f21b9c7 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:15.095706 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef6fefaa76d44fe798201aab2f21b9c7" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:15.098141 10918 leader_election.cc:341] T ef6fefaa76d44fe798201aab2f21b9c7 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:15.098480 10918 leader_election.cc:302] T ef6fefaa76d44fe798201aab2f21b9c7 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:15.099531 11416 raft_consensus.cc:2726] T ef6fefaa76d44fe798201aab2f21b9c7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:15.087031 11320 raft_consensus.cc:504] T fd73d0d52350444499abae34e8c02fab P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:15.108732 10389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_f0ebee2143d14ff8b4f798e61ec58bdf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:15.110111 10389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f0ebee2143d14ff8b4f798e61ec58bdf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:15.111449 11082 tablet_bootstrap.cc:492] T c16e096038084b2b9148390b6e773294 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:15.091264 11463 leader_election.cc:288] T ef6fefaa76d44fe798201aab2f21b9c7 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:15.114132 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd73d0d52350444499abae34e8c02fab" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:15.114658 10831 raft_consensus.cc:3037] T fd73d0d52350444499abae34e8c02fab P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:15.124060 10831 raft_consensus.cc:2454] T fd73d0d52350444499abae34e8c02fab P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
W20260925 09:57:15.125061 10619 leader_election.cc:341] T 4c4e8790282a49fd849a34ca11bef5bf P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 4c4e8790282a49fd849a34ca11bef5bf
I20260925 09:57:15.131690 11082 ts_tablet_manager.cc:1397] T c16e096038084b2b9148390b6e773294 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.606s user 0.013s sys 0.000s
I20260925 09:57:15.132314 10389 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_4c02305911f84ebb95187deaf905c14c" } modify_external_catalogs: true
I20260925 09:57:15.132820 10389 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_4c02305911f84ebb95187deaf905c14c" } modify_external_catalogs: true
I20260925 09:57:15.134059 11082 raft_consensus.cc:348] T c16e096038084b2b9148390b6e773294 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:15.135689 11082 raft_consensus.cc:374] T c16e096038084b2b9148390b6e773294 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:15.135964 11082 raft_consensus.cc:729] T c16e096038084b2b9148390b6e773294 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
W20260925 09:57:15.136890 10619 leader_election.cc:341] T 4c4e8790282a49fd849a34ca11bef5bf P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 4c4e8790282a49fd849a34ca11bef5bf
I20260925 09:57:15.138384 11320 leader_election.cc:288] T fd73d0d52350444499abae34e8c02fab P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:15.141220 10621 leader_election.cc:302] T fd73d0d52350444499abae34e8c02fab P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:15.146052 11389 raft_consensus.cc:2781] T fd73d0d52350444499abae34e8c02fab P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:15.154088 10619 leader_election.cc:302] T c07c6205d1284a6fbb369c010a226da7 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:15.155447 11388 raft_consensus.cc:2781] T c07c6205d1284a6fbb369c010a226da7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:15.155917 11388 raft_consensus.cc:482] T c07c6205d1284a6fbb369c010a226da7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:15.156543 11388 raft_consensus.cc:3037] T c07c6205d1284a6fbb369c010a226da7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:15.164954 11084 tablet_bootstrap.cc:492] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:15.165357 11084 ts_tablet_manager.cc:1397] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.271s user 0.011s sys 0.000s
W20260925 09:57:15.165499 11435 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:57:15.165889 11435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:57:15.166255 11435 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:57:15.166574 11435 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:57:15.167761 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 8b281fc20e6f42239fee80d05da88ab6 (table default.Table_8dfe0230298f4ff9b7e34161a9e90f53 [id=50788524d4b24e429c978c44428dc892]) successfully deleted
I20260925 09:57:15.149983 11085 consensus_queue.cc:260] T c07c6205d1284a6fbb369c010a226da7 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:15.174116 11085 ts_tablet_manager.cc:1428] T c07c6205d1284a6fbb369c010a226da7 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.123s user 0.005s sys 0.000s
I20260925 09:57:15.174891 11085 tablet_bootstrap.cc:492] T 833fac37485141f6a18104eaea907b0e P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:15.170926 11084 raft_consensus.cc:348] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:15.179111 11084 raft_consensus.cc:374] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:15.180241 11084 raft_consensus.cc:729] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:15.185248 11084 consensus_queue.cc:260] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:15.189855 10619 leader_election.cc:302] T 833fac37485141f6a18104eaea907b0e P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
W20260925 09:57:15.190577 10619 raft_consensus.cc:2670] T 833fac37485141f6a18104eaea907b0e P b8850818e2ee42a082802d0095c08d21: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260925 09:57:15.207688 10619 leader_election.cc:302] T 47a9fb9392244e1d89f8af2bfb42edd6 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:15.208221 11084 ts_tablet_manager.cc:1428] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.043s user 0.007s sys 0.000s
I20260925 09:57:15.208453 10385 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 91c09118b43b41eca8f7040323ba76f7
I20260925 09:57:15.209065 11320 raft_consensus.cc:2781] T 47a9fb9392244e1d89f8af2bfb42edd6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:15.209532 11320 raft_consensus.cc:482] T 47a9fb9392244e1d89f8af2bfb42edd6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:15.209972 11320 raft_consensus.cc:3037] T 47a9fb9392244e1d89f8af2bfb42edd6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:15.210389 10665 tablet_service.cc:1514] Processing DeleteTablet for tablet 91c09118b43b41eca8f7040323ba76f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:15 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:15.219350 10817 tablet_service.cc:1514] Processing DeleteTablet for tablet 91c09118b43b41eca8f7040323ba76f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:15 UTC) from {username='slave'} at 127.0.0.1:33344
W20260925 09:57:15.220602 10772 consensus_peers.cc:487] T 0e882284a014455192375daad391efaf P c43461c4ee854ae1ac52b873ccada318 -> Peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Couldn't send request to peer be97ab5d75934a648d4ab76341193ec0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e882284a014455192375daad391efaf. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:15.223637 10962 tablet_service.cc:1514] Processing DeleteTablet for tablet 752105c1514f4c1db2a11b95b413b4f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:14 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:15.224016 11389 raft_consensus.cc:686] T fd73d0d52350444499abae34e8c02fab P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:15.224936 11389 consensus_queue.cc:237] T fd73d0d52350444499abae34e8c02fab P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:15.225699 10392 catalog_manager.cc:5495] T 25db96e00e6646eaa1190b85c989958f P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:15.227905 11084 tablet_bootstrap.cc:492] T 752105c1514f4c1db2a11b95b413b4f6 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:15.212538 10385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_32e7bdf7eaf147ea90740b6b3cfb2fd3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:15.227368 10392 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 1969ac471b6f471ebe8e048a96c22837 (table default.Table_76a153b441c24e029c2997b568a50df0 [id=f17a11d223204599afb24e2e9d1d9d9d]). Expected version 5 got 4
W20260925 09:57:15.231266 10385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_32e7bdf7eaf147ea90740b6b3cfb2fd3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:15.233203 10392 catalog_manager.cc:5495] T 3fa527403d98425ea5cff9215d3a5e7f P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:15.239634 10817 tablet_service.cc:1467] Processing CreateTablet for tablet ec1b78e61e8d46cb8b87f60786059658 (DEFAULT_TABLE table=default.Table_09fe8f366734488c93a79eeeec2c96f9 [id=a7023f3c32884ce8a7ca21fc95afc721]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:15.239662 11082 consensus_queue.cc:260] T c16e096038084b2b9148390b6e773294 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:15.240820 11084 tablet_bootstrap.cc:654] T 752105c1514f4c1db2a11b95b413b4f6 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:15.241189 11435 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.9.20:38279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-2/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.10.9.62:34455,127.10.9.61:33047,127.10.9.60:36985
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.9.60:36985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.9.60
--webserver_port=41643
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:06:08 UTC on ccycloud.quasar-lntfnh.root.comops.site
build id 4672078
TSAN enabled
I20260925 09:57:15.242127 11082 ts_tablet_manager.cc:1428] T c16e096038084b2b9148390b6e773294 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.110s user 0.005s sys 0.000s
I20260925 09:57:15.242929 11082 tablet_bootstrap.cc:492] T 28d60531970a4ab58ce5ef99cb3a96bf P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:15.242861 11435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:57:15.245051 11435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 09:57:15.250669 11085 tablet_bootstrap.cc:654] T 833fac37485141f6a18104eaea907b0e P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:15.258453 11388 raft_consensus.cc:504] T c07c6205d1284a6fbb369c010a226da7 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:15.260022 10962 tablet_service.cc:1514] Processing DeleteTablet for tablet 833fac37485141f6a18104eaea907b0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:14 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:15.260663 11245 tablet_replica.cc:330] T 833fac37485141f6a18104eaea907b0e P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:15.261365 11245 raft_consensus.cc:2229] T 833fac37485141f6a18104eaea907b0e P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:15.261787 11245 raft_consensus.cc:2258] T 833fac37485141f6a18104eaea907b0e P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:15.263899 11245 ts_tablet_manager.cc:1905] T 833fac37485141f6a18104eaea907b0e P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:15.265159 10817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec1b78e61e8d46cb8b87f60786059658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:15.274420 10962 tablet_service.cc:1514] Processing DeleteTablet for tablet a0976e0c01234b38a7ab934dc1b289c4 with delete_type TABLET_DATA_DELETED (replaced by tablet 2d087d3e35814c90a03eccd90eeea86c) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:15.276721 10962 tablet_service.cc:1514] Processing DeleteTablet for tablet 91c09118b43b41eca8f7040323ba76f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:15 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:15.278064 10665 tablet_service.cc:1467] Processing CreateTablet for tablet ec1b78e61e8d46cb8b87f60786059658 (DEFAULT_TABLE table=default.Table_09fe8f366734488c93a79eeeec2c96f9 [id=a7023f3c32884ce8a7ca21fc95afc721]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:15.280032 10665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec1b78e61e8d46cb8b87f60786059658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:15.280442 10983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd73d0d52350444499abae34e8c02fab" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:15.281066 10983 raft_consensus.cc:3037] T fd73d0d52350444499abae34e8c02fab P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:15.287963 10392 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet c07c6205d1284a6fbb369c010a226da7 (table default.Table_5a6ecccec7f44c31bbfe8d64580ec33b [id=2d6bc0e2aaee4b1483b01fc678505bb0]). Expected version 1 got 0
I20260925 09:57:15.290124 10387 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:35140 (request call id 138) took 1466 ms. Trace:
I20260925 09:57:15.291983 11084 tablet_bootstrap.cc:492] T 752105c1514f4c1db2a11b95b413b4f6 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:15.292371 11084 ts_tablet_manager.cc:1397] T 752105c1514f4c1db2a11b95b413b4f6 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.065s user 0.010s sys 0.000s
I20260925 09:57:15.295410 11084 raft_consensus.cc:348] T 752105c1514f4c1db2a11b95b413b4f6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:15.295995 11084 raft_consensus.cc:374] T 752105c1514f4c1db2a11b95b413b4f6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:15.296226 11084 raft_consensus.cc:729] T 752105c1514f4c1db2a11b95b413b4f6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:15.291960 10983 raft_consensus.cc:2454] T fd73d0d52350444499abae34e8c02fab P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:15.296777 11084 consensus_queue.cc:260] T 752105c1514f4c1db2a11b95b413b4f6 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:15.298979 10377 catalog_manager.cc:4821] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 752105c1514f4c1db2a11b95b413b4f6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 752105c1514f4c1db2a11b95b413b4f6 already in progress: creating tablet
I20260925 09:57:15.300208 11389 consensus_queue.cc:1035] T 25db96e00e6646eaa1190b85c989958f P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:15.302588 11389 consensus_queue.cc:1035] T 3fa527403d98425ea5cff9215d3a5e7f P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:15.290524 10387 rpcz_store.cc:276] 0925 09:57:13.824075 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:13.828327 (+ 4252us) service_pool.cc:227] Handling call
0925 09:57:13.836487 (+ 8160us) spinlock_profiling.cc:243] Waited 6.85 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2adb7d6 00007fabf0621634 00007fabf06215c2 00007fabf06214f1 00007fabf062146c 00007fabf0620722 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:13.837409 (+ 922us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:57:13.838161 (+ 752us) catalog_manager.cc:3616] Apply alter schema
0925 09:57:13.838920 (+ 759us) catalog_manager.cc:3639] Acquired catalog manager lock
0925 09:57:13.839139 (+ 219us) catalog_manager.cc:3786] Updating metadata on disk
0925 09:57:14.107276 (+268137us) catalog_manager.cc:3817] Committing alterations to in-memory state
0925 09:57:15.290019 (+1182743us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:13.901656 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:13.902052 (+ 396us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:13.902073 (+ 21us) write_op.cc:435] Acquired schema lock
0925 09:57:13.902086 (+ 13us) tablet.cc:660] Decoding operations
0925 09:57:13.902348 (+ 262us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:13.902383 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:13.902407 (+ 24us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:13.902554 (+ 147us) tablet.cc:699] Row locks acquired
0925 09:57:13.902568 (+ 14us) write_op.cc:260] PREPARE: finished
0925 09:57:13.902678 (+ 110us) write_op.cc:270] Start()
0925 09:57:13.902756 (+ 78us) write_op.cc:276] Timestamp: P: 1790330233902565 usec, L: 0
0925 09:57:13.902773 (+ 17us) op_driver.cc:331] REPLICATION: starting
0925 09:57:13.903220 (+ 447us) log.cc:842] Serialized 405 byte log entry
0925 09:57:14.093622 (+190402us) op_driver.cc:444] REPLICATION: finished
0925 09:57:14.105851 (+ 12229us) write_op.cc:301] APPLY: starting
0925 09:57:14.105908 (+ 57us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:14.106005 (+ 97us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:14.106109 (+ 104us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:14.106268 (+ 159us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:14.106394 (+ 126us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:14.106439 (+ 45us) write_op.cc:312] APPLY: finished
0925 09:57:14.106604 (+ 165us) log.cc:842] Serialized 23 byte log entry
0925 09:57:14.106749 (+ 145us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:14.106804 (+ 55us) write_op.cc:454] Released schema lock
0925 09:57:14.106832 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:14.106993 (+ 161us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:14.107015 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":15764736,"child_traces":[["op",{"apply.queue_time_us":12083,"apply.run_cpu_time_us":1306,"apply.run_wall_time_us":1307,"num_ops":1,"prepare.queue_time_us":34756,"prepare.run_cpu_time_us":1864,"prepare.run_wall_time_us":151486,"raft.queue_time_us":2122,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":186,"replication_time_us":190782}]]}
I20260925 09:57:15.311440 11084 ts_tablet_manager.cc:1428] T 752105c1514f4c1db2a11b95b413b4f6 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.019s user 0.006s sys 0.000s
W20260925 09:57:15.311719 10388 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.3:41469 (request call id 14) took 2760 ms (client timeout 1000 ms). Trace:
W20260925 09:57:15.322305 10388 rpcz_store.cc:269] 0925 09:57:12.550698 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:12.550850 (+ 152us) service_pool.cc:227] Handling call
0925 09:57:15.311445 (+2760595us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":23}
I20260925 09:57:15.324402 10388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_591aa6b2533844f1904ae4982a7f01ba"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:15.325229 11084 tablet_bootstrap.cc:492] T 42ec275c905343c4be1b72dc2bd0f6e3 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:15.329919 11084 tablet_bootstrap.cc:654] T 42ec275c905343c4be1b72dc2bd0f6e3 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:15.330303 10962 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d661d5000894c45ac3d1c27f0e72889 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:12 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:15.328040 10387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_95a4c3e1f6014e63811447ba4a0e34cf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:15.331230 10962 tablet_service.cc:1514] Processing DeleteTablet for tablet b7646a1cde1b45b9bf7928c3e13bcbd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:12 UTC) from {username='slave'} at 127.0.0.1:53560
W20260925 09:57:15.334375 10387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_95a4c3e1f6014e63811447ba4a0e34cf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:15.350684 11388 leader_election.cc:288] T c07c6205d1284a6fbb369c010a226da7 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:15.336364 11458 ts_tablet_manager.cc:1918] T 833fac37485141f6a18104eaea907b0e P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:15.356843 11320 raft_consensus.cc:504] T 47a9fb9392244e1d89f8af2bfb42edd6 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:15.351346 11458 log.cc:1196] T 833fac37485141f6a18104eaea907b0e P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/833fac37485141f6a18104eaea907b0e
I20260925 09:57:15.360847 11458 ts_tablet_manager.cc:1939] T 833fac37485141f6a18104eaea907b0e P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:15.363029 11458 tablet_replica.cc:330] T a0976e0c01234b38a7ab934dc1b289c4 P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:15.374341 11361 consensus_queue.cc:1035] T 83b0cf21f2a144719a2e13990e1921c0 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:15.379614 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c07c6205d1284a6fbb369c010a226da7" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:15.380241 10835 raft_consensus.cc:3037] T c07c6205d1284a6fbb369c010a226da7 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:15.383777 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47a9fb9392244e1d89f8af2bfb42edd6" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:15.386633 11389 raft_consensus.cc:482] T c16e096038084b2b9148390b6e773294 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:15.387123 11389 raft_consensus.cc:504] T c16e096038084b2b9148390b6e773294 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:15.389089 11389 leader_election.cc:288] T c16e096038084b2b9148390b6e773294 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:15.389572 11389 consensus_queue.cc:1035] T 3fa527403d98425ea5cff9215d3a5e7f P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:15.396656 10836 raft_consensus.cc:3037] T 25db96e00e6646eaa1190b85c989958f P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:57:15.332665 10388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_591aa6b2533844f1904ae4982a7f01ba in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:15.357450 10389 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet fd73d0d52350444499abae34e8c02fab
I20260925 09:57:15.362375 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 833fac37485141f6a18104eaea907b0e (table default.Table_8e0c5caa8df14066af83c49a50a172bf [id=fc3b3c18f00c42559612468a21e96694]) successfully deleted
I20260925 09:57:15.362773 10384 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: default.Table_8705f75ef7b2448ab645f1d25e3c2637 [id=aed8a2ddc51545ad9cec4d272dcf3daf] alter complete (version 1)
I20260925 09:57:15.370469 10391 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:35140 (request call id 145) took 1103 ms. Trace:
W20260925 09:57:15.406860 10621 leader_election.cc:341] T 47a9fb9392244e1d89f8af2bfb42edd6 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:15.409549 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c16e096038084b2b9148390b6e773294" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:15.410986 10816 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e882284a014455192375daad391efaf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:09 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:15.412343 10816 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d661d5000894c45ac3d1c27f0e72889 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:12 UTC) from {username='slave'} at 127.0.0.1:33344
W20260925 09:57:15.380384 10393 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.2:40761 (request call id 22) took 936 ms (client timeout 1000 ms). Trace:
I20260925 09:57:15.435176 10815 tablet_service.cc:1514] Processing DeleteTablet for tablet b7646a1cde1b45b9bf7928c3e13bcbd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:12 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:15.435521 10814 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b281fc20e6f42239fee80d05da88ab6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:13 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:15.436615 10816 tablet_service.cc:1514] Processing DeleteTablet for tablet e48d86c20e9e4db990c23359ff1d1a6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:11 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:15.445039 10835 raft_consensus.cc:2454] T c07c6205d1284a6fbb369c010a226da7 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
W20260925 09:57:15.419641 10393 rpcz_store.cc:269] 0925 09:57:14.444113 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:14.808284 (+364171us) service_pool.cc:227] Handling call
0925 09:57:15.380272 (+571988us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:14.938118 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:14.938638 (+ 520us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:14.938660 (+ 22us) write_op.cc:435] Acquired schema lock
0925 09:57:14.938674 (+ 14us) tablet.cc:660] Decoding operations
0925 09:57:14.938917 (+ 243us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:14.938952 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:14.938972 (+ 20us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:14.939106 (+ 134us) tablet.cc:699] Row locks acquired
0925 09:57:14.939130 (+ 24us) write_op.cc:260] PREPARE: finished
0925 09:57:14.939240 (+ 110us) write_op.cc:270] Start()
0925 09:57:14.939316 (+ 76us) write_op.cc:276] Timestamp: P: 1790330234939182 usec, L: 0
0925 09:57:14.939333 (+ 17us) op_driver.cc:331] REPLICATION: starting
0925 09:57:14.939952 (+ 619us) log.cc:842] Serialized 524 byte log entry
0925 09:57:14.962803 (+ 22851us) op_driver.cc:444] REPLICATION: finished
0925 09:57:14.965290 (+ 2487us) write_op.cc:301] APPLY: starting
0925 09:57:14.965342 (+ 52us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:14.965441 (+ 99us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:14.965542 (+ 101us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:14.965734 (+ 192us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:14.965872 (+ 138us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:14.965923 (+ 51us) write_op.cc:312] APPLY: finished
0925 09:57:14.966090 (+ 167us) log.cc:842] Serialized 23 byte log entry
0925 09:57:14.966225 (+ 135us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:14.966288 (+ 63us) write_op.cc:454] Released schema lock
0925 09:57:14.966322 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:14.966502 (+ 180us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:14.966527 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":29401,"reported_tablets":12,"child_traces":[["op",{"apply.queue_time_us":2345,"apply.run_cpu_time_us":1386,"apply.run_wall_time_us":1385,"num_ops":1,"prepare.queue_time_us":9112,"prepare.run_cpu_time_us":2175,"prepare.run_wall_time_us":2176,"raft.queue_time_us":674,"raft.run_cpu_time_us":15,"raft.run_wall_time_us":15,"replication_time_us":23407}]]}
I20260925 09:57:15.447575 10659 tablet_service.cc:1514] Processing DeleteTablet for tablet fd73d0d52350444499abae34e8c02fab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:15 UTC) from {username='slave'} at 127.0.0.1:37876
W20260925 09:57:15.448704 10621 leader_election.cc:341] T c16e096038084b2b9148390b6e773294 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:15.451043 10621 leader_election.cc:302] T c07c6205d1284a6fbb369c010a226da7 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:15.451882 11475 raft_consensus.cc:2781] T c07c6205d1284a6fbb369c010a226da7 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:15.452399 11475 raft_consensus.cc:686] T c07c6205d1284a6fbb369c010a226da7 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:15.453302 11475 consensus_queue.cc:237] T c07c6205d1284a6fbb369c010a226da7 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:15.460224 10659 tablet_service.cc:1467] Processing CreateTablet for tablet 3706253e53d841aeaa43889ead13af32 (DEFAULT_TABLE table=default.Table_bbc3c50c06634d119ae96dcf649f44e7 [id=0260f0e6cdd6431db06788bde54ec38c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:15.461326 10659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3706253e53d841aeaa43889ead13af32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:15.477901 11389 consensus_queue.cc:1035] T 25db96e00e6646eaa1190b85c989958f P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:15.504612 11461 ts_tablet_manager.cc:1918] T a0976e0c01234b38a7ab934dc1b289c4 P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:15.505113 11461 log.cc:1196] T a0976e0c01234b38a7ab934dc1b289c4 P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/a0976e0c01234b38a7ab934dc1b289c4
I20260925 09:57:15.506083 11461 ts_tablet_manager.cc:1939] T a0976e0c01234b38a7ab934dc1b289c4 P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:15.516737 11082 tablet_bootstrap.cc:654] T 28d60531970a4ab58ce5ef99cb3a96bf P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:15.528472 11458 raft_consensus.cc:2229] T a0976e0c01234b38a7ab934dc1b289c4 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:15.529201 11458 raft_consensus.cc:2258] T a0976e0c01234b38a7ab934dc1b289c4 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:15.533372 10666 tablet_service.cc:1467] Processing CreateTablet for tablet 93e1b5837aff48d7907890c40dfcd0d0 (DEFAULT_TABLE table=default.Table_5adf3f6c195e490fa4c978d7e1b4c187 [id=23d14ed1d2d647fc8b3b5ef40c7dc0bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:15.534621 11458 ts_tablet_manager.cc:1905] T a0976e0c01234b38a7ab934dc1b289c4 P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:15.534627 10666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93e1b5837aff48d7907890c40dfcd0d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:15.544239 11320 leader_election.cc:288] T 47a9fb9392244e1d89f8af2bfb42edd6 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:15.355365 11245 ts_tablet_manager.cc:1918] T 833fac37485141f6a18104eaea907b0e P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:15.493597 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c07c6205d1284a6fbb369c010a226da7" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:15.554507 11245 log.cc:1196] T 833fac37485141f6a18104eaea907b0e P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/833fac37485141f6a18104eaea907b0e
I20260925 09:57:15.554562 10984 raft_consensus.cc:3037] T c07c6205d1284a6fbb369c010a226da7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:15.555179 11245 ts_tablet_manager.cc:1939] T 833fac37485141f6a18104eaea907b0e P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:15.494349 10981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47a9fb9392244e1d89f8af2bfb42edd6" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:15.556150 10981 raft_consensus.cc:3037] T 47a9fb9392244e1d89f8af2bfb42edd6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:15.557391 11245 tablet_replica.cc:330] T a0976e0c01234b38a7ab934dc1b289c4 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:15.558086 11245 raft_consensus.cc:2229] T a0976e0c01234b38a7ab934dc1b289c4 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:15.558495 11245 raft_consensus.cc:2258] T a0976e0c01234b38a7ab934dc1b289c4 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:15.545725 10962 tablet_service.cc:1467] Processing CreateTablet for tablet ec1b78e61e8d46cb8b87f60786059658 (DEFAULT_TABLE table=default.Table_09fe8f366734488c93a79eeeec2c96f9 [id=a7023f3c32884ce8a7ca21fc95afc721]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:15.560369 10962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec1b78e61e8d46cb8b87f60786059658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:15.563628 10958 tablet_service.cc:1514] Processing DeleteTablet for tablet fd73d0d52350444499abae34e8c02fab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:15 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:15.356248 11084 tablet_bootstrap.cc:492] T 42ec275c905343c4be1b72dc2bd0f6e3 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:15.495028 10982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c16e096038084b2b9148390b6e773294" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:15.564749 11084 ts_tablet_manager.cc:1397] T 42ec275c905343c4be1b72dc2bd0f6e3 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.240s user 0.006s sys 0.006s
I20260925 09:57:15.564939 10982 raft_consensus.cc:2454] T c16e096038084b2b9148390b6e773294 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:15.449805 10389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_4cd66bd0a28d4f618a89f8e8bedca672"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:15.566562 10389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4cd66bd0a28d4f618a89f8e8bedca672 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:15.567023 11084 raft_consensus.cc:348] T 42ec275c905343c4be1b72dc2bd0f6e3 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:15.568869 11084 raft_consensus.cc:374] T 42ec275c905343c4be1b72dc2bd0f6e3 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:15.569240 11084 raft_consensus.cc:729] T 42ec275c905343c4be1b72dc2bd0f6e3 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
W20260925 09:57:15.491639 10379 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.AlterSchema -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 97974.3us
I20260925 09:57:15.569980 11084 consensus_queue.cc:260] T 42ec275c905343c4be1b72dc2bd0f6e3 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:15.570703 10984 raft_consensus.cc:2454] T c07c6205d1284a6fbb369c010a226da7 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:15.568451 10981 raft_consensus.cc:2454] T 47a9fb9392244e1d89f8af2bfb42edd6 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:15.571712 11084 ts_tablet_manager.cc:1428] T 42ec275c905343c4be1b72dc2bd0f6e3 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.007s user 0.003s sys 0.002s
I20260925 09:57:15.572588 11084 tablet_bootstrap.cc:492] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:15.576203 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet a0976e0c01234b38a7ab934dc1b289c4 (table default.Table_bcf18c926c6e4a62b31f3bffe9b0037e [id=53df4f0ffd794afab1aa17fcfb4d56bc]) successfully deleted
I20260925 09:57:15.579221 10816 tablet_service.cc:1514] Processing DeleteTablet for tablet fd73d0d52350444499abae34e8c02fab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:15 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:15.581948 10816 tablet_service.cc:1467] Processing CreateTablet for tablet 3706253e53d841aeaa43889ead13af32 (DEFAULT_TABLE table=default.Table_bbc3c50c06634d119ae96dcf649f44e7 [id=0260f0e6cdd6431db06788bde54ec38c]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 09:57:15.391860 10386 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.2:40761 (request call id 21) took 2077 ms (client timeout 1000 ms). Trace:
W20260925 09:57:15.584313 10386 rpcz_store.cc:269] 0925 09:57:13.314505 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:13.323639 (+ 9134us) service_pool.cc:227] Handling call
0925 09:57:13.513218 (+189579us) spinlock_profiling.cc:243] Waited 185 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2ad9c3e 00007fabf0610fb4 00007fabf0610f42 00007fabf0610e71 00007fabf0610dec 00007fabf06100a2 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:15.391578 (+1878360us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":3212,"reported_tablets":12,"spinlock_wait_cycles":425027968}
I20260925 09:57:15.586006 10386 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_8705f75ef7b2448ab645f1d25e3c2637" } new_table_name: "default.Table_988762e7d6264b1cb64927ff31711470" modify_external_catalogs: true
I20260925 09:57:15.587733 11084 tablet_bootstrap.cc:654] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:15.591939 10958 tablet_service.cc:1467] Processing CreateTablet for tablet 3706253e53d841aeaa43889ead13af32 (DEFAULT_TABLE table=default.Table_bbc3c50c06634d119ae96dcf649f44e7 [id=0260f0e6cdd6431db06788bde54ec38c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:15.593067 10958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3706253e53d841aeaa43889ead13af32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:15.593233 11085 tablet_bootstrap.cc:492] T 833fac37485141f6a18104eaea907b0e P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:15.593585 11085 ts_tablet_manager.cc:1397] T 833fac37485141f6a18104eaea907b0e P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.419s user 0.008s sys 0.002s
I20260925 09:57:15.595732 11085 raft_consensus.cc:348] T 833fac37485141f6a18104eaea907b0e P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:15.507934 11461 tablet_replica.cc:330] T 0e882284a014455192375daad391efaf P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
W20260925 09:57:15.576944 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 91c09118b43b41eca8f7040323ba76f7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 91c09118b43b41eca8f7040323ba76f7 already in progress: creating tablet
I20260925 09:57:15.398684 10391 rpcz_store.cc:276] 0925 09:57:14.266697 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:14.266842 (+ 145us) service_pool.cc:227] Handling call
0925 09:57:14.270669 (+ 3827us) spinlock_profiling.cc:243] Waited 3.05 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2adb7d6 00007fabf0621634 00007fabf06215c2 00007fabf06214f1 00007fabf062146c 00007fabf0620722 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:14.271534 (+ 865us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:57:14.272242 (+ 708us) catalog_manager.cc:3616] Apply alter schema
0925 09:57:14.272917 (+ 675us) catalog_manager.cc:3639] Acquired catalog manager lock
0925 09:57:14.273095 (+ 178us) catalog_manager.cc:3786] Updating metadata on disk
0925 09:57:14.845313 (+572218us) catalog_manager.cc:3817] Committing alterations to in-memory state
0925 09:57:15.370378 (+525065us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:14.653205 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:14.653556 (+ 351us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:14.653582 (+ 26us) write_op.cc:435] Acquired schema lock
0925 09:57:14.653592 (+ 10us) tablet.cc:660] Decoding operations
0925 09:57:14.655611 (+ 2019us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:14.655646 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:14.655666 (+ 20us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:14.655806 (+ 140us) tablet.cc:699] Row locks acquired
0925 09:57:14.655822 (+ 16us) write_op.cc:260] PREPARE: finished
0925 09:57:14.655923 (+ 101us) write_op.cc:270] Start()
0925 09:57:14.655998 (+ 75us) write_op.cc:276] Timestamp: P: 1790330234655887 usec, L: 0
0925 09:57:14.656013 (+ 15us) op_driver.cc:331] REPLICATION: starting
0925 09:57:14.656483 (+ 470us) log.cc:842] Serialized 405 byte log entry
0925 09:57:14.841162 (+184679us) op_driver.cc:444] REPLICATION: finished
0925 09:57:14.841759 (+ 597us) write_op.cc:301] APPLY: starting
0925 09:57:14.841817 (+ 58us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:14.841947 (+ 130us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:14.842055 (+ 108us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:14.842224 (+ 169us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:14.842353 (+ 129us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:14.842400 (+ 47us) write_op.cc:312] APPLY: finished
0925 09:57:14.842587 (+ 187us) log.cc:842] Serialized 23 byte log entry
0925 09:57:14.842756 (+ 169us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:14.842816 (+ 60us) write_op.cc:454] Released schema lock
0925 09:57:14.842848 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:14.843019 (+ 171us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:14.843044 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":7020416,"child_traces":[["op",{"apply.queue_time_us":441,"apply.run_cpu_time_us":1454,"apply.run_wall_time_us":1453,"num_ops":1,"prepare.queue_time_us":378897,"prepare.run_cpu_time_us":1738,"prepare.run_wall_time_us":5433,"replication_time_us":185059}]]}
I20260925 09:57:15.602844 10391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_eb065d5979c64931abf91d82b8623a0a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:15.604890 10391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eb065d5979c64931abf91d82b8623a0a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:15.608342 10961 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b281fc20e6f42239fee80d05da88ab6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:13 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:15.608628 10816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3706253e53d841aeaa43889ead13af32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:15.609318 10961 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d55154ceb064f22bd6f37c8ae5dce74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:12 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:15.612838 10658 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b281fc20e6f42239fee80d05da88ab6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:13 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:15.615064 10619 leader_election.cc:302] T c16e096038084b2b9148390b6e773294 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:15.615566 10658 tablet_service.cc:1514] Processing DeleteTablet for tablet b7646a1cde1b45b9bf7928c3e13bcbd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:12 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:15.616549 10619 leader_election.cc:302] T 47a9fb9392244e1d89f8af2bfb42edd6 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:15.619560 11320 raft_consensus.cc:2781] T c16e096038084b2b9148390b6e773294 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:15.619949 11320 raft_consensus.cc:482] T c16e096038084b2b9148390b6e773294 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:15.620322 11320 raft_consensus.cc:3037] T c16e096038084b2b9148390b6e773294 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:15.623339 10658 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d661d5000894c45ac3d1c27f0e72889 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:12 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:15.632280 11389 raft_consensus.cc:2781] T 47a9fb9392244e1d89f8af2bfb42edd6 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:15.632822 11389 raft_consensus.cc:686] T 47a9fb9392244e1d89f8af2bfb42edd6 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:15.633769 11389 consensus_queue.cc:237] T 47a9fb9392244e1d89f8af2bfb42edd6 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:15.641574 10658 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c13a2f288b14d03a3ca4fdd4889cbad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:09 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:15.644622 11495 tablet.cc:1722] T 1969ac471b6f471ebe8e048a96c22837 P b8850818e2ee42a082802d0095c08d21: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 6
I20260925 09:57:15.661375 10658 tablet_service.cc:1514] Processing DeleteTablet for tablet e48d86c20e9e4db990c23359ff1d1a6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:11 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:15.664752 11494 tablet.cc:1722] T 1969ac471b6f471ebe8e048a96c22837 P c43461c4ee854ae1ac52b873ccada318: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 6
I20260925 09:57:15.666138 10658 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e882284a014455192375daad391efaf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:09 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:15.673009 11461 raft_consensus.cc:2229] T 0e882284a014455192375daad391efaf P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:15.611223 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 833fac37485141f6a18104eaea907b0e (table default.Table_8e0c5caa8df14066af83c49a50a172bf [id=fc3b3c18f00c42559612468a21e96694]) successfully deleted
I20260925 09:57:15.673939 11461 raft_consensus.cc:2258] T 0e882284a014455192375daad391efaf P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:15.565460 10393 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_fea25102a7f84d58927645a0feae51e3" } modify_external_catalogs: true
W20260925 09:57:15.598191 10390 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.3:41469 (request call id 15) took 1593 ms (client timeout 1000 ms). Trace:
W20260925 09:57:15.674969 10390 rpcz_store.cc:269] 0925 09:57:14.004402 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:14.004501 (+ 99us) service_pool.cc:227] Handling call
0925 09:57:14.057667 (+ 53166us) spinlock_profiling.cc:243] Waited 49 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2ad9c3e 00007fabf0610fb4 00007fabf0610f42 00007fabf0610e71 00007fabf0610dec 00007fabf06100a2 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:15.598114 (+1540447us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":26,"spinlock_wait_cycles":112787840}
I20260925 09:57:15.675738 10814 tablet_service.cc:1467] Processing CreateTablet for tablet 93e1b5837aff48d7907890c40dfcd0d0 (DEFAULT_TABLE table=default.Table_5adf3f6c195e490fa4c978d7e1b4c187 [id=23d14ed1d2d647fc8b3b5ef40c7dc0bb]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 09:57:15.676544 10619 consensus_peers.cc:487] T 4c4e8790282a49fd849a34ca11bef5bf P b8850818e2ee42a082802d0095c08d21 -> Peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Couldn't send request to peer be97ab5d75934a648d4ab76341193ec0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4c4e8790282a49fd849a34ca11bef5bf. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:15.676883 10814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93e1b5837aff48d7907890c40dfcd0d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:15.677470 10390 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_8839d4ce47434a51a850fd37841e859a" } modify_external_catalogs: true
I20260925 09:57:15.679786 11082 tablet_bootstrap.cc:492] T 28d60531970a4ab58ce5ef99cb3a96bf P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:15.680315 11082 ts_tablet_manager.cc:1397] T 28d60531970a4ab58ce5ef99cb3a96bf P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.438s user 0.008s sys 0.003s
I20260925 09:57:15.682492 11082 raft_consensus.cc:348] T 28d60531970a4ab58ce5ef99cb3a96bf P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:15.683111 11082 raft_consensus.cc:374] T 28d60531970a4ab58ce5ef99cb3a96bf P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260925 09:57:15.686085 10384 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.1:56563 (request call id 24) took 2299 ms (client timeout 1000 ms). Trace:
I20260925 09:57:15.692364 11245 ts_tablet_manager.cc:1905] T a0976e0c01234b38a7ab934dc1b289c4 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 09:57:15.702193 11502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:57:15.698630 10379 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 121699us
W20260925 09:57:15.718544 10377 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 107328us
I20260925 09:57:15.721704 10962 tablet_service.cc:1467] Processing CreateTablet for tablet 93e1b5837aff48d7907890c40dfcd0d0 (DEFAULT_TABLE table=default.Table_5adf3f6c195e490fa4c978d7e1b4c187 [id=23d14ed1d2d647fc8b3b5ef40c7dc0bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:15.722949 10962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93e1b5837aff48d7907890c40dfcd0d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:15.723412 11500 tablet.cc:1722] T 1969ac471b6f471ebe8e048a96c22837 P be97ab5d75934a648d4ab76341193ec0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 6
I20260925 09:57:15.737793 11084 tablet_bootstrap.cc:492] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:15.738178 11084 ts_tablet_manager.cc:1397] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.166s user 0.008s sys 0.001s
I20260925 09:57:15.740952 11084 raft_consensus.cc:348] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:15.741663 11084 raft_consensus.cc:374] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:15.741961 11084 raft_consensus.cc:729] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:15.742537 11084 consensus_queue.cc:260] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:15.744863 11084 ts_tablet_manager.cc:1428] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260925 09:57:15.745659 11084 tablet_bootstrap.cc:492] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:15.683367 11082 raft_consensus.cc:729] T 28d60531970a4ab58ce5ef99cb3a96bf P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:15.748076 11082 consensus_queue.cc:260] T 28d60531970a4ab58ce5ef99cb3a96bf P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:15.750523 11082 ts_tablet_manager.cc:1428] T 28d60531970a4ab58ce5ef99cb3a96bf P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.070s user 0.001s sys 0.003s
I20260925 09:57:15.751413 11082 tablet_bootstrap.cc:492] T 3be0eccc65564bfba3d6e5c398c61fb6 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:15.753937 11461 ts_tablet_manager.cc:1905] T 0e882284a014455192375daad391efaf P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:15.760172 11507 tablet.cc:1722] T e57065f2bb224a56ad1c3518a0c375da P be97ab5d75934a648d4ab76341193ec0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:15.596343 11085 raft_consensus.cc:374] T 833fac37485141f6a18104eaea907b0e P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:15.761731 11085 raft_consensus.cc:729] T 833fac37485141f6a18104eaea907b0e P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:15.762521 11463 raft_consensus.cc:482] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:15.762575 11085 consensus_queue.cc:260] T 833fac37485141f6a18104eaea907b0e P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:15.762923 11463 raft_consensus.cc:504] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:15.764719 11463 leader_election.cc:288] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:15.770013 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41df75fe4b8e4856bdbb5ead3cf4fd24" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:15.771215 10918 leader_election.cc:341] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:15.687852 11320 raft_consensus.cc:504] T c16e096038084b2b9148390b6e773294 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:15.702626 10659 tablet_service.cc:1467] Processing CreateTablet for tablet 3d652a76d96f4dbeafde15f3c16534a6 (DEFAULT_TABLE table=default.Table_f8d796315be4442394ff13a053dd3294 [id=73083c1560cb4db5a12964387cb49b73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:15.768579 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41df75fe4b8e4856bdbb5ead3cf4fd24" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:15.777943 11475 consensus_queue.cc:1035] T 4c4e8790282a49fd849a34ca11bef5bf P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:15.729758 11504 tablet.cc:1722] T e57065f2bb224a56ad1c3518a0c375da P b8850818e2ee42a082802d0095c08d21: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:15.721132 11458 ts_tablet_manager.cc:1918] T a0976e0c01234b38a7ab934dc1b289c4 P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260925 09:57:15.799759 10919 leader_election.cc:341] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:15.800182 11458 log.cc:1196] T a0976e0c01234b38a7ab934dc1b289c4 P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/a0976e0c01234b38a7ab934dc1b289c4
I20260925 09:57:15.800195 10919 leader_election.cc:302] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:15.800374 10815 tablet_service.cc:1467] Processing CreateTablet for tablet 3d652a76d96f4dbeafde15f3c16534a6 (DEFAULT_TABLE table=default.Table_f8d796315be4442394ff13a053dd3294 [id=73083c1560cb4db5a12964387cb49b73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:15.800657 10659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d652a76d96f4dbeafde15f3c16534a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:15.800992 11458 ts_tablet_manager.cc:1939] T a0976e0c01234b38a7ab934dc1b289c4 P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:15.801262 11463 raft_consensus.cc:2726] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:15.801429 10815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d652a76d96f4dbeafde15f3c16534a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:15.802134 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet a0976e0c01234b38a7ab934dc1b289c4 (table default.Table_bcf18c926c6e4a62b31f3bffe9b0037e [id=53df4f0ffd794afab1aa17fcfb4d56bc]) successfully deleted
I20260925 09:57:15.802608 11458 tablet_replica.cc:330] T fd73d0d52350444499abae34e8c02fab P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
W20260925 09:57:15.803244 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 91c09118b43b41eca8f7040323ba76f7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 91c09118b43b41eca8f7040323ba76f7 already in progress: creating tablet
I20260925 09:57:15.808497 11320 leader_election.cc:288] T c16e096038084b2b9148390b6e773294 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:15.815990 10981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c16e096038084b2b9148390b6e773294" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:15.816664 10981 raft_consensus.cc:3037] T c16e096038084b2b9148390b6e773294 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:15.821763 11085 ts_tablet_manager.cc:1428] T 833fac37485141f6a18104eaea907b0e P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.228s user 0.005s sys 0.000s
I20260925 09:57:15.822649 11085 tablet_bootstrap.cc:492] T 69f84509cafc44dd9444218738ebc2fe P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:15.826951 10958 tablet_service.cc:1467] Processing CreateTablet for tablet 3d652a76d96f4dbeafde15f3c16534a6 (DEFAULT_TABLE table=default.Table_f8d796315be4442394ff13a053dd3294 [id=73083c1560cb4db5a12964387cb49b73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:15.833050 11084 tablet_bootstrap.cc:654] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:15.834575 10379 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.62:34455, user_credentials={real_user=slave}} blocked reactor thread for 44195.2us
I20260925 09:57:15.846618 11458 raft_consensus.cc:2229] T fd73d0d52350444499abae34e8c02fab P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:15.847348 11458 pending_rounds.cc:62] T fd73d0d52350444499abae34e8c02fab P b8850818e2ee42a082802d0095c08d21: Trying to abort 1 pending ops.
I20260925 09:57:15.847594 11458 pending_rounds.cc:68] T fd73d0d52350444499abae34e8c02fab P b8850818e2ee42a082802d0095c08d21: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192643503923200 op_type: NO_OP noop_request { }
I20260925 09:57:15.847958 11458 raft_consensus.cc:2866] T fd73d0d52350444499abae34e8c02fab P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:57:15.848299 11458 raft_consensus.cc:2258] T fd73d0d52350444499abae34e8c02fab P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:15.849287 11245 ts_tablet_manager.cc:1918] T a0976e0c01234b38a7ab934dc1b289c4 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260925 09:57:15.827492 10621 outbound_call.cc:322] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.10.9.62:34455, user_credentials={real_user=slave}} blocked reactor thread for 46916.2us
I20260925 09:57:15.852116 11245 log.cc:1196] T a0976e0c01234b38a7ab934dc1b289c4 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/a0976e0c01234b38a7ab934dc1b289c4
I20260925 09:57:15.854195 11245 ts_tablet_manager.cc:1939] T a0976e0c01234b38a7ab934dc1b289c4 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:15.854116 11458 ts_tablet_manager.cc:1905] T fd73d0d52350444499abae34e8c02fab P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:15.860625 11084 tablet_bootstrap.cc:492] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:15.861086 11084 ts_tablet_manager.cc:1397] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.116s user 0.006s sys 0.003s
I20260925 09:57:15.864241 11084 raft_consensus.cc:348] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:15.864773 11082 tablet_bootstrap.cc:654] T 3be0eccc65564bfba3d6e5c398c61fb6 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:15.867323 11084 raft_consensus.cc:374] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:15.870054 11084 raft_consensus.cc:729] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:15.870846 10658 tablet_service.cc:1467] Processing CreateTablet for tablet a21909e2e5794f57971f4e42795235c7 (DEFAULT_TABLE table=default.Table_48f8cb39156845c8bffbb2f9f17a900b [id=093bf6e622054652ad406255d557d0c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:15.872323 10658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a21909e2e5794f57971f4e42795235c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:15.870922 11084 consensus_queue.cc:260] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:15.874053 10961 tablet_service.cc:1467] Processing CreateTablet for tablet a21909e2e5794f57971f4e42795235c7 (DEFAULT_TABLE table=default.Table_48f8cb39156845c8bffbb2f9f17a900b [id=093bf6e622054652ad406255d557d0c8]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 09:57:15.713992 10384 rpcz_store.cc:269] 0925 09:57:13.386970 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:13.412712 (+ 25742us) service_pool.cc:227] Handling call
0925 09:57:13.513825 (+101113us) spinlock_profiling.cc:243] Waited 97.7 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2ad9c3e 00007fabf0610fb4 00007fabf0610f42 00007fabf0610e71 00007fabf0610dec 00007fabf06100a2 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:15.685996 (+2172171us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:14.151733 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:14.152077 (+ 344us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:14.152097 (+ 20us) write_op.cc:435] Acquired schema lock
0925 09:57:14.152108 (+ 11us) tablet.cc:660] Decoding operations
0925 09:57:14.152348 (+ 240us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:14.152389 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:14.152408 (+ 19us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:14.166289 (+ 13881us) tablet.cc:699] Row locks acquired
0925 09:57:14.166327 (+ 38us) write_op.cc:260] PREPARE: finished
0925 09:57:14.166448 (+ 121us) write_op.cc:270] Start()
0925 09:57:14.166541 (+ 93us) write_op.cc:276] Timestamp: P: 1790330234166395 usec, L: 0
0925 09:57:14.166555 (+ 14us) op_driver.cc:331] REPLICATION: starting
0925 09:57:14.191645 (+ 25090us) log.cc:842] Serialized 356 byte log entry
0925 09:57:14.275795 (+ 84150us) op_driver.cc:444] REPLICATION: finished
0925 09:57:14.277676 (+ 1881us) write_op.cc:301] APPLY: starting
0925 09:57:14.277742 (+ 66us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:14.277831 (+ 89us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:14.277931 (+ 100us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:14.278116 (+ 185us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:14.278241 (+ 125us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:14.278283 (+ 42us) write_op.cc:312] APPLY: finished
0925 09:57:14.278461 (+ 178us) log.cc:842] Serialized 23 byte log entry
0925 09:57:14.278603 (+ 142us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:14.278662 (+ 59us) write_op.cc:454] Released schema lock
0925 09:57:14.278693 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:14.278864 (+ 171us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:14.278889 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":11,"spinlock_wait_cycles":224629120,"child_traces":[["op",{"apply.queue_time_us":1667,"apply.run_cpu_time_us":1365,"apply.run_wall_time_us":1365,"num_ops":1,"prepare.queue_time_us":205368,"prepare.run_cpu_time_us":2071,"prepare.run_wall_time_us":49798,"raft.queue_time_us":7587,"raft.run_cpu_time_us":721,"raft.run_wall_time_us":722,"replication_time_us":109169}]]}
I20260925 09:57:15.887622 11084 ts_tablet_manager.cc:1428] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.026s user 0.006s sys 0.000s
I20260925 09:57:15.907722 10958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d652a76d96f4dbeafde15f3c16534a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:15.864779 11508 tablet.cc:1722] T e57065f2bb224a56ad1c3518a0c375da P c43461c4ee854ae1ac52b873ccada318: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:15.921299 11245 tablet_replica.cc:330] T 91c09118b43b41eca8f7040323ba76f7 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:15.922063 11245 raft_consensus.cc:2229] T 91c09118b43b41eca8f7040323ba76f7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:15.922493 11245 raft_consensus.cc:2258] T 91c09118b43b41eca8f7040323ba76f7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:15.924220 10961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a21909e2e5794f57971f4e42795235c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:15.930409 11085 tablet_bootstrap.cc:654] T 69f84509cafc44dd9444218738ebc2fe P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:15.931010 11084 tablet_bootstrap.cc:492] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:15.941823 11082 tablet_bootstrap.cc:492] T 3be0eccc65564bfba3d6e5c398c61fb6 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:15.942373 11082 ts_tablet_manager.cc:1397] T 3be0eccc65564bfba3d6e5c398c61fb6 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.191s user 0.008s sys 0.000s
I20260925 09:57:15.953377 11245 ts_tablet_manager.cc:1905] T 91c09118b43b41eca8f7040323ba76f7 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:15.955250 10981 raft_consensus.cc:2454] T c16e096038084b2b9148390b6e773294 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:15.956707 10619 leader_election.cc:302] T c16e096038084b2b9148390b6e773294 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:15.957566 11320 raft_consensus.cc:2781] T c16e096038084b2b9148390b6e773294 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:15.957964 11320 raft_consensus.cc:686] T c16e096038084b2b9148390b6e773294 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:15.958686 11320 consensus_queue.cc:237] T c16e096038084b2b9148390b6e773294 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:15.968901 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c16e096038084b2b9148390b6e773294" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:15.969502 11084 tablet_bootstrap.cc:654] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:15.991680 10621 leader_election.cc:341] T c16e096038084b2b9148390b6e773294 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:15.992818 10621 consensus_peers.cc:487] T 69f84509cafc44dd9444218738ebc2fe P b8850818e2ee42a082802d0095c08d21 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:15.995528 10384 master_service.cc:946] ReplaceTablet: received request to replace tablet 41df75fe4b8e4856bdbb5ead3cf4fd24 from {username='slave'} at 127.0.0.1:35140
I20260925 09:57:15.896561 10393 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_fea25102a7f84d58927645a0feae51e3" } modify_external_catalogs: true
I20260925 09:57:15.897326 10390 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_8839d4ce47434a51a850fd37841e859a" } modify_external_catalogs: true
I20260925 09:57:15.931855 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet a0976e0c01234b38a7ab934dc1b289c4 (table default.Table_bcf18c926c6e4a62b31f3bffe9b0037e [id=53df4f0ffd794afab1aa17fcfb4d56bc]) successfully deleted
I20260925 09:57:15.999770 11082 raft_consensus.cc:348] T 3be0eccc65564bfba3d6e5c398c61fb6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.000419 11082 raft_consensus.cc:374] T 3be0eccc65564bfba3d6e5c398c61fb6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:16.000643 11082 raft_consensus.cc:729] T 3be0eccc65564bfba3d6e5c398c61fb6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
W20260925 09:57:16.000211 10377 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 68354.5us
I20260925 09:57:15.977277 10389 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_f8d796315be4442394ff13a053dd3294" } new_table_name: "default.Table_be0f658aa41f4188bc16db37e96f4ad1" modify_external_catalogs: true
I20260925 09:57:16.002717 10816 tablet_service.cc:1467] Processing CreateTablet for tablet a21909e2e5794f57971f4e42795235c7 (DEFAULT_TABLE table=default.Table_48f8cb39156845c8bffbb2f9f17a900b [id=093bf6e622054652ad406255d557d0c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:16.004038 10816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a21909e2e5794f57971f4e42795235c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:16.011137 11458 ts_tablet_manager.cc:1918] T fd73d0d52350444499abae34e8c02fab P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:16.011629 11458 log.cc:1196] T fd73d0d52350444499abae34e8c02fab P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/fd73d0d52350444499abae34e8c02fab
I20260925 09:57:16.012390 11458 ts_tablet_manager.cc:1939] T fd73d0d52350444499abae34e8c02fab P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
W20260925 09:57:16.012753 10918 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 43830.6us
I20260925 09:57:16.013605 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet fd73d0d52350444499abae34e8c02fab (table default.Table_4c02305911f84ebb95187deaf905c14c [id=82e36829eee74495bec16d263e259c09]) successfully deleted
I20260925 09:57:16.013379 11082 consensus_queue.cc:260] T 3be0eccc65564bfba3d6e5c398c61fb6 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.016003 11082 ts_tablet_manager.cc:1428] T 3be0eccc65564bfba3d6e5c398c61fb6 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.073s user 0.007s sys 0.001s
I20260925 09:57:16.016870 11082 tablet_bootstrap.cc:492] T 91c09118b43b41eca8f7040323ba76f7 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:16.019673 11463 raft_consensus.cc:482] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:16.020462 11463 raft_consensus.cc:504] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.025868 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "359090244c4748e8a2d45e3c77e89498" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:16.026961 10378 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 28d60531970a4ab58ce5ef99cb3a96bf (table default.Table_ce09d3a8d8e84973894f33b18bbfdb22 [id=6851500c7dba44f1bf353bb79bafd97a], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 28d60531970a4ab58ce5ef99cb3a96bf consensus state has no leader
I20260925 09:57:16.028362 11084 tablet_bootstrap.cc:492] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:16.028750 11084 ts_tablet_manager.cc:1397] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.098s user 0.006s sys 0.006s
W20260925 09:57:16.032344 10919 leader_election.cc:341] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:16.033212 11463 leader_election.cc:288] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
W20260925 09:57:16.046347 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 8b281fc20e6f42239fee80d05da88ab6 because the tablet was not found. No further retry: Not found: Tablet not found: 8b281fc20e6f42239fee80d05da88ab6
W20260925 09:57:16.047286 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet b7646a1cde1b45b9bf7928c3e13bcbd3 because the tablet was not found. No further retry: Not found: Tablet not found: b7646a1cde1b45b9bf7928c3e13bcbd3
I20260925 09:57:16.030916 11084 raft_consensus.cc:348] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.047834 11084 raft_consensus.cc:374] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:16.048098 11084 raft_consensus.cc:729] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
W20260925 09:57:16.048014 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 2d661d5000894c45ac3d1c27f0e72889 because the tablet was not found. No further retry: Not found: Tablet not found: 2d661d5000894c45ac3d1c27f0e72889
I20260925 09:57:16.048617 11458 tablet_replica.cc:330] T 0e882284a014455192375daad391efaf P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
W20260925 09:57:16.048661 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 8c13a2f288b14d03a3ca4fdd4889cbad because the tablet was not found. No further retry: Not found: Tablet not found: 8c13a2f288b14d03a3ca4fdd4889cbad
I20260925 09:57:16.048655 11084 consensus_queue.cc:260] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:16.050273 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet e48d86c20e9e4db990c23359ff1d1a6d because the tablet was not found. No further retry: Not found: Tablet not found: e48d86c20e9e4db990c23359ff1d1a6d
I20260925 09:57:16.050591 11084 ts_tablet_manager.cc:1428] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.022s user 0.006s sys 0.000s
I20260925 09:57:16.050920 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "359090244c4748e8a2d45e3c77e89498" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:16.054569 10918 leader_election.cc:341] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:16.059036 11082 tablet_bootstrap.cc:654] T 91c09118b43b41eca8f7040323ba76f7 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:16.067615 11245 ts_tablet_manager.cc:1918] T 91c09118b43b41eca8f7040323ba76f7 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:16.067971 11245 log.cc:1196] T 91c09118b43b41eca8f7040323ba76f7 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/91c09118b43b41eca8f7040323ba76f7
I20260925 09:57:16.068560 11245 ts_tablet_manager.cc:1939] T 91c09118b43b41eca8f7040323ba76f7 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:16.070887 11084 tablet_bootstrap.cc:492] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:16.055100 10918 leader_election.cc:302] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:16.078399 11463 raft_consensus.cc:2726] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:16.086114 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 91c09118b43b41eca8f7040323ba76f7 (table default.Table_33b40a0d6ca44f15be47ca34bab6a093 [id=453beec106a64428b163c7db8132fc21]) successfully deleted
I20260925 09:57:16.087272 10658 tablet_service.cc:1467] Processing CreateTablet for tablet 7360a869f3be45a884a0379f4ef78e85 (DEFAULT_TABLE table=default.Table_a45881b9149d451ea82bef40dc98e75c [id=3566a060b9c54792920cc6583aec2ef0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:16.087558 11361 raft_consensus.cc:482] T 833fac37485141f6a18104eaea907b0e P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260925 09:57:16.087559 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 2d661d5000894c45ac3d1c27f0e72889 because the tablet was not found. No further retry: Not found: Tablet not found: 2d661d5000894c45ac3d1c27f0e72889
I20260925 09:57:16.087960 11361 raft_consensus.cc:504] T 833fac37485141f6a18104eaea907b0e P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.088562 10658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7360a869f3be45a884a0379f4ef78e85. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:16.088837 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet b7646a1cde1b45b9bf7928c3e13bcbd3 because the tablet was not found. No further retry: Not found: Tablet not found: b7646a1cde1b45b9bf7928c3e13bcbd3
I20260925 09:57:16.089579 11245 tablet_replica.cc:330] T fd73d0d52350444499abae34e8c02fab P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:16.089780 11361 leader_election.cc:288] T 833fac37485141f6a18104eaea907b0e P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
W20260925 09:57:16.093552 10772 leader_election.cc:341] T 833fac37485141f6a18104eaea907b0e P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 833fac37485141f6a18104eaea907b0e
I20260925 09:57:16.098979 11458 raft_consensus.cc:2229] T 0e882284a014455192375daad391efaf P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:16.101058 10815 tablet_service.cc:1467] Processing CreateTablet for tablet 7360a869f3be45a884a0379f4ef78e85 (DEFAULT_TABLE table=default.Table_a45881b9149d451ea82bef40dc98e75c [id=3566a060b9c54792920cc6583aec2ef0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:16.102325 10815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7360a869f3be45a884a0379f4ef78e85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:16.104727 10958 tablet_service.cc:1467] Processing CreateTablet for tablet 7360a869f3be45a884a0379f4ef78e85 (DEFAULT_TABLE table=default.Table_a45881b9149d451ea82bef40dc98e75c [id=3566a060b9c54792920cc6583aec2ef0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:16.105676 10958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7360a869f3be45a884a0379f4ef78e85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:16.109900 11463 raft_consensus.cc:482] T 752105c1514f4c1db2a11b95b413b4f6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:16.110570 11463 raft_consensus.cc:504] T 752105c1514f4c1db2a11b95b413b4f6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.113885 11463 leader_election.cc:288] T 752105c1514f4c1db2a11b95b413b4f6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
W20260925 09:57:16.115201 10918 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 60659.7us
I20260925 09:57:16.117834 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "752105c1514f4c1db2a11b95b413b4f6" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:16.119184 10918 leader_election.cc:341] T 752105c1514f4c1db2a11b95b413b4f6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:16.120707 11245 raft_consensus.cc:2229] T fd73d0d52350444499abae34e8c02fab P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:16.121277 11245 raft_consensus.cc:2258] T fd73d0d52350444499abae34e8c02fab P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:16.123056 11245 ts_tablet_manager.cc:1905] T fd73d0d52350444499abae34e8c02fab P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:16.132905 11082 tablet_bootstrap.cc:492] T 91c09118b43b41eca8f7040323ba76f7 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:16.134550 11082 ts_tablet_manager.cc:1397] T 91c09118b43b41eca8f7040323ba76f7 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.118s user 0.006s sys 0.004s
I20260925 09:57:16.135444 11461 ts_tablet_manager.cc:1918] T 0e882284a014455192375daad391efaf P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:16.135888 11461 log.cc:1196] T 0e882284a014455192375daad391efaf P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/0e882284a014455192375daad391efaf
I20260925 09:57:16.136615 11461 ts_tablet_manager.cc:1939] T 0e882284a014455192375daad391efaf P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:16.139107 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 0e882284a014455192375daad391efaf (table default.Table_62ea48a919ec4159bcc86ebb048e3545 [id=051a096f481f4f20a59081fbd08b3b1f]) successfully deleted
I20260925 09:57:16.095227 11084 tablet_bootstrap.cc:654] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:16.147097 11085 tablet_bootstrap.cc:492] T 69f84509cafc44dd9444218738ebc2fe P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:16.147536 11085 ts_tablet_manager.cc:1397] T 69f84509cafc44dd9444218738ebc2fe P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.325s user 0.000s sys 0.014s
I20260925 09:57:16.149597 11085 raft_consensus.cc:348] T 69f84509cafc44dd9444218738ebc2fe P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.150193 11085 raft_consensus.cc:374] T 69f84509cafc44dd9444218738ebc2fe P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:16.150431 11085 raft_consensus.cc:729] T 69f84509cafc44dd9444218738ebc2fe P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:16.169064 11084 tablet_bootstrap.cc:492] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:16.169582 11084 ts_tablet_manager.cc:1397] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.118s user 0.006s sys 0.005s
W20260925 09:57:16.172227 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 2d661d5000894c45ac3d1c27f0e72889 because the tablet was not found. No further retry: Not found: Tablet not found: 2d661d5000894c45ac3d1c27f0e72889
W20260925 09:57:16.173645 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet b7646a1cde1b45b9bf7928c3e13bcbd3 because the tablet was not found. No further retry: Not found: Tablet not found: b7646a1cde1b45b9bf7928c3e13bcbd3
W20260925 09:57:16.174860 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 8b281fc20e6f42239fee80d05da88ab6 because the tablet was not found. No further retry: Not found: Tablet not found: 8b281fc20e6f42239fee80d05da88ab6
I20260925 09:57:16.172372 11084 raft_consensus.cc:348] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.175765 11084 raft_consensus.cc:374] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:16.177155 11085 consensus_queue.cc:260] T 69f84509cafc44dd9444218738ebc2fe P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:16.178856 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet e48d86c20e9e4db990c23359ff1d1a6d because the tablet was not found. No further retry: Not found: Tablet not found: e48d86c20e9e4db990c23359ff1d1a6d
I20260925 09:57:16.181130 11461 tablet_replica.cc:330] T fd73d0d52350444499abae34e8c02fab P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:16.179284 11084 raft_consensus.cc:729] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:16.184859 11084 consensus_queue.cc:260] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.198594 11085 ts_tablet_manager.cc:1428] T 69f84509cafc44dd9444218738ebc2fe P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.051s user 0.001s sys 0.004s
I20260925 09:57:16.201424 11084 ts_tablet_manager.cc:1428] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.031s user 0.005s sys 0.000s
W20260925 09:57:16.206151 10389 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3d652a76d96f4dbeafde15f3c16534a6 consensus state has no leader
I20260925 09:57:16.206760 11461 raft_consensus.cc:2229] T fd73d0d52350444499abae34e8c02fab P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:16.207314 11461 raft_consensus.cc:2258] T fd73d0d52350444499abae34e8c02fab P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:16.208730 10389 master_service.cc:946] ReplaceTablet: received request to replace tablet 1969ac471b6f471ebe8e048a96c22837 from {username='slave'} at 127.0.0.1:35140
I20260925 09:57:16.209772 10390 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 9c4a26ceb53f47bba07058682eb09850
I20260925 09:57:16.210942 11084 tablet_bootstrap.cc:492] T 67ce2e3325214aa6bdc3347e9e8f82b0 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:16.219843 11085 tablet_bootstrap.cc:492] T c16e096038084b2b9148390b6e773294 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
W20260925 09:57:16.231489 10379 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.9.60:36985: connect: Connection refused (error 111) [suppressed 7 similar messages]
I20260925 09:57:16.233072 11084 tablet_bootstrap.cc:654] T 67ce2e3325214aa6bdc3347e9e8f82b0 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:16.240167 11461 ts_tablet_manager.cc:1905] T fd73d0d52350444499abae34e8c02fab P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:16.249790 10390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_15b07a280b634333985637e0f6746764"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:16.251747 11245 ts_tablet_manager.cc:1918] T fd73d0d52350444499abae34e8c02fab P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:16.252216 11245 log.cc:1196] T fd73d0d52350444499abae34e8c02fab P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/fd73d0d52350444499abae34e8c02fab
I20260925 09:57:16.254518 11245 ts_tablet_manager.cc:1939] T fd73d0d52350444499abae34e8c02fab P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:16.255223 11085 tablet_bootstrap.cc:654] T c16e096038084b2b9148390b6e773294 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:16.259625 10621 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 175965us
W20260925 09:57:16.160348 10282 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 97002.7us
I20260925 09:57:16.266536 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet fd73d0d52350444499abae34e8c02fab (table default.Table_4c02305911f84ebb95187deaf905c14c [id=82e36829eee74495bec16d263e259c09]) successfully deleted
W20260925 09:57:16.268038 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 8b281fc20e6f42239fee80d05da88ab6 because the tablet was not found. No further retry: Not found: Tablet not found: 8b281fc20e6f42239fee80d05da88ab6
I20260925 09:57:16.268532 11526 raft_consensus.cc:482] T 69f84509cafc44dd9444218738ebc2fe P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:16.268812 11245 tablet_replica.cc:330] T 2d55154ceb064f22bd6f37c8ae5dce74 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:16.269655 11245 raft_consensus.cc:2229] T 2d55154ceb064f22bd6f37c8ae5dce74 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:16.269004 11526 raft_consensus.cc:504] T 69f84509cafc44dd9444218738ebc2fe P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.263059 11458 raft_consensus.cc:2258] T 0e882284a014455192375daad391efaf P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:16.278563 11458 ts_tablet_manager.cc:1905] T 0e882284a014455192375daad391efaf P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:16.279145 11526 leader_election.cc:288] T 69f84509cafc44dd9444218738ebc2fe P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:16.279688 11245 raft_consensus.cc:2258] T 2d55154ceb064f22bd6f37c8ae5dce74 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:16.267942 11082 raft_consensus.cc:348] T 91c09118b43b41eca8f7040323ba76f7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.283273 11082 raft_consensus.cc:374] T 91c09118b43b41eca8f7040323ba76f7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:16.283646 11082 raft_consensus.cc:729] T 91c09118b43b41eca8f7040323ba76f7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
W20260925 09:57:16.285234 10775 leader_election.cc:341] T 833fac37485141f6a18104eaea907b0e P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Not found: Tablet not found: 833fac37485141f6a18104eaea907b0e
I20260925 09:57:16.285504 11084 tablet_bootstrap.cc:492] T 67ce2e3325214aa6bdc3347e9e8f82b0 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:16.285630 10775 leader_election.cc:302] T 833fac37485141f6a18104eaea907b0e P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:16.285872 11084 ts_tablet_manager.cc:1397] T 67ce2e3325214aa6bdc3347e9e8f82b0 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.075s user 0.006s sys 0.002s
I20260925 09:57:16.286904 11361 raft_consensus.cc:2726] T 833fac37485141f6a18104eaea907b0e P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:16.282528 10981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69f84509cafc44dd9444218738ebc2fe" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:16.288188 10981 raft_consensus.cc:2379] T 69f84509cafc44dd9444218738ebc2fe P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c43461c4ee854ae1ac52b873ccada318 in current term 1: Already voted for candidate b8850818e2ee42a082802d0095c08d21 in this term.
I20260925 09:57:16.288081 11084 raft_consensus.cc:348] T 67ce2e3325214aa6bdc3347e9e8f82b0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.288715 11084 raft_consensus.cc:374] T 67ce2e3325214aa6bdc3347e9e8f82b0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:16.288964 11084 raft_consensus.cc:729] T 67ce2e3325214aa6bdc3347e9e8f82b0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:16.269045 11497 raft_consensus.cc:482] T 3be0eccc65564bfba3d6e5c398c61fb6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:16.290170 11245 ts_tablet_manager.cc:1905] T 2d55154ceb064f22bd6f37c8ae5dce74 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:16.269899 10683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "752105c1514f4c1db2a11b95b413b4f6" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:16.290972 11085 tablet_bootstrap.cc:492] T c16e096038084b2b9148390b6e773294 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:16.281282 11505 consensus_queue.cc:1035] T 69f84509cafc44dd9444218738ebc2fe P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:16.291380 11085 ts_tablet_manager.cc:1397] T c16e096038084b2b9148390b6e773294 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.072s user 0.005s sys 0.004s
I20260925 09:57:16.292060 11084 consensus_queue.cc:260] T 67ce2e3325214aa6bdc3347e9e8f82b0 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.292079 11082 consensus_queue.cc:260] T 91c09118b43b41eca8f7040323ba76f7 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.293459 11085 raft_consensus.cc:348] T c16e096038084b2b9148390b6e773294 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.294041 11085 raft_consensus.cc:374] T c16e096038084b2b9148390b6e773294 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:16.294283 11085 raft_consensus.cc:729] T c16e096038084b2b9148390b6e773294 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:16.298300 11085 consensus_queue.cc:260] T c16e096038084b2b9148390b6e773294 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.303663 11085 ts_tablet_manager.cc:1428] T c16e096038084b2b9148390b6e773294 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.012s user 0.005s sys 0.000s
I20260925 09:57:16.304446 11085 tablet_bootstrap.cc:492] T 3be0eccc65564bfba3d6e5c398c61fb6 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:16.297300 11497 raft_consensus.cc:504] T 3be0eccc65564bfba3d6e5c398c61fb6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.316884 11497 leader_election.cc:288] T 3be0eccc65564bfba3d6e5c398c61fb6 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:16.317135 11461 ts_tablet_manager.cc:1918] T fd73d0d52350444499abae34e8c02fab P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:16.317509 11461 log.cc:1196] T fd73d0d52350444499abae34e8c02fab P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/fd73d0d52350444499abae34e8c02fab
I20260925 09:57:16.318118 11461 ts_tablet_manager.cc:1939] T fd73d0d52350444499abae34e8c02fab P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:16.319975 11082 ts_tablet_manager.cc:1428] T 91c09118b43b41eca8f7040323ba76f7 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.185s user 0.007s sys 0.000s
I20260925 09:57:16.321012 11082 tablet_bootstrap.cc:492] T ef22f64036684dce9d7c69c37722f468 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
W20260925 09:57:16.322242 10919 leader_election.cc:341] T 752105c1514f4c1db2a11b95b413b4f6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:16.322793 10919 leader_election.cc:302] T 752105c1514f4c1db2a11b95b413b4f6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:16.323748 11463 raft_consensus.cc:2726] T 752105c1514f4c1db2a11b95b413b4f6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:16.325799 10681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69f84509cafc44dd9444218738ebc2fe" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:16.327015 11084 ts_tablet_manager.cc:1428] T 67ce2e3325214aa6bdc3347e9e8f82b0 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.041s user 0.006s sys 0.001s
I20260925 09:57:16.329025 11085 tablet_bootstrap.cc:654] T 3be0eccc65564bfba3d6e5c398c61fb6 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:16.329484 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet fd73d0d52350444499abae34e8c02fab (table default.Table_4c02305911f84ebb95187deaf905c14c [id=82e36829eee74495bec16d263e259c09]) successfully deleted
I20260925 09:57:16.334509 11463 raft_consensus.cc:482] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:16.334890 11463 raft_consensus.cc:504] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.336817 10775 leader_election.cc:302] T 69f84509cafc44dd9444218738ebc2fe P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:16.338086 11463 leader_election.cc:288] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:16.339593 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "689ab2b6cf1640129f6db8b113cf2fc6" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:16.340871 10918 leader_election.cc:341] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:16.341511 10683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "689ab2b6cf1640129f6db8b113cf2fc6" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:16.344266 10981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3be0eccc65564bfba3d6e5c398c61fb6" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:16.344882 10981 raft_consensus.cc:2454] T 3be0eccc65564bfba3d6e5c398c61fb6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:16.346010 10619 leader_election.cc:302] T 3be0eccc65564bfba3d6e5c398c61fb6 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:16.346318 11361 raft_consensus.cc:3037] T 69f84509cafc44dd9444218738ebc2fe P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:57:16.344899 10919 leader_election.cc:341] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:16.347286 10919 leader_election.cc:302] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:16.348240 11463 raft_consensus.cc:2726] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:16.351725 11497 raft_consensus.cc:2781] T 3be0eccc65564bfba3d6e5c398c61fb6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:16.353533 11497 raft_consensus.cc:482] T 3be0eccc65564bfba3d6e5c398c61fb6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:16.353958 11497 raft_consensus.cc:3037] T 3be0eccc65564bfba3d6e5c398c61fb6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:16.354509 11389 raft_consensus.cc:482] T 91c09118b43b41eca8f7040323ba76f7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:16.355191 11389 raft_consensus.cc:504] T 91c09118b43b41eca8f7040323ba76f7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.356400 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3be0eccc65564bfba3d6e5c398c61fb6" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:16.357048 11389 leader_election.cc:288] T 91c09118b43b41eca8f7040323ba76f7 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
W20260925 09:57:16.359205 10619 leader_election.cc:341] T 91c09118b43b41eca8f7040323ba76f7 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 91c09118b43b41eca8f7040323ba76f7
I20260925 09:57:16.359573 11085 tablet_bootstrap.cc:492] T 3be0eccc65564bfba3d6e5c398c61fb6 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:16.360061 11085 ts_tablet_manager.cc:1397] T 3be0eccc65564bfba3d6e5c398c61fb6 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.056s user 0.008s sys 0.004s
I20260925 09:57:16.362149 11085 raft_consensus.cc:348] T 3be0eccc65564bfba3d6e5c398c61fb6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.362751 11085 raft_consensus.cc:374] T 3be0eccc65564bfba3d6e5c398c61fb6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:16.363058 11085 raft_consensus.cc:729] T 3be0eccc65564bfba3d6e5c398c61fb6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:16.362326 11361 raft_consensus.cc:2726] T 69f84509cafc44dd9444218738ebc2fe P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:16.364547 11084 tablet_bootstrap.cc:492] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:16.364212 11085 consensus_queue.cc:260] T 3be0eccc65564bfba3d6e5c398c61fb6 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.366402 11085 ts_tablet_manager.cc:1428] T 3be0eccc65564bfba3d6e5c398c61fb6 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260925 09:57:16.375613 11085 tablet_bootstrap.cc:492] T 47a9fb9392244e1d89f8af2bfb42edd6 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
W20260925 09:57:16.379949 10621 leader_election.cc:341] T 3be0eccc65564bfba3d6e5c398c61fb6 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:16.387504 11085 tablet_bootstrap.cc:654] T 47a9fb9392244e1d89f8af2bfb42edd6 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:16.401479 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91c09118b43b41eca8f7040323ba76f7" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:16.404603 11085 tablet_bootstrap.cc:492] T 47a9fb9392244e1d89f8af2bfb42edd6 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:16.405037 11085 ts_tablet_manager.cc:1397] T 47a9fb9392244e1d89f8af2bfb42edd6 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.030s user 0.013s sys 0.000s
I20260925 09:57:16.405956 11463 raft_consensus.cc:482] T 42ec275c905343c4be1b72dc2bd0f6e3 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:16.406368 11463 raft_consensus.cc:504] T 42ec275c905343c4be1b72dc2bd0f6e3 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.407219 11085 raft_consensus.cc:348] T 47a9fb9392244e1d89f8af2bfb42edd6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.408387 11085 raft_consensus.cc:374] T 47a9fb9392244e1d89f8af2bfb42edd6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:16.408618 11085 raft_consensus.cc:729] T 47a9fb9392244e1d89f8af2bfb42edd6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:16.409228 11463 leader_election.cc:288] T 42ec275c905343c4be1b72dc2bd0f6e3 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:16.409322 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42ec275c905343c4be1b72dc2bd0f6e3" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:16.410528 10918 leader_election.cc:341] T 42ec275c905343c4be1b72dc2bd0f6e3 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:16.410848 10681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42ec275c905343c4be1b72dc2bd0f6e3" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:16.412145 10919 leader_election.cc:341] T 42ec275c905343c4be1b72dc2bd0f6e3 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:16.412539 10919 leader_election.cc:302] T 42ec275c905343c4be1b72dc2bd0f6e3 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:16.415307 11085 consensus_queue.cc:260] T 47a9fb9392244e1d89f8af2bfb42edd6 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.416877 11526 raft_consensus.cc:482] T c16e096038084b2b9148390b6e773294 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:16.417315 11526 raft_consensus.cc:504] T c16e096038084b2b9148390b6e773294 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.420854 11526 leader_election.cc:288] T c16e096038084b2b9148390b6e773294 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:16.421522 11463 raft_consensus.cc:2726] T 42ec275c905343c4be1b72dc2bd0f6e3 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:16.421671 10981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c16e096038084b2b9148390b6e773294" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:16.422485 10981 raft_consensus.cc:2379] T c16e096038084b2b9148390b6e773294 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c43461c4ee854ae1ac52b873ccada318 in current term 1: Already voted for candidate b8850818e2ee42a082802d0095c08d21 in this term.
I20260925 09:57:16.425477 11085 ts_tablet_manager.cc:1428] T 47a9fb9392244e1d89f8af2bfb42edd6 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.020s user 0.005s sys 0.000s
I20260925 09:57:16.426326 11085 tablet_bootstrap.cc:492] T 91c09118b43b41eca8f7040323ba76f7 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:16.431854 11085 tablet_bootstrap.cc:654] T 91c09118b43b41eca8f7040323ba76f7 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:16.441371 10681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c16e096038084b2b9148390b6e773294" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:16.443737 10775 leader_election.cc:302] T c16e096038084b2b9148390b6e773294 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:16.445227 11526 raft_consensus.cc:3037] T c16e096038084b2b9148390b6e773294 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:16.453553 11526 raft_consensus.cc:2726] T c16e096038084b2b9148390b6e773294 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:16.456030 10621 leader_election.cc:341] T 91c09118b43b41eca8f7040323ba76f7 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:16.456538 10621 leader_election.cc:302] T 91c09118b43b41eca8f7040323ba76f7 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8850818e2ee42a082802d0095c08d21; no voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:16.457427 11505 raft_consensus.cc:2726] T 91c09118b43b41eca8f7040323ba76f7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:16.461383 11082 tablet_bootstrap.cc:654] T ef22f64036684dce9d7c69c37722f468 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:16.462632 11085 tablet_bootstrap.cc:492] T 91c09118b43b41eca8f7040323ba76f7 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:16.463181 11085 ts_tablet_manager.cc:1397] T 91c09118b43b41eca8f7040323ba76f7 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.037s user 0.010s sys 0.000s
I20260925 09:57:16.466256 10393 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet cf35f39bcd6a41128acb2a4936fcc316
I20260925 09:57:16.469691 10659 tablet_service.cc:1514] Processing DeleteTablet for tablet cf35f39bcd6a41128acb2a4936fcc316 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:16 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:16.471339 10393 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:35140 (request call id 159) took 1057 ms. Trace:
I20260925 09:57:16.472276 10958 tablet_service.cc:1514] Processing DeleteTablet for tablet cf35f39bcd6a41128acb2a4936fcc316 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:16 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:16.473450 11085 raft_consensus.cc:348] T 91c09118b43b41eca8f7040323ba76f7 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.474109 11085 raft_consensus.cc:374] T 91c09118b43b41eca8f7040323ba76f7 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:16.427320 11084 tablet_bootstrap.cc:654] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:16.476923 10390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_15b07a280b634333985637e0f6746764 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:16.474344 11085 raft_consensus.cc:729] T 91c09118b43b41eca8f7040323ba76f7 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:16.488063 10816 tablet_service.cc:1514] Processing DeleteTablet for tablet cf35f39bcd6a41128acb2a4936fcc316 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:16 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:16.489459 11085 consensus_queue.cc:260] T 91c09118b43b41eca8f7040323ba76f7 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.493338 11085 ts_tablet_manager.cc:1428] T 91c09118b43b41eca8f7040323ba76f7 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.030s user 0.003s sys 0.001s
I20260925 09:57:16.494130 11085 tablet_bootstrap.cc:492] T 28d60531970a4ab58ce5ef99cb3a96bf P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:16.494422 10391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_a4437c88d32f4288a6d9581edb106440"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:16.497076 10391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a4437c88d32f4288a6d9581edb106440 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:16.503486 10384 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 41df75fe4b8e4856bdbb5ead3cf4fd24
I20260925 09:57:16.505584 10659 tablet_service.cc:1514] Processing DeleteTablet for tablet 41df75fe4b8e4856bdbb5ead3cf4fd24 with delete_type TABLET_DATA_DELETED (replaced by tablet c7cd1d6a603642afa2c8c737079961b5) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:16.507453 11085 tablet_bootstrap.cc:654] T 28d60531970a4ab58ce5ef99cb3a96bf P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:16.511052 11084 tablet_bootstrap.cc:492] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:16.511427 11084 ts_tablet_manager.cc:1397] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.147s user 0.008s sys 0.002s
I20260925 09:57:16.513799 11245 ts_tablet_manager.cc:1918] T 2d55154ceb064f22bd6f37c8ae5dce74 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:16.517050 11245 log.cc:1196] T 2d55154ceb064f22bd6f37c8ae5dce74 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/2d55154ceb064f22bd6f37c8ae5dce74
I20260925 09:57:16.517742 11245 ts_tablet_manager.cc:1939] T 2d55154ceb064f22bd6f37c8ae5dce74 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:16.519055 10384 catalog_manager.cc:6565] ReplaceTablet: tablet 41df75fe4b8e4856bdbb5ead3cf4fd24 deleted and replaced by tablet c7cd1d6a603642afa2c8c737079961b5
I20260925 09:57:16.518469 10958 tablet_service.cc:1514] Processing DeleteTablet for tablet 41df75fe4b8e4856bdbb5ead3cf4fd24 with delete_type TABLET_DATA_DELETED (replaced by tablet c7cd1d6a603642afa2c8c737079961b5) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:16.521667 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 2d55154ceb064f22bd6f37c8ae5dce74 (table default.Table_7eaf247fb74f497d80a88b67fa301cab [id=efe6d05ed9084a1c998e95e99672d4fc]) successfully deleted
I20260925 09:57:16.524580 11084 raft_consensus.cc:348] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.525233 11084 raft_consensus.cc:374] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:16.525552 11084 raft_consensus.cc:729] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:16.527325 11084 consensus_queue.cc:260] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.536103 11085 tablet_bootstrap.cc:492] T 28d60531970a4ab58ce5ef99cb3a96bf P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:16.536468 11085 ts_tablet_manager.cc:1397] T 28d60531970a4ab58ce5ef99cb3a96bf P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.042s user 0.004s sys 0.006s
I20260925 09:57:16.538432 11085 raft_consensus.cc:348] T 28d60531970a4ab58ce5ef99cb3a96bf P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.539067 11085 raft_consensus.cc:374] T 28d60531970a4ab58ce5ef99cb3a96bf P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:16.539304 11085 raft_consensus.cc:729] T 28d60531970a4ab58ce5ef99cb3a96bf P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:16.539914 11085 consensus_queue.cc:260] T 28d60531970a4ab58ce5ef99cb3a96bf P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.542062 11084 ts_tablet_manager.cc:1428] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.030s user 0.003s sys 0.003s
I20260925 09:57:16.544272 11084 tablet_bootstrap.cc:492] T c588b0c05a1a4f3ea545066183e76502 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:16.546868 11085 ts_tablet_manager.cc:1428] T 28d60531970a4ab58ce5ef99cb3a96bf P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20260925 09:57:16.547636 11085 tablet_bootstrap.cc:492] T 2d55154ceb064f22bd6f37c8ae5dce74 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
W20260925 09:57:16.549170 10377 catalog_manager.cc:4821] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet cf35f39bcd6a41128acb2a4936fcc316 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cf35f39bcd6a41128acb2a4936fcc316 already in progress: creating tablet
I20260925 09:57:16.553701 11085 tablet_bootstrap.cc:654] T 2d55154ceb064f22bd6f37c8ae5dce74 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:16.565850 11084 tablet_bootstrap.cc:654] T c588b0c05a1a4f3ea545066183e76502 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:16.566298 11245 tablet_replica.cc:330] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
W20260925 09:57:16.566273 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet cf35f39bcd6a41128acb2a4936fcc316 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cf35f39bcd6a41128acb2a4936fcc316 already in progress: creating tablet
I20260925 09:57:16.567260 11245 raft_consensus.cc:2229] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:16.568235 10836 raft_consensus.cc:3037] T 47a9fb9392244e1d89f8af2bfb42edd6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:16.568457 11245 raft_consensus.cc:2258] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:16.579454 11245 ts_tablet_manager.cc:1905] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:16.580467 11463 raft_consensus.cc:482] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:16.580907 11463 raft_consensus.cc:504] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.582911 11085 tablet_bootstrap.cc:492] T 2d55154ceb064f22bd6f37c8ae5dce74 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:16.582849 11463 leader_election.cc:288] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:16.583284 11085 ts_tablet_manager.cc:1397] T 2d55154ceb064f22bd6f37c8ae5dce74 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.036s user 0.006s sys 0.004s
I20260925 09:57:16.583177 11361 raft_consensus.cc:482] T 47a9fb9392244e1d89f8af2bfb42edd6 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:16.583801 11361 raft_consensus.cc:504] T 47a9fb9392244e1d89f8af2bfb42edd6 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.584318 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26fae3fbfe374f348a1729c21ec751b9" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:16.585469 11085 raft_consensus.cc:348] T 2d55154ceb064f22bd6f37c8ae5dce74 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.586105 11085 raft_consensus.cc:374] T 2d55154ceb064f22bd6f37c8ae5dce74 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260925 09:57:16.586140 10918 leader_election.cc:341] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:16.587335 10681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26fae3fbfe374f348a1729c21ec751b9" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:16.588089 11084 tablet_bootstrap.cc:492] T c588b0c05a1a4f3ea545066183e76502 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:16.588467 11084 ts_tablet_manager.cc:1397] T c588b0c05a1a4f3ea545066183e76502 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.044s user 0.009s sys 0.001s
W20260925 09:57:16.588706 10919 leader_election.cc:341] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:16.589089 10919 leader_election.cc:302] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:16.589370 11361 leader_election.cc:288] T 47a9fb9392244e1d89f8af2bfb42edd6 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:16.590070 11463 raft_consensus.cc:2726] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:16.590246 10981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47a9fb9392244e1d89f8af2bfb42edd6" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:16.590647 11084 raft_consensus.cc:348] T c588b0c05a1a4f3ea545066183e76502 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.591403 11084 raft_consensus.cc:374] T c588b0c05a1a4f3ea545066183e76502 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:16.591084 10981 raft_consensus.cc:2454] T 47a9fb9392244e1d89f8af2bfb42edd6 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:16.592917 10772 leader_election.cc:302] T 47a9fb9392244e1d89f8af2bfb42edd6 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:16.591636 11084 raft_consensus.cc:729] T c588b0c05a1a4f3ea545066183e76502 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:16.593883 11084 consensus_queue.cc:260] T c588b0c05a1a4f3ea545066183e76502 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.595713 11361 raft_consensus.cc:2781] T 47a9fb9392244e1d89f8af2bfb42edd6 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 09:57:16.596182 11361 raft_consensus.cc:482] T 47a9fb9392244e1d89f8af2bfb42edd6 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:16.596660 11361 raft_consensus.cc:3037] T 47a9fb9392244e1d89f8af2bfb42edd6 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:57:16.586350 11085 raft_consensus.cc:729] T 2d55154ceb064f22bd6f37c8ae5dce74 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:16.598637 11085 consensus_queue.cc:260] T 2d55154ceb064f22bd6f37c8ae5dce74 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.600871 11085 ts_tablet_manager.cc:1428] T 2d55154ceb064f22bd6f37c8ae5dce74 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.017s user 0.002s sys 0.002s
I20260925 09:57:16.601670 11085 tablet_bootstrap.cc:492] T 3c31682bf400494f82ee3ce5e6eec441 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:16.610020 11084 ts_tablet_manager.cc:1428] T c588b0c05a1a4f3ea545066183e76502 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.021s user 0.005s sys 0.003s
I20260925 09:57:16.611407 11084 tablet_bootstrap.cc:492] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:16.618630 11085 tablet_bootstrap.cc:654] T 3c31682bf400494f82ee3ce5e6eec441 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:16.623319 11389 raft_consensus.cc:482] T 28d60531970a4ab58ce5ef99cb3a96bf P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:16.623755 11389 raft_consensus.cc:504] T 28d60531970a4ab58ce5ef99cb3a96bf P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.625367 11084 tablet_bootstrap.cc:654] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:16.625586 11389 leader_election.cc:288] T 28d60531970a4ab58ce5ef99cb3a96bf P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:16.629765 10981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28d60531970a4ab58ce5ef99cb3a96bf" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:16.630641 10981 raft_consensus.cc:2454] T 28d60531970a4ab58ce5ef99cb3a96bf P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:16.632027 10619 leader_election.cc:302] T 28d60531970a4ab58ce5ef99cb3a96bf P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:16.633970 11389 raft_consensus.cc:2781] T 28d60531970a4ab58ce5ef99cb3a96bf P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:16.634389 11389 raft_consensus.cc:482] T 28d60531970a4ab58ce5ef99cb3a96bf P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:16.634815 11389 raft_consensus.cc:3037] T 28d60531970a4ab58ce5ef99cb3a96bf P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:16.644851 11389 raft_consensus.cc:504] T 28d60531970a4ab58ce5ef99cb3a96bf P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.647861 11085 tablet_bootstrap.cc:492] T 3c31682bf400494f82ee3ce5e6eec441 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:16.471985 10393 rpcz_store.cc:276] 0925 09:57:15.414072 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:15.565156 (+151084us) service_pool.cc:227] Handling call
0925 09:57:15.998295 (+433139us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:57:15.999666 (+ 1371us) catalog_manager.cc:2752] Modifying in-memory table state
0925 09:57:16.014516 (+ 14850us) catalog_manager.cc:2761] Locking tablets
0925 09:57:16.015244 (+ 728us) catalog_manager.cc:2775] Removing table and tablets from system table
0925 09:57:16.420118 (+404874us) catalog_manager.cc:2793] Removing table from by-name map
0925 09:57:16.465993 (+ 45875us) catalog_manager.cc:2809] Committing in-memory state
0925 09:57:16.466142 (+ 149us) catalog_manager.cc:2813] Aborting table tasks
0925 09:57:16.471266 (+ 5124us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:16.043578 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:16.044129 (+ 551us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:16.044164 (+ 35us) write_op.cc:435] Acquired schema lock
0925 09:57:16.044184 (+ 20us) tablet.cc:660] Decoding operations
0925 09:57:16.051165 (+ 6981us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:16.051278 (+ 113us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:16.051313 (+ 35us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:16.051923 (+ 610us) tablet.cc:699] Row locks acquired
0925 09:57:16.051943 (+ 20us) write_op.cc:260] PREPARE: finished
0925 09:57:16.052085 (+ 142us) write_op.cc:270] Start()
0925 09:57:16.052204 (+ 119us) write_op.cc:276] Timestamp: P: 1790330236051998 usec, L: 0
0925 09:57:16.052229 (+ 25us) op_driver.cc:331] REPLICATION: starting
0925 09:57:16.156596 (+104367us) log.cc:842] Serialized 722 byte log entry
0925 09:57:16.240349 (+ 83753us) op_driver.cc:444] REPLICATION: finished
0925 09:57:16.417370 (+177021us) write_op.cc:301] APPLY: starting
0925 09:57:16.417447 (+ 77us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:16.417629 (+ 182us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:16.417961 (+ 332us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:16.418265 (+ 304us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:16.418472 (+ 207us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 09:57:16.418542 (+ 70us) write_op.cc:312] APPLY: finished
0925 09:57:16.418837 (+ 295us) log.cc:842] Serialized 29 byte log entry
0925 09:57:16.419727 (+ 890us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:16.419809 (+ 82us) write_op.cc:454] Released schema lock
0925 09:57:16.419871 (+ 62us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:16.420435 (+ 564us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:16.420471 (+ 36us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":176775,"apply.run_cpu_time_us":2536,"apply.run_wall_time_us":3551,"num_ops":2,"prepare.queue_time_us":1914,"prepare.run_cpu_time_us":2782,"prepare.run_wall_time_us":114603,"replication_time_us":188036}]]}
I20260925 09:57:16.648217 11085 ts_tablet_manager.cc:1397] T 3c31682bf400494f82ee3ce5e6eec441 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.047s user 0.009s sys 0.002s
I20260925 09:57:16.650370 11085 raft_consensus.cc:348] T 3c31682bf400494f82ee3ce5e6eec441 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.650957 11085 raft_consensus.cc:374] T 3c31682bf400494f82ee3ce5e6eec441 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:16.653422 11084 tablet_bootstrap.cc:492] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:16.653805 11084 ts_tablet_manager.cc:1397] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.043s user 0.005s sys 0.005s
I20260925 09:57:16.656056 11084 raft_consensus.cc:348] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.656654 11084 raft_consensus.cc:374] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:16.656891 11084 raft_consensus.cc:729] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:16.657483 11084 consensus_queue.cc:260] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.651175 11085 raft_consensus.cc:729] T 3c31682bf400494f82ee3ce5e6eec441 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:16.670708 11084 ts_tablet_manager.cc:1428] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.017s user 0.005s sys 0.001s
I20260925 09:57:16.670651 11085 consensus_queue.cc:260] T 3c31682bf400494f82ee3ce5e6eec441 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.674085 11084 tablet_bootstrap.cc:492] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:16.675004 10385 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 1969ac471b6f471ebe8e048a96c22837 (table default.Table_ff4e7021d36b43f8b51f19960e707ac8 [id=f17a11d223204599afb24e2e9d1d9d9d]). Expected version 6 got 5
I20260925 09:57:16.680390 11245 ts_tablet_manager.cc:1918] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:16.687552 11245 log.cc:1196] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/41df75fe4b8e4856bdbb5ead3cf4fd24
I20260925 09:57:16.688262 11245 ts_tablet_manager.cc:1939] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:16.688802 11085 ts_tablet_manager.cc:1428] T 3c31682bf400494f82ee3ce5e6eec441 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.040s user 0.006s sys 0.000s
I20260925 09:57:16.683199 11084 tablet_bootstrap.cc:654] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:16.693573 11085 tablet_bootstrap.cc:492] T ef22f64036684dce9d7c69c37722f468 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:16.695466 10816 tablet_service.cc:1514] Processing DeleteTablet for tablet 41df75fe4b8e4856bdbb5ead3cf4fd24 with delete_type TABLET_DATA_DELETED (replaced by tablet c7cd1d6a603642afa2c8c737079961b5) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:16.699172 11537 raft_consensus.cc:482] T 91c09118b43b41eca8f7040323ba76f7 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:16.700156 11085 tablet_bootstrap.cc:654] T ef22f64036684dce9d7c69c37722f468 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:16.699729 11537 raft_consensus.cc:504] T 91c09118b43b41eca8f7040323ba76f7 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.705231 11537 leader_election.cc:288] T 91c09118b43b41eca8f7040323ba76f7 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:16.711419 11085 tablet_bootstrap.cc:492] T ef22f64036684dce9d7c69c37722f468 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:16.711748 11085 ts_tablet_manager.cc:1397] T ef22f64036684dce9d7c69c37722f468 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.018s user 0.014s sys 0.000s
I20260925 09:57:16.713934 11085 raft_consensus.cc:348] T ef22f64036684dce9d7c69c37722f468 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.714550 11085 raft_consensus.cc:374] T ef22f64036684dce9d7c69c37722f468 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:16.714780 11085 raft_consensus.cc:729] T ef22f64036684dce9d7c69c37722f468 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:16.715329 11085 consensus_queue.cc:260] T ef22f64036684dce9d7c69c37722f468 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.716686 11085 ts_tablet_manager.cc:1428] T ef22f64036684dce9d7c69c37722f468 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260925 09:57:16.717444 11085 tablet_bootstrap.cc:492] T 752105c1514f4c1db2a11b95b413b4f6 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:16.727962 11085 tablet_bootstrap.cc:654] T 752105c1514f4c1db2a11b95b413b4f6 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:16.747051 11537 raft_consensus.cc:482] T ef22f64036684dce9d7c69c37722f468 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:16.747727 11537 raft_consensus.cc:504] T ef22f64036684dce9d7c69c37722f468 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.751647 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef22f64036684dce9d7c69c37722f468" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:16.752310 10984 raft_consensus.cc:2454] T ef22f64036684dce9d7c69c37722f468 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
I20260925 09:57:16.753619 10772 leader_election.cc:302] T ef22f64036684dce9d7c69c37722f468 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:16.754639 11526 raft_consensus.cc:2781] T ef22f64036684dce9d7c69c37722f468 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:16.755038 11526 raft_consensus.cc:482] T ef22f64036684dce9d7c69c37722f468 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:16.755419 11526 raft_consensus.cc:3037] T ef22f64036684dce9d7c69c37722f468 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:16.759434 11543 raft_consensus.cc:482] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:16.759855 11543 raft_consensus.cc:504] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.761272 11537 leader_election.cc:288] T ef22f64036684dce9d7c69c37722f468 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:16.761756 11543 leader_election.cc:288] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:16.776549 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28d60531970a4ab58ce5ef99cb3a96bf" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:16.777370 10836 raft_consensus.cc:2454] T 28d60531970a4ab58ce5ef99cb3a96bf P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:16.780115 11361 raft_consensus.cc:504] T 47a9fb9392244e1d89f8af2bfb42edd6 P c43461c4ee854ae1ac52b873ccada318 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.664983 10681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47a9fb9392244e1d89f8af2bfb42edd6" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:16.693869 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 41df75fe4b8e4856bdbb5ead3cf4fd24 (table default.Table_0507f47042b9445abaf8671c979f8994 [id=55860c5dba9f4c818cfa6d523041ec5f]) successfully deleted
W20260925 09:57:16.797001 10377 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 103159us
I20260925 09:57:16.782711 10393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_698f813273574f79887bdaac95b977ce"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:16.801241 10393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_698f813273574f79887bdaac95b977ce in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 09:57:16.781096 10772 leader_election.cc:341] T 91c09118b43b41eca8f7040323ba76f7 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 91c09118b43b41eca8f7040323ba76f7
I20260925 09:57:16.781816 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b555eeba505740988c704df5c734e7c0" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:16.804889 10918 leader_election.cc:341] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:16.805200 11084 tablet_bootstrap.cc:492] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:16.805557 11084 ts_tablet_manager.cc:1397] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.132s user 0.019s sys 0.024s
I20260925 09:57:16.807632 11084 raft_consensus.cc:348] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.808290 11084 raft_consensus.cc:374] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:16.808557 11084 raft_consensus.cc:729] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:16.767287 11526 raft_consensus.cc:504] T ef22f64036684dce9d7c69c37722f468 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.809099 11084 consensus_queue.cc:260] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.816190 11084 ts_tablet_manager.cc:1428] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.010s user 0.006s sys 0.001s
I20260925 09:57:16.817289 11084 tablet_bootstrap.cc:492] T cf35f39bcd6a41128acb2a4936fcc316 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:16.822084 11084 tablet_bootstrap.cc:654] T cf35f39bcd6a41128acb2a4936fcc316 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:16.839769 10981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef22f64036684dce9d7c69c37722f468" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:16.840547 10981 raft_consensus.cc:3037] T ef22f64036684dce9d7c69c37722f468 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:16.844967 11084 tablet_bootstrap.cc:492] T cf35f39bcd6a41128acb2a4936fcc316 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:16.845362 11084 ts_tablet_manager.cc:1397] T cf35f39bcd6a41128acb2a4936fcc316 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.028s user 0.010s sys 0.000s
I20260925 09:57:16.846396 11526 leader_election.cc:288] T ef22f64036684dce9d7c69c37722f468 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
W20260925 09:57:16.793268 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 41df75fe4b8e4856bdbb5ead3cf4fd24 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 41df75fe4b8e4856bdbb5ead3cf4fd24 already in progress: creating tablet
W20260925 09:57:16.847544 10379 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 54320us
I20260925 09:57:16.847765 11084 raft_consensus.cc:348] T cf35f39bcd6a41128acb2a4936fcc316 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.848338 11084 raft_consensus.cc:374] T cf35f39bcd6a41128acb2a4936fcc316 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:16.848562 11084 raft_consensus.cc:729] T cf35f39bcd6a41128acb2a4936fcc316 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:16.711130 10390 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_055cf78170f24a8e94074c1f02e2f5a3" } new_table_name: "default.Table_48f368d811494119be0dc534e2fdcfbd" modify_external_catalogs: true
I20260925 09:57:16.798399 10385 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet c07c6205d1284a6fbb369c010a226da7 (table default.Table_5a6ecccec7f44c31bbfe8d64580ec33b [id=2d6bc0e2aaee4b1483b01fc678505bb0]). Expected version 1 got 0
I20260925 09:57:16.849133 11084 consensus_queue.cc:260] T cf35f39bcd6a41128acb2a4936fcc316 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.849521 10385 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 28d60531970a4ab58ce5ef99cb3a96bf (table default.Table_ce09d3a8d8e84973894f33b18bbfdb22 [id=6851500c7dba44f1bf353bb79bafd97a]). Expected version 2 got 0
I20260925 09:57:16.851635 11084 ts_tablet_manager.cc:1428] T cf35f39bcd6a41128acb2a4936fcc316 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260925 09:57:16.852573 11084 tablet_bootstrap.cc:492] T c8522c08bdd94806b5c84f5a41ca5bf4 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:16.858945 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47a9fb9392244e1d89f8af2bfb42edd6" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:16.876051 11084 tablet_bootstrap.cc:654] T c8522c08bdd94806b5c84f5a41ca5bf4 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:16.876188 10984 raft_consensus.cc:3037] T 47a9fb9392244e1d89f8af2bfb42edd6 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:57:16.889897 11361 leader_election.cc:288] T 47a9fb9392244e1d89f8af2bfb42edd6 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:16.668136 11505 consensus_queue.cc:1035] T c07c6205d1284a6fbb369c010a226da7 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:16.669602 11497 raft_consensus.cc:504] T 3be0eccc65564bfba3d6e5c398c61fb6 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.896214 10683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91c09118b43b41eca8f7040323ba76f7" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:16.896942 10683 raft_consensus.cc:2454] T 91c09118b43b41eca8f7040323ba76f7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
I20260925 09:57:16.909821 11084 tablet_bootstrap.cc:492] T c8522c08bdd94806b5c84f5a41ca5bf4 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:16.910236 11084 ts_tablet_manager.cc:1397] T c8522c08bdd94806b5c84f5a41ca5bf4 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.058s user 0.013s sys 0.000s
I20260925 09:57:16.912400 11084 raft_consensus.cc:348] T c8522c08bdd94806b5c84f5a41ca5bf4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.913081 11084 raft_consensus.cc:374] T c8522c08bdd94806b5c84f5a41ca5bf4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:16.913342 11084 raft_consensus.cc:729] T c8522c08bdd94806b5c84f5a41ca5bf4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:16.920710 11084 consensus_queue.cc:260] T c8522c08bdd94806b5c84f5a41ca5bf4 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.925382 11084 ts_tablet_manager.cc:1428] T c8522c08bdd94806b5c84f5a41ca5bf4 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.015s user 0.006s sys 0.000s
I20260925 09:57:16.926249 11084 tablet_bootstrap.cc:492] T ec1b78e61e8d46cb8b87f60786059658 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:16.935130 11505 consensus_queue.cc:1035] T 47a9fb9392244e1d89f8af2bfb42edd6 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:16.942369 11084 tablet_bootstrap.cc:654] T ec1b78e61e8d46cb8b87f60786059658 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:16.946687 10683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b555eeba505740988c704df5c734e7c0" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:16.948318 10919 leader_election.cc:341] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:16.948798 10919 leader_election.cc:302] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:16.950008 11543 raft_consensus.cc:2726] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:16.955168 11497 leader_election.cc:288] T 3be0eccc65564bfba3d6e5c398c61fb6 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:16.957772 10980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3be0eccc65564bfba3d6e5c398c61fb6" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:16.958472 10980 raft_consensus.cc:3037] T 3be0eccc65564bfba3d6e5c398c61fb6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:16.967841 10775 leader_election.cc:302] T 91c09118b43b41eca8f7040323ba76f7 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters: be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:16.968487 11084 tablet_bootstrap.cc:492] T ec1b78e61e8d46cb8b87f60786059658 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:16.968866 11084 ts_tablet_manager.cc:1397] T ec1b78e61e8d46cb8b87f60786059658 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.043s user 0.008s sys 0.002s
I20260925 09:57:16.971194 11475 consensus_queue.cc:1035] T c07c6205d1284a6fbb369c010a226da7 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:16.977571 11084 raft_consensus.cc:348] T ec1b78e61e8d46cb8b87f60786059658 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.978339 11084 raft_consensus.cc:374] T ec1b78e61e8d46cb8b87f60786059658 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:16.978571 11084 raft_consensus.cc:729] T ec1b78e61e8d46cb8b87f60786059658 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:16.979171 11084 consensus_queue.cc:260] T ec1b78e61e8d46cb8b87f60786059658 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:16.980672 11389 leader_election.cc:288] T 28d60531970a4ab58ce5ef99cb3a96bf P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:16.982091 11084 ts_tablet_manager.cc:1428] T ec1b78e61e8d46cb8b87f60786059658 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.013s user 0.003s sys 0.003s
I20260925 09:57:16.984577 10982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28d60531970a4ab58ce5ef99cb3a96bf" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:16.985090 10982 raft_consensus.cc:3037] T 28d60531970a4ab58ce5ef99cb3a96bf P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:16.991072 10982 raft_consensus.cc:2454] T 28d60531970a4ab58ce5ef99cb3a96bf P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:16.992486 11084 tablet_bootstrap.cc:492] T 3706253e53d841aeaa43889ead13af32 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:17.002429 11084 tablet_bootstrap.cc:654] T 3706253e53d841aeaa43889ead13af32 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:17.017429 11084 tablet_bootstrap.cc:492] T 3706253e53d841aeaa43889ead13af32 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:17.017809 11084 ts_tablet_manager.cc:1397] T 3706253e53d841aeaa43889ead13af32 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.025s user 0.004s sys 0.004s
I20260925 09:57:17.019951 11084 raft_consensus.cc:348] T 3706253e53d841aeaa43889ead13af32 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.014590 10619 leader_election.cc:302] T 28d60531970a4ab58ce5ef99cb3a96bf P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:17.020570 11084 raft_consensus.cc:374] T 3706253e53d841aeaa43889ead13af32 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:17.020804 11084 raft_consensus.cc:729] T 3706253e53d841aeaa43889ead13af32 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:17.021389 11084 consensus_queue.cc:260] T 3706253e53d841aeaa43889ead13af32 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.021658 11475 raft_consensus.cc:2781] T 28d60531970a4ab58ce5ef99cb3a96bf P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:17.022354 11475 raft_consensus.cc:686] T 28d60531970a4ab58ce5ef99cb3a96bf P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:17.023351 11475 consensus_queue.cc:237] T 28d60531970a4ab58ce5ef99cb3a96bf P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.029368 11084 ts_tablet_manager.cc:1428] T 3706253e53d841aeaa43889ead13af32 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20260925 09:57:17.035647 11084 tablet_bootstrap.cc:492] T 93e1b5837aff48d7907890c40dfcd0d0 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:17.037999 11082 tablet_bootstrap.cc:492] T ef22f64036684dce9d7c69c37722f468 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:17.038662 11082 ts_tablet_manager.cc:1397] T ef22f64036684dce9d7c69c37722f468 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.718s user 0.015s sys 0.002s
I20260925 09:57:17.041344 11082 raft_consensus.cc:348] T ef22f64036684dce9d7c69c37722f468 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.046532 11084 tablet_bootstrap.cc:654] T 93e1b5837aff48d7907890c40dfcd0d0 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:17.064090 11084 tablet_bootstrap.cc:492] T 93e1b5837aff48d7907890c40dfcd0d0 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:17.063907 11543 raft_consensus.cc:482] T cf35f39bcd6a41128acb2a4936fcc316 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:17.064447 11543 raft_consensus.cc:504] T cf35f39bcd6a41128acb2a4936fcc316 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.067065 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf35f39bcd6a41128acb2a4936fcc316" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:17.068341 10918 leader_election.cc:341] T cf35f39bcd6a41128acb2a4936fcc316 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:17.069319 11543 leader_election.cc:288] T cf35f39bcd6a41128acb2a4936fcc316 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:17.069878 11084 ts_tablet_manager.cc:1397] T 93e1b5837aff48d7907890c40dfcd0d0 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.039s user 0.012s sys 0.000s
I20260925 09:57:17.072023 11084 raft_consensus.cc:348] T 93e1b5837aff48d7907890c40dfcd0d0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.072748 11084 raft_consensus.cc:374] T 93e1b5837aff48d7907890c40dfcd0d0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:17.073015 11084 raft_consensus.cc:729] T 93e1b5837aff48d7907890c40dfcd0d0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:17.073634 11084 consensus_queue.cc:260] T 93e1b5837aff48d7907890c40dfcd0d0 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.076853 11084 ts_tablet_manager.cc:1428] T 93e1b5837aff48d7907890c40dfcd0d0 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20260925 09:57:17.080087 11084 tablet_bootstrap.cc:492] T 3d652a76d96f4dbeafde15f3c16534a6 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:17.091617 11084 tablet_bootstrap.cc:654] T 3d652a76d96f4dbeafde15f3c16534a6 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:17.095057 11526 raft_consensus.cc:2781] T 91c09118b43b41eca8f7040323ba76f7 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:17.095604 11526 raft_consensus.cc:482] T 91c09118b43b41eca8f7040323ba76f7 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:17.096040 11526 raft_consensus.cc:3037] T 91c09118b43b41eca8f7040323ba76f7 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:17.111737 11084 tablet_bootstrap.cc:492] T 3d652a76d96f4dbeafde15f3c16534a6 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:17.112119 11543 raft_consensus.cc:482] T 67ce2e3325214aa6bdc3347e9e8f82b0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260925 09:57:17.111562 10774 outbound_call.cc:322] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.10.9.62:34455, user_credentials={real_user=slave}} blocked reactor thread for 227714us
I20260925 09:57:17.112538 11543 raft_consensus.cc:504] T 67ce2e3325214aa6bdc3347e9e8f82b0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.113574 11537 raft_consensus.cc:482] T 3be0eccc65564bfba3d6e5c398c61fb6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:17.114264 11543 leader_election.cc:288] T 67ce2e3325214aa6bdc3347e9e8f82b0 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:17.112104 11084 ts_tablet_manager.cc:1397] T 3d652a76d96f4dbeafde15f3c16534a6 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.032s user 0.007s sys 0.004s
I20260925 09:57:17.115804 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67ce2e3325214aa6bdc3347e9e8f82b0" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:17.117682 10918 leader_election.cc:341] T 67ce2e3325214aa6bdc3347e9e8f82b0 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:17.114126 11537 raft_consensus.cc:504] T 3be0eccc65564bfba3d6e5c398c61fb6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.122450 10979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3be0eccc65564bfba3d6e5c398c61fb6" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:17.121891 11537 leader_election.cc:288] T 3be0eccc65564bfba3d6e5c398c61fb6 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:16.856071 10385 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet ef6fefaa76d44fe798201aab2f21b9c7 (table default.Table_46e4c497898b454c85b7c2a299ebf3d9 [id=80b48dd413e3446eb7ac65e5f957b5dc]). Expected version 2 got 0
I20260925 09:57:17.125653 10385 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet e57065f2bb224a56ad1c3518a0c375da (table default.Table_3a00773e32a34a41bead0b4116e4c6df [id=7889081642604700bbc6d253ab105260]). Expected version 1 got 0
I20260925 09:57:17.134047 11543 raft_consensus.cc:482] T 3706253e53d841aeaa43889ead13af32 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:17.134461 11543 raft_consensus.cc:504] T 3706253e53d841aeaa43889ead13af32 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.136685 11543 leader_election.cc:288] T 3706253e53d841aeaa43889ead13af32 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:17.137976 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3706253e53d841aeaa43889ead13af32" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:17.139240 10918 leader_election.cc:341] T 3706253e53d841aeaa43889ead13af32 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:17.117448 11084 raft_consensus.cc:348] T 3d652a76d96f4dbeafde15f3c16534a6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.179477 11084 raft_consensus.cc:374] T 3d652a76d96f4dbeafde15f3c16534a6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:17.179833 11084 raft_consensus.cc:729] T 3d652a76d96f4dbeafde15f3c16534a6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:17.180706 11084 consensus_queue.cc:260] T 3d652a76d96f4dbeafde15f3c16534a6 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.186383 11543 raft_consensus.cc:482] T ec1b78e61e8d46cb8b87f60786059658 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:17.186786 11543 raft_consensus.cc:504] T ec1b78e61e8d46cb8b87f60786059658 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.188503 11543 leader_election.cc:288] T ec1b78e61e8d46cb8b87f60786059658 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:17.189899 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec1b78e61e8d46cb8b87f60786059658" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:17.191145 10918 leader_election.cc:341] T ec1b78e61e8d46cb8b87f60786059658 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:17.191776 11543 raft_consensus.cc:482] T 3d652a76d96f4dbeafde15f3c16534a6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:17.192188 11543 raft_consensus.cc:504] T 3d652a76d96f4dbeafde15f3c16534a6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.194895 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d652a76d96f4dbeafde15f3c16534a6" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:17.196086 10918 leader_election.cc:341] T 3d652a76d96f4dbeafde15f3c16534a6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:17.198139 11543 leader_election.cc:288] T 3d652a76d96f4dbeafde15f3c16534a6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:17.208773 10835 raft_consensus.cc:1226] T 47a9fb9392244e1d89f8af2bfb42edd6 P c43461c4ee854ae1ac52b873ccada318 [term 2 FOLLOWER]: Rejecting Update request from peer b8850818e2ee42a082802d0095c08d21 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260925 09:57:17.213033 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3be0eccc65564bfba3d6e5c398c61fb6" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:17.213668 10836 raft_consensus.cc:3037] T 3be0eccc65564bfba3d6e5c398c61fb6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:17.222419 10836 raft_consensus.cc:2454] T 3be0eccc65564bfba3d6e5c398c61fb6 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:17.238384 11543 raft_consensus.cc:482] T c8522c08bdd94806b5c84f5a41ca5bf4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:17.238953 11543 raft_consensus.cc:504] T c8522c08bdd94806b5c84f5a41ca5bf4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.241281 11543 leader_election.cc:288] T c8522c08bdd94806b5c84f5a41ca5bf4 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:17.241921 11361 raft_consensus.cc:482] T 2d55154ceb064f22bd6f37c8ae5dce74 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:17.242379 11361 raft_consensus.cc:504] T 2d55154ceb064f22bd6f37c8ae5dce74 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.245165 11361 leader_election.cc:288] T 2d55154ceb064f22bd6f37c8ae5dce74 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
W20260925 09:57:17.246168 10772 leader_election.cc:341] T 2d55154ceb064f22bd6f37c8ae5dce74 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 2d55154ceb064f22bd6f37c8ae5dce74
W20260925 09:57:17.246526 11085 thread.cc:636] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.517s user 0.051s sys 0.171s
W20260925 09:57:17.246763 11085 thread.cc:602] log-alloc [worker] (thread pool) Time spent starting thread: real 0.517s user 0.051s sys 0.171s
I20260925 09:57:17.255905 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8522c08bdd94806b5c84f5a41ca5bf4" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:17.257534 10918 leader_election.cc:341] T c8522c08bdd94806b5c84f5a41ca5bf4 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:17.276935 11085 tablet_bootstrap.cc:492] T 752105c1514f4c1db2a11b95b413b4f6 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:17.277453 11085 ts_tablet_manager.cc:1397] T 752105c1514f4c1db2a11b95b413b4f6 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.560s user 0.055s sys 0.173s
I20260925 09:57:17.280534 11085 raft_consensus.cc:348] T 752105c1514f4c1db2a11b95b413b4f6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.281536 11085 raft_consensus.cc:374] T 752105c1514f4c1db2a11b95b413b4f6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:17.281857 11085 raft_consensus.cc:729] T 752105c1514f4c1db2a11b95b413b4f6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:17.283178 11085 consensus_queue.cc:260] T 752105c1514f4c1db2a11b95b413b4f6 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.286716 11085 ts_tablet_manager.cc:1428] T 752105c1514f4c1db2a11b95b413b4f6 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260925 09:57:17.295483 11543 raft_consensus.cc:482] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:17.295913 11543 raft_consensus.cc:504] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.298424 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa619b699004882a82976f0ef8204ad" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:17.299757 10918 leader_election.cc:341] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:17.300745 11543 leader_election.cc:288] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:17.295159 10391 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35140 (request call id 167) took 1199 ms. Trace:
I20260925 09:57:17.296972 10389 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 1969ac471b6f471ebe8e048a96c22837
I20260925 09:57:17.308837 10958 tablet_service.cc:1514] Processing DeleteTablet for tablet 1969ac471b6f471ebe8e048a96c22837 with delete_type TABLET_DATA_DELETED (replaced by tablet 977a59175e5b43eeb6f23b938afd00a3) from {username='slave'} at 127.0.0.1:53560
W20260925 09:57:17.043007 10658 consensus_meta.cc:287] T 7360a869f3be45a884a0379f4ef78e85 P b8850818e2ee42a082802d0095c08d21: Time spent flushing consensus metadata: real 0.518s user 0.002s sys 0.007s
I20260925 09:57:17.316361 10658 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37876 (request call id 120) took 1230 ms. Trace:
I20260925 09:57:17.316782 10658 rpcz_store.cc:276] 0925 09:57:16.086257 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:16.086396 (+ 139us) service_pool.cc:227] Handling call
0925 09:57:16.088020 (+ 1624us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0925 09:57:16.088127 (+ 107us) ts_tablet_manager.cc:676] Creating new metadata...
0925 09:57:16.416829 (+328702us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:17.315807 (+898978us) tablet_replica.cc:205] Creating consensus instance
0925 09:57:17.316289 (+ 482us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260925 09:57:17.056291 11458 ts_tablet_manager.cc:1918] T 0e882284a014455192375daad391efaf P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:17.073230 10683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf35f39bcd6a41128acb2a4936fcc316" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:17.318589 10677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d55154ceb064f22bd6f37c8ae5dce74" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:17.119223 10681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67ce2e3325214aa6bdc3347e9e8f82b0" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:17.139360 10680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3706253e53d841aeaa43889ead13af32" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:17.189859 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec1b78e61e8d46cb8b87f60786059658" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:17.266629 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d652a76d96f4dbeafde15f3c16534a6" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:17.319855 10919 leader_election.cc:341] T cf35f39bcd6a41128acb2a4936fcc316 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:17.267616 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8522c08bdd94806b5c84f5a41ca5bf4" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:17.320272 10919 leader_election.cc:302] T cf35f39bcd6a41128acb2a4936fcc316 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:17.292474 10682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef22f64036684dce9d7c69c37722f468" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:17.302865 10685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef22f64036684dce9d7c69c37722f468" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:17.303825 10688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa619b699004882a82976f0ef8204ad" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:17.310549 10659 tablet_service.cc:1514] Processing DeleteTablet for tablet 1969ac471b6f471ebe8e048a96c22837 with delete_type TABLET_DATA_DELETED (replaced by tablet 977a59175e5b43eeb6f23b938afd00a3) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:17.312662 10679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47a9fb9392244e1d89f8af2bfb42edd6" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:17.313710 10678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3be0eccc65564bfba3d6e5c398c61fb6" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:17.321982 11543 raft_consensus.cc:2726] T cf35f39bcd6a41128acb2a4936fcc316 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:17.322436 10678 raft_consensus.cc:2379] T 3be0eccc65564bfba3d6e5c398c61fb6 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c43461c4ee854ae1ac52b873ccada318 in current term 1: Already voted for candidate b8850818e2ee42a082802d0095c08d21 in this term.
I20260925 09:57:17.178017 10980 raft_consensus.cc:2454] T 3be0eccc65564bfba3d6e5c398c61fb6 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
W20260925 09:57:17.322662 10919 leader_election.cc:341] T 67ce2e3325214aa6bdc3347e9e8f82b0 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:17.323302 11562 tablet_replica.cc:330] T 1969ac471b6f471ebe8e048a96c22837 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:17.323707 11458 log.cc:1196] T 0e882284a014455192375daad391efaf P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/0e882284a014455192375daad391efaf
I20260925 09:57:17.323855 10619 leader_election.cc:302] T 3be0eccc65564bfba3d6e5c398c61fb6 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:17.324884 11082 raft_consensus.cc:374] T ef22f64036684dce9d7c69c37722f468 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:17.324635 11458 ts_tablet_manager.cc:1939] T 0e882284a014455192375daad391efaf P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:17.325008 11553 raft_consensus.cc:2781] T 3be0eccc65564bfba3d6e5c398c61fb6 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:17.325663 11082 raft_consensus.cc:729] T ef22f64036684dce9d7c69c37722f468 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:17.325791 11553 raft_consensus.cc:686] T 3be0eccc65564bfba3d6e5c398c61fb6 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:17.327123 11082 consensus_queue.cc:260] T ef22f64036684dce9d7c69c37722f468 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.327296 11553 consensus_queue.cc:237] T 3be0eccc65564bfba3d6e5c398c61fb6 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.327950 10682 raft_consensus.cc:2454] T ef22f64036684dce9d7c69c37722f468 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
I20260925 09:57:17.332370 11085 tablet_bootstrap.cc:492] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:17.333391 11082 ts_tablet_manager.cc:1428] T ef22f64036684dce9d7c69c37722f468 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.294s user 0.005s sys 0.000s
I20260925 09:57:17.334136 11082 tablet_bootstrap.cc:492] T 2d55154ceb064f22bd6f37c8ae5dce74 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:17.323173 10919 leader_election.cc:302] T 67ce2e3325214aa6bdc3347e9e8f82b0 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:17.325166 11084 ts_tablet_manager.cc:1428] T 3d652a76d96f4dbeafde15f3c16534a6 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.210s user 0.004s sys 0.003s
I20260925 09:57:17.335847 11543 raft_consensus.cc:2726] T 67ce2e3325214aa6bdc3347e9e8f82b0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:17.336334 10919 leader_election.cc:341] T 3706253e53d841aeaa43889ead13af32 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:17.336690 11084 tablet_bootstrap.cc:492] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:17.336771 10919 leader_election.cc:302] T 3706253e53d841aeaa43889ead13af32 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:17.337276 11526 raft_consensus.cc:504] T 91c09118b43b41eca8f7040323ba76f7 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.338102 11543 raft_consensus.cc:2726] T 3706253e53d841aeaa43889ead13af32 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:17.338406 10984 raft_consensus.cc:2454] T 47a9fb9392244e1d89f8af2bfb42edd6 P be97ab5d75934a648d4ab76341193ec0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 2.
W20260925 09:57:17.338774 10919 leader_election.cc:341] T ec1b78e61e8d46cb8b87f60786059658 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:17.339082 10983 raft_consensus.cc:1226] T 47a9fb9392244e1d89f8af2bfb42edd6 P be97ab5d75934a648d4ab76341193ec0 [term 2 FOLLOWER]: Rejecting Update request from peer b8850818e2ee42a082802d0095c08d21 for earlier term 1. Current term is 2. Ops: []
I20260925 09:57:17.339175 10919 leader_election.cc:302] T ec1b78e61e8d46cb8b87f60786059658 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:17.339890 11543 raft_consensus.cc:2726] T ec1b78e61e8d46cb8b87f60786059658 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:17.340256 10919 leader_election.cc:341] T 3d652a76d96f4dbeafde15f3c16534a6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:17.340611 10919 leader_election.cc:302] T 3d652a76d96f4dbeafde15f3c16534a6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:17.341804 11562 raft_consensus.cc:2229] T 1969ac471b6f471ebe8e048a96c22837 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:17.342283 11562 raft_consensus.cc:2258] T 1969ac471b6f471ebe8e048a96c22837 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:17.343194 11475 consensus_queue.cc:1046] T 47a9fb9392244e1d89f8af2bfb42edd6 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:17.344065 11562 ts_tablet_manager.cc:1905] T 1969ac471b6f471ebe8e048a96c22837 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:17.344296 11475 raft_consensus.cc:3032] T 47a9fb9392244e1d89f8af2bfb42edd6 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Stepping down as leader of term 1
I20260925 09:57:17.344623 11475 raft_consensus.cc:729] T 47a9fb9392244e1d89f8af2bfb42edd6 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:17.344529 11543 raft_consensus.cc:2726] T 3d652a76d96f4dbeafde15f3c16534a6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:17.345023 10919 leader_election.cc:341] T c8522c08bdd94806b5c84f5a41ca5bf4 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:17.345523 10919 leader_election.cc:302] T c8522c08bdd94806b5c84f5a41ca5bf4 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:17.345387 11475 consensus_queue.cc:260] T 47a9fb9392244e1d89f8af2bfb42edd6 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.346423 11475 raft_consensus.cc:3037] T 47a9fb9392244e1d89f8af2bfb42edd6 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Advancing to term 2
W20260925 09:57:17.346565 10919 leader_election.cc:341] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:17.347023 10919 leader_election.cc:302] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:17.347299 11543 raft_consensus.cc:2726] T c8522c08bdd94806b5c84f5a41ca5bf4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:17.326427 11458 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 117) took 1660 ms. Trace:
I20260925 09:57:17.348150 11458 rpcz_store.cc:276] 0925 09:57:15.665833 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:15.665968 (+ 135us) service_pool.cc:227] Handling call
0925 09:57:16.048465 (+382497us) ts_tablet_manager.cc:1145] Deleting tablet 0e882284a014455192375daad391efaf
0925 09:57:16.048497 (+ 32us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:16.530685 (+482188us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:17.055991 (+525306us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:17.326368 (+270377us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":162602,"tablet-delete.queue_time_us":381825}
I20260925 09:57:17.349807 11543 raft_consensus.cc:2726] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:17.350838 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 0e882284a014455192375daad391efaf (table default.Table_62ea48a919ec4159bcc86ebb048e3545 [id=051a096f481f4f20a59081fbd08b3b1f]) successfully deleted
I20260925 09:57:17.354404 11458 tablet_replica.cc:330] T 1969ac471b6f471ebe8e048a96c22837 P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:17.346594 10772 leader_election.cc:302] T 47a9fb9392244e1d89f8af2bfb42edd6 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters:
W20260925 09:57:17.367189 10981 consensus_meta.cc:287] T ef22f64036684dce9d7c69c37722f468 P be97ab5d75934a648d4ab76341193ec0: Time spent flushing consensus metadata: real 0.526s user 0.009s sys 0.000s
I20260925 09:57:17.368084 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28d60531970a4ab58ce5ef99cb3a96bf" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:17.368674 10835 raft_consensus.cc:3037] T 28d60531970a4ab58ce5ef99cb3a96bf P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:17.376027 10835 raft_consensus.cc:2454] T 28d60531970a4ab58ce5ef99cb3a96bf P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:17.380858 11526 leader_election.cc:288] T 91c09118b43b41eca8f7040323ba76f7 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:17.375197 11562 ts_tablet_manager.cc:1918] T 1969ac471b6f471ebe8e048a96c22837 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 09:57:17.391616 11458 raft_consensus.cc:2229] T 1969ac471b6f471ebe8e048a96c22837 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:17.391737 11562 log.cc:1196] T 1969ac471b6f471ebe8e048a96c22837 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/1969ac471b6f471ebe8e048a96c22837
I20260925 09:57:17.392669 11562 ts_tablet_manager.cc:1939] T 1969ac471b6f471ebe8e048a96c22837 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:17.395807 10682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91c09118b43b41eca8f7040323ba76f7" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:17.396628 10682 raft_consensus.cc:3037] T 91c09118b43b41eca8f7040323ba76f7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:57:17.404604 10772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 58091.5us
I20260925 09:57:17.392607 11458 raft_consensus.cc:2258] T 1969ac471b6f471ebe8e048a96c22837 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:17.407410 11537 raft_consensus.cc:2781] T 47a9fb9392244e1d89f8af2bfb42edd6 P c43461c4ee854ae1ac52b873ccada318 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 09:57:17.407831 11537 raft_consensus.cc:686] T 47a9fb9392244e1d89f8af2bfb42edd6 P c43461c4ee854ae1ac52b873ccada318 [term 2 LEADER]: Becoming Leader. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Running, Role: LEADER
I20260925 09:57:17.409139 11458 ts_tablet_manager.cc:1905] T 1969ac471b6f471ebe8e048a96c22837 P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:17.408782 11537 consensus_queue.cc:237] T 47a9fb9392244e1d89f8af2bfb42edd6 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.411060 10682 raft_consensus.cc:2454] T 91c09118b43b41eca8f7040323ba76f7 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
W20260925 09:57:17.412748 10772 leader_election.cc:341] T 91c09118b43b41eca8f7040323ba76f7 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 91c09118b43b41eca8f7040323ba76f7
I20260925 09:57:17.412987 11475 consensus_queue.cc:1035] T c16e096038084b2b9148390b6e773294 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:17.432786 11082 tablet_bootstrap.cc:654] T 2d55154ceb064f22bd6f37c8ae5dce74 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:17.367590 10981 raft_consensus.cc:2454] T ef22f64036684dce9d7c69c37722f468 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:17.440610 10772 leader_election.cc:302] T ef22f64036684dce9d7c69c37722f468 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:17.441658 11565 raft_consensus.cc:482] T c588b0c05a1a4f3ea545066183e76502 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:17.441710 11526 raft_consensus.cc:2781] T ef22f64036684dce9d7c69c37722f468 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:17.442413 11526 raft_consensus.cc:686] T ef22f64036684dce9d7c69c37722f468 P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Becoming Leader. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Running, Role: LEADER
I20260925 09:57:17.442214 11565 raft_consensus.cc:504] T c588b0c05a1a4f3ea545066183e76502 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.443378 11526 consensus_queue.cc:237] T ef22f64036684dce9d7c69c37722f468 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.444418 11565 leader_election.cc:288] T c588b0c05a1a4f3ea545066183e76502 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:17.445092 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c588b0c05a1a4f3ea545066183e76502" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:17.445783 10682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c588b0c05a1a4f3ea545066183e76502" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:17.446260 10918 leader_election.cc:341] T c588b0c05a1a4f3ea545066183e76502 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:17.446995 10919 leader_election.cc:341] T c588b0c05a1a4f3ea545066183e76502 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:17.447350 10919 leader_election.cc:302] T c588b0c05a1a4f3ea545066183e76502 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:17.448379 11565 raft_consensus.cc:2726] T c588b0c05a1a4f3ea545066183e76502 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:17.307055 10391 rpcz_store.cc:276] 0925 09:57:16.095991 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:16.236401 (+140410us) service_pool.cc:227] Handling call
0925 09:57:16.494198 (+257797us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2adb156 00007fabf0618034 00007fabf0617fc2 00007fabf0617ef1 00007fabf0617e6c 00007fabf0617122 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:16.497731 (+ 3533us) catalog_manager.cc:2075] Acquired catalog manager lock
0925 09:57:16.723205 (+225474us) catalog_manager.cc:2121] Created new table and tablet info
0925 09:57:16.795987 (+ 72782us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:16.796028 (+ 41us) write_op.cc:454] Released schema lock
0925 09:57:16.796286 (+ 258us) catalog_manager.cc:2181] Wrote table and tablets to system table
0925 09:57:17.294782 (+498496us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0925 09:57:17.295097 (+ 315us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:16.725022 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:16.725543 (+ 521us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:16.725568 (+ 25us) write_op.cc:435] Acquired schema lock
0925 09:57:16.725580 (+ 12us) tablet.cc:660] Decoding operations
0925 09:57:16.725859 (+ 279us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:16.725897 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:16.725917 (+ 20us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:16.726109 (+ 192us) tablet.cc:699] Row locks acquired
0925 09:57:16.726125 (+ 16us) write_op.cc:260] PREPARE: finished
0925 09:57:16.729891 (+ 3766us) write_op.cc:270] Start()
0925 09:57:16.729972 (+ 81us) write_op.cc:276] Timestamp: P: 1790330236729830 usec, L: 0
0925 09:57:16.729989 (+ 17us) op_driver.cc:331] REPLICATION: starting
0925 09:57:16.730786 (+ 797us) log.cc:842] Serialized 433 byte log entry
0925 09:57:16.740569 (+ 9783us) op_driver.cc:444] REPLICATION: finished
0925 09:57:16.771425 (+ 30856us) write_op.cc:301] APPLY: starting
0925 09:57:16.771497 (+ 72us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:16.771630 (+ 133us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:16.771796 (+ 166us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:16.772370 (+ 574us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:16.772562 (+ 192us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 09:57:16.772622 (+ 60us) write_op.cc:312] APPLY: finished
0925 09:57:16.772851 (+ 229us) log.cc:842] Serialized 29 byte log entry
0925 09:57:16.773721 (+ 870us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:16.773902 (+ 181us) write_op.cc:454] Released schema lock
0925 09:57:16.773945 (+ 43us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":590471936,"child_traces":[["op",{"apply.queue_time_us":30632,"apply.run_cpu_time_us":1936,"apply.run_wall_time_us":2795,"num_ops":2,"prepare.queue_time_us":117,"prepare.run_cpu_time_us":2281,"prepare.run_wall_time_us":45002,"raft.queue_time_us":96,"raft.run_cpu_time_us":691,"raft.run_wall_time_us":1565,"replication_time_us":10499}]]}
I20260925 09:57:17.459693 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 1969ac471b6f471ebe8e048a96c22837 (table default.Table_ff4e7021d36b43f8b51f19960e707ac8 [id=f17a11d223204599afb24e2e9d1d9d9d]) successfully deleted
W20260925 09:57:17.463552 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet cf35f39bcd6a41128acb2a4936fcc316 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cf35f39bcd6a41128acb2a4936fcc316 already in progress: creating tablet
W20260925 09:57:17.464584 10380 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet c07c6205d1284a6fbb369c010a226da7 (table default.Table_5a6ecccec7f44c31bbfe8d64580ec33b [id=2d6bc0e2aaee4b1483b01fc678505bb0], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet c07c6205d1284a6fbb369c010a226da7 consensus state has no leader
W20260925 09:57:17.465015 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 41df75fe4b8e4856bdbb5ead3cf4fd24 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 41df75fe4b8e4856bdbb5ead3cf4fd24 already in progress: creating tablet
I20260925 09:57:17.476894 10393 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35140 (request call id 168) took 1295 ms. Trace:
W20260925 09:57:17.481458 10775 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 67204us
W20260925 09:57:17.482518 10775 leader_election.cc:341] T 2d55154ceb064f22bd6f37c8ae5dce74 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:17.482900 10775 leader_election.cc:302] T 2d55154ceb064f22bd6f37c8ae5dce74 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:17.483553 11361 raft_consensus.cc:2726] T 2d55154ceb064f22bd6f37c8ae5dce74 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:17.484946 10775 leader_election.cc:302] T 3be0eccc65564bfba3d6e5c398c61fb6 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:17.485635 11361 raft_consensus.cc:2726] T 3be0eccc65564bfba3d6e5c398c61fb6 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:17.477195 10393 rpcz_store.cc:276] 0925 09:57:16.181865 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:16.650019 (+468154us) service_pool.cc:227] Handling call
0925 09:57:16.782511 (+132492us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2adb156 00007fabf0618034 00007fabf0617fc2 00007fabf0617ef1 00007fabf0617e6c 00007fabf0617122 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:17.303411 (+520900us) catalog_manager.cc:2075] Acquired catalog manager lock
0925 09:57:17.305448 (+ 2037us) catalog_manager.cc:2121] Created new table and tablet info
0925 09:57:17.476343 (+170895us) catalog_manager.cc:2181] Wrote table and tablets to system table
0925 09:57:17.476607 (+ 264us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0925 09:57:17.476842 (+ 235us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:17.308760 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:17.461098 (+152338us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:17.461122 (+ 24us) write_op.cc:435] Acquired schema lock
0925 09:57:17.461135 (+ 13us) tablet.cc:660] Decoding operations
0925 09:57:17.461382 (+ 247us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:17.461420 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:17.461441 (+ 21us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:17.461610 (+ 169us) tablet.cc:699] Row locks acquired
0925 09:57:17.461623 (+ 13us) write_op.cc:260] PREPARE: finished
0925 09:57:17.461750 (+ 127us) write_op.cc:270] Start()
0925 09:57:17.461825 (+ 75us) write_op.cc:276] Timestamp: P: 1790330237461659 usec, L: 0
0925 09:57:17.461840 (+ 15us) op_driver.cc:331] REPLICATION: starting
0925 09:57:17.462351 (+ 511us) log.cc:842] Serialized 433 byte log entry
0925 09:57:17.474265 (+ 11914us) op_driver.cc:444] REPLICATION: finished
0925 09:57:17.474590 (+ 325us) write_op.cc:301] APPLY: starting
0925 09:57:17.474644 (+ 54us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:17.474753 (+ 109us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:17.474835 (+ 82us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:17.475254 (+ 419us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:17.475391 (+ 137us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 09:57:17.475440 (+ 49us) write_op.cc:312] APPLY: finished
0925 09:57:17.475659 (+ 219us) log.cc:842] Serialized 29 byte log entry
0925 09:57:17.475800 (+ 141us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:17.475862 (+ 62us) write_op.cc:454] Released schema lock
0925 09:57:17.475892 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:17.476063 (+ 171us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:17.476085 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":302315776,"child_traces":[["op",{"apply.queue_time_us":182,"apply.run_cpu_time_us":1644,"apply.run_wall_time_us":1627,"num_ops":2,"prepare.queue_time_us":2282,"prepare.run_cpu_time_us":1873,"prepare.run_wall_time_us":153844,"raft.queue_time_us":4127,"raft.run_cpu_time_us":654,"raft.run_wall_time_us":654,"replication_time_us":12352}]]}
I20260925 09:57:17.497699 11565 raft_consensus.cc:482] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:17.498147 11565 raft_consensus.cc:504] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.500067 11565 leader_election.cc:288] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:17.500156 11085 tablet_bootstrap.cc:654] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:17.501075 10682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "229aaa1fc7b84fb887a8c0d8a53bf0ae" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:17.501718 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "229aaa1fc7b84fb887a8c0d8a53bf0ae" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:17.502314 10919 leader_election.cc:341] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:17.502905 10918 leader_election.cc:341] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:17.503234 10918 leader_election.cc:302] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:17.504130 11565 raft_consensus.cc:2726] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:17.507586 10389 catalog_manager.cc:6565] ReplaceTablet: tablet 1969ac471b6f471ebe8e048a96c22837 deleted and replaced by tablet 977a59175e5b43eeb6f23b938afd00a3
I20260925 09:57:17.507661 11084 tablet_bootstrap.cc:654] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:17.508841 11553 consensus_queue.cc:1035] T c16e096038084b2b9148390b6e773294 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:17.507934 10389 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:35140 (request call id 165) took 1438 ms. Trace:
I20260925 09:57:17.510442 10816 tablet_service.cc:1514] Processing DeleteTablet for tablet 1969ac471b6f471ebe8e048a96c22837 with delete_type TABLET_DATA_DELETED (replaced by tablet 977a59175e5b43eeb6f23b938afd00a3) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:17.510098 10389 rpcz_store.cc:276] 0925 09:57:16.069799 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:16.208558 (+138759us) service_pool.cc:227] Handling call
0925 09:57:16.252717 (+ 44159us) spinlock_profiling.cc:243] Waited 39.6 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2addb81 00007fabf0638d37 00007fabf0638cc2 00007fabf0638bf1 00007fabf0638b6c 00007fabf0637e22 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:17.507880 (+1255163us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:17.131474 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:17.131923 (+ 449us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:17.131955 (+ 32us) write_op.cc:435] Acquired schema lock
0925 09:57:17.131972 (+ 17us) tablet.cc:660] Decoding operations
0925 09:57:17.132324 (+ 352us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:17.132373 (+ 49us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:17.132403 (+ 30us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:17.133487 (+ 1084us) tablet.cc:699] Row locks acquired
0925 09:57:17.133508 (+ 21us) write_op.cc:260] PREPARE: finished
0925 09:57:17.133778 (+ 270us) write_op.cc:270] Start()
0925 09:57:17.133896 (+ 118us) write_op.cc:276] Timestamp: P: 1790330237133727 usec, L: 0
0925 09:57:17.133917 (+ 21us) op_driver.cc:331] REPLICATION: starting
0925 09:57:17.134607 (+ 690us) log.cc:842] Serialized 653 byte log entry
0925 09:57:17.145881 (+ 11274us) op_driver.cc:444] REPLICATION: finished
0925 09:57:17.150887 (+ 5006us) write_op.cc:301] APPLY: starting
0925 09:57:17.150950 (+ 63us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:17.151078 (+ 128us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:17.151204 (+ 126us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:17.151648 (+ 444us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:17.151910 (+ 262us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:17.151969 (+ 59us) write_op.cc:312] APPLY: finished
0925 09:57:17.152160 (+ 191us) log.cc:842] Serialized 29 byte log entry
0925 09:57:17.152911 (+ 751us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:17.152998 (+ 87us) write_op.cc:454] Released schema lock
0925 09:57:17.153039 (+ 41us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:17.153493 (+ 454us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:17.153561 (+ 68us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":91022080,"child_traces":[["op",{"apply.queue_time_us":4834,"apply.run_cpu_time_us":2130,"apply.run_wall_time_us":3218,"num_ops":2,"prepare.queue_time_us":139,"prepare.run_cpu_time_us":2500,"prepare.run_wall_time_us":6670,"raft.queue_time_us":95,"raft.run_cpu_time_us":665,"raft.run_wall_time_us":665,"replication_time_us":11887}]]}
I20260925 09:57:17.526309 10775 leader_election.cc:302] T 91c09118b43b41eca8f7040323ba76f7 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters: be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:17.533919 11526 raft_consensus.cc:2781] T 91c09118b43b41eca8f7040323ba76f7 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:17.534415 11526 raft_consensus.cc:686] T 91c09118b43b41eca8f7040323ba76f7 P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Becoming Leader. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Running, Role: LEADER
I20260925 09:57:17.535182 11526 consensus_queue.cc:237] T 91c09118b43b41eca8f7040323ba76f7 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.541177 11085 tablet_bootstrap.cc:492] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:17.541577 11085 ts_tablet_manager.cc:1397] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.209s user 0.007s sys 0.004s
I20260925 09:57:17.544370 11085 raft_consensus.cc:348] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.545008 11085 raft_consensus.cc:374] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:17.545249 11085 raft_consensus.cc:729] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:17.545796 11085 consensus_queue.cc:260] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.561976 11084 tablet_bootstrap.cc:492] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:17.562383 11084 ts_tablet_manager.cc:1397] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.226s user 0.007s sys 0.005s
I20260925 09:57:17.571714 11085 ts_tablet_manager.cc:1428] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.030s user 0.007s sys 0.000s
I20260925 09:57:17.574867 11085 tablet_bootstrap.cc:492] T ef6fefaa76d44fe798201aab2f21b9c7 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:17.580257 11084 raft_consensus.cc:348] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.581024 11084 raft_consensus.cc:374] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:17.581357 11084 raft_consensus.cc:729] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:17.582125 11084 consensus_queue.cc:260] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.586887 11579 tablet_replica.cc:330] T 1969ac471b6f471ebe8e048a96c22837 P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:17.588318 11579 raft_consensus.cc:2229] T 1969ac471b6f471ebe8e048a96c22837 P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:17.596312 11458 ts_tablet_manager.cc:1918] T 1969ac471b6f471ebe8e048a96c22837 P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 09:57:17.603780 11084 ts_tablet_manager.cc:1428] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.041s user 0.007s sys 0.000s
I20260925 09:57:17.612386 11579 raft_consensus.cc:2258] T 1969ac471b6f471ebe8e048a96c22837 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:17.614379 11579 ts_tablet_manager.cc:1905] T 1969ac471b6f471ebe8e048a96c22837 P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:17.607364 11458 log.cc:1196] T 1969ac471b6f471ebe8e048a96c22837 P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/1969ac471b6f471ebe8e048a96c22837
I20260925 09:57:17.621186 11458 ts_tablet_manager.cc:1939] T 1969ac471b6f471ebe8e048a96c22837 P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:17.625309 11082 tablet_bootstrap.cc:492] T 2d55154ceb064f22bd6f37c8ae5dce74 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:17.625747 11082 ts_tablet_manager.cc:1397] T 2d55154ceb064f22bd6f37c8ae5dce74 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.292s user 0.039s sys 0.026s
I20260925 09:57:17.628734 11082 raft_consensus.cc:348] T 2d55154ceb064f22bd6f37c8ae5dce74 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.629410 11082 raft_consensus.cc:374] T 2d55154ceb064f22bd6f37c8ae5dce74 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:17.629657 11082 raft_consensus.cc:729] T 2d55154ceb064f22bd6f37c8ae5dce74 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:17.630227 11082 consensus_queue.cc:260] T 2d55154ceb064f22bd6f37c8ae5dce74 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.634083 11082 ts_tablet_manager.cc:1428] T 2d55154ceb064f22bd6f37c8ae5dce74 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.008s user 0.006s sys 0.001s
I20260925 09:57:17.635004 11082 tablet_bootstrap.cc:492] T 3c31682bf400494f82ee3ce5e6eec441 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:17.635232 11085 tablet_bootstrap.cc:654] T ef6fefaa76d44fe798201aab2f21b9c7 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:17.650497 11084 tablet_bootstrap.cc:492] T 7360a869f3be45a884a0379f4ef78e85 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:17.662253 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 1969ac471b6f471ebe8e048a96c22837 (table default.Table_ff4e7021d36b43f8b51f19960e707ac8 [id=f17a11d223204599afb24e2e9d1d9d9d]) successfully deleted
I20260925 09:57:17.662983 11526 raft_consensus.cc:482] T 3c31682bf400494f82ee3ce5e6eec441 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:17.663465 11526 raft_consensus.cc:504] T 3c31682bf400494f82ee3ce5e6eec441 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.670142 10682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c31682bf400494f82ee3ce5e6eec441" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:17.670621 10980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c31682bf400494f82ee3ce5e6eec441" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:17.671267 10980 raft_consensus.cc:2454] T 3c31682bf400494f82ee3ce5e6eec441 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
W20260925 09:57:17.671501 10775 leader_election.cc:341] T 3c31682bf400494f82ee3ce5e6eec441 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:17.672526 10772 leader_election.cc:302] T 3c31682bf400494f82ee3ce5e6eec441 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21
I20260925 09:57:17.672766 11082 tablet_bootstrap.cc:654] T 3c31682bf400494f82ee3ce5e6eec441 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:17.673333 11537 raft_consensus.cc:2781] T 3c31682bf400494f82ee3ce5e6eec441 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:17.673628 11537 raft_consensus.cc:482] T 3c31682bf400494f82ee3ce5e6eec441 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:17.673915 11537 raft_consensus.cc:3037] T 3c31682bf400494f82ee3ce5e6eec441 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:17.681247 10388 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet e57065f2bb224a56ad1c3518a0c375da (table default.Table_3a00773e32a34a41bead0b4116e4c6df [id=7889081642604700bbc6d253ab105260]). Expected version 1 got 0
I20260925 09:57:17.682086 10388 catalog_manager.cc:5495] T 69f84509cafc44dd9444218738ebc2fe P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:17.684130 10388 catalog_manager.cc:5495] T c07c6205d1284a6fbb369c010a226da7 P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:17.685444 10388 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet c07c6205d1284a6fbb369c010a226da7 (table default.Table_5a6ecccec7f44c31bbfe8d64580ec33b [id=2d6bc0e2aaee4b1483b01fc678505bb0]). Expected version 1 got 0
I20260925 09:57:17.687093 11526 leader_election.cc:288] T 3c31682bf400494f82ee3ce5e6eec441 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:17.688779 11537 raft_consensus.cc:504] T 3c31682bf400494f82ee3ce5e6eec441 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.690085 10958 tablet_service.cc:1467] Processing CreateTablet for tablet a44d9f2f7e3e48aeb0f31fa29ae1988a (DEFAULT_TABLE table=default.Table_eb065d5979c64931abf91d82b8623a0a [id=5b74f0855d2046e3a9463f732ae19e62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:17.690896 11537 leader_election.cc:288] T 3c31682bf400494f82ee3ce5e6eec441 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:17.691008 10958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a44d9f2f7e3e48aeb0f31fa29ae1988a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:17.691220 10682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c31682bf400494f82ee3ce5e6eec441" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:17.691437 10980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c31682bf400494f82ee3ce5e6eec441" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:17.691965 10980 raft_consensus.cc:3037] T 3c31682bf400494f82ee3ce5e6eec441 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:57:17.692520 10775 leader_election.cc:341] T 3c31682bf400494f82ee3ce5e6eec441 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:17.696130 10816 tablet_service.cc:1467] Processing CreateTablet for tablet a44d9f2f7e3e48aeb0f31fa29ae1988a (DEFAULT_TABLE table=default.Table_eb065d5979c64931abf91d82b8623a0a [id=5b74f0855d2046e3a9463f732ae19e62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:17.697137 10816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a44d9f2f7e3e48aeb0f31fa29ae1988a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:17.698653 10659 tablet_service.cc:1467] Processing CreateTablet for tablet a44d9f2f7e3e48aeb0f31fa29ae1988a (DEFAULT_TABLE table=default.Table_eb065d5979c64931abf91d82b8623a0a [id=5b74f0855d2046e3a9463f732ae19e62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:17.699373 11082 tablet_bootstrap.cc:492] T 3c31682bf400494f82ee3ce5e6eec441 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:17.699817 10659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a44d9f2f7e3e48aeb0f31fa29ae1988a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:17.699903 11082 ts_tablet_manager.cc:1397] T 3c31682bf400494f82ee3ce5e6eec441 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.065s user 0.011s sys 0.002s
I20260925 09:57:17.704411 11084 tablet_bootstrap.cc:654] T 7360a869f3be45a884a0379f4ef78e85 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:17.706288 10980 raft_consensus.cc:2454] T 3c31682bf400494f82ee3ce5e6eec441 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:17.707597 10772 leader_election.cc:302] T 3c31682bf400494f82ee3ce5e6eec441 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21
I20260925 09:57:17.708606 11537 raft_consensus.cc:2781] T 3c31682bf400494f82ee3ce5e6eec441 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:17.709019 11537 raft_consensus.cc:686] T 3c31682bf400494f82ee3ce5e6eec441 P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Becoming Leader. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Running, Role: LEADER
I20260925 09:57:17.710201 11537 consensus_queue.cc:237] T 3c31682bf400494f82ee3ce5e6eec441 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.711323 11085 tablet_bootstrap.cc:492] T ef6fefaa76d44fe798201aab2f21b9c7 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:17.711706 11085 ts_tablet_manager.cc:1397] T ef6fefaa76d44fe798201aab2f21b9c7 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.137s user 0.012s sys 0.000s
I20260925 09:57:17.703660 11082 raft_consensus.cc:348] T 3c31682bf400494f82ee3ce5e6eec441 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.714974 11082 raft_consensus.cc:374] T 3c31682bf400494f82ee3ce5e6eec441 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:17.715318 11082 raft_consensus.cc:729] T 3c31682bf400494f82ee3ce5e6eec441 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:17.716156 11082 consensus_queue.cc:260] T 3c31682bf400494f82ee3ce5e6eec441 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.718542 10961 tablet_service.cc:1467] Processing CreateTablet for tablet 55d14d4d8abe4ec9ac9aeef0d027e804 (DEFAULT_TABLE table=default.Table_4cd66bd0a28d4f618a89f8e8bedca672 [id=71e2eff1279847a5a97e91d80453615e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:17.719571 10961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55d14d4d8abe4ec9ac9aeef0d027e804. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:17.727334 11084 tablet_bootstrap.cc:492] T 7360a869f3be45a884a0379f4ef78e85 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:17.727743 11084 ts_tablet_manager.cc:1397] T 7360a869f3be45a884a0379f4ef78e85 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.077s user 0.009s sys 0.000s
I20260925 09:57:17.729753 11084 raft_consensus.cc:348] T 7360a869f3be45a884a0379f4ef78e85 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.730331 11084 raft_consensus.cc:374] T 7360a869f3be45a884a0379f4ef78e85 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:17.730558 11084 raft_consensus.cc:729] T 7360a869f3be45a884a0379f4ef78e85 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:17.732820 11565 raft_consensus.cc:482] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:17.733294 11565 raft_consensus.cc:504] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.734534 11082 ts_tablet_manager.cc:1428] T 3c31682bf400494f82ee3ce5e6eec441 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.034s user 0.009s sys 0.000s
I20260925 09:57:17.736452 11565 leader_election.cc:288] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:17.737038 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "197f61525b0c4f9da441d98701d231a3" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:17.738492 10918 leader_election.cc:341] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:17.739074 11082 tablet_bootstrap.cc:492] T ef6fefaa76d44fe798201aab2f21b9c7 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:17.741101 10815 tablet_service.cc:1467] Processing CreateTablet for tablet 55d14d4d8abe4ec9ac9aeef0d027e804 (DEFAULT_TABLE table=default.Table_4cd66bd0a28d4f618a89f8e8bedca672 [id=71e2eff1279847a5a97e91d80453615e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:17.740840 11084 consensus_queue.cc:260] T 7360a869f3be45a884a0379f4ef78e85 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.742147 10815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55d14d4d8abe4ec9ac9aeef0d027e804. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:17.743017 11084 ts_tablet_manager.cc:1428] T 7360a869f3be45a884a0379f4ef78e85 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.015s user 0.004s sys 0.000s
I20260925 09:57:17.744156 10814 tablet_service.cc:1467] Processing CreateTablet for tablet d9e454f82a4e4c429b58f32926319b7c (DEFAULT_TABLE table=default.Table_95a4c3e1f6014e63811447ba4a0e34cf [id=626e547271e6456e81ccaa3b1e59bd97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:17.745258 10814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9e454f82a4e4c429b58f32926319b7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:17.755532 11082 tablet_bootstrap.cc:654] T ef6fefaa76d44fe798201aab2f21b9c7 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:17.758090 10682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "197f61525b0c4f9da441d98701d231a3" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:17.758579 10658 tablet_service.cc:1467] Processing CreateTablet for tablet 55d14d4d8abe4ec9ac9aeef0d027e804 (DEFAULT_TABLE table=default.Table_4cd66bd0a28d4f618a89f8e8bedca672 [id=71e2eff1279847a5a97e91d80453615e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:17.759605 10665 tablet_service.cc:1467] Processing CreateTablet for tablet d9e454f82a4e4c429b58f32926319b7c (DEFAULT_TABLE table=default.Table_95a4c3e1f6014e63811447ba4a0e34cf [id=626e547271e6456e81ccaa3b1e59bd97]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 09:57:17.761641 10919 leader_election.cc:341] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:17.762035 10919 leader_election.cc:302] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:17.763298 11565 raft_consensus.cc:2726] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:17.763914 11543 raft_consensus.cc:482] T 7360a869f3be45a884a0379f4ef78e85 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:17.764554 11543 raft_consensus.cc:504] T 7360a869f3be45a884a0379f4ef78e85 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.766573 11543 leader_election.cc:288] T 7360a869f3be45a884a0379f4ef78e85 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:17.768412 10682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7360a869f3be45a884a0379f4ef78e85" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:17.769804 10919 leader_election.cc:341] T 7360a869f3be45a884a0379f4ef78e85 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:17.788646 10387 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet e57065f2bb224a56ad1c3518a0c375da (table default.Table_3a00773e32a34a41bead0b4116e4c6df [id=7889081642604700bbc6d253ab105260]). Expected version 1 got 0
I20260925 09:57:17.791425 10658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55d14d4d8abe4ec9ac9aeef0d027e804. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:17.792349 10665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9e454f82a4e4c429b58f32926319b7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:17.793145 10387 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet c07c6205d1284a6fbb369c010a226da7 (table default.Table_5a6ecccec7f44c31bbfe8d64580ec33b [id=2d6bc0e2aaee4b1483b01fc678505bb0]). Expected version 1 got 0
I20260925 09:57:17.801438 10962 tablet_service.cc:1467] Processing CreateTablet for tablet d9e454f82a4e4c429b58f32926319b7c (DEFAULT_TABLE table=default.Table_95a4c3e1f6014e63811447ba4a0e34cf [id=626e547271e6456e81ccaa3b1e59bd97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:17.802520 10962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9e454f82a4e4c429b58f32926319b7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:17.807847 10384 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet c07c6205d1284a6fbb369c010a226da7 (table default.Table_5a6ecccec7f44c31bbfe8d64580ec33b [id=2d6bc0e2aaee4b1483b01fc678505bb0]). Expected version 1 got 0
I20260925 09:57:17.807542 11585 raft_consensus.cc:482] T 2d55154ceb064f22bd6f37c8ae5dce74 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:17.808455 11585 raft_consensus.cc:504] T 2d55154ceb064f22bd6f37c8ae5dce74 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.810593 11585 leader_election.cc:288] T 2d55154ceb064f22bd6f37c8ae5dce74 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
W20260925 09:57:17.812853 10619 leader_election.cc:341] T 2d55154ceb064f22bd6f37c8ae5dce74 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 2d55154ceb064f22bd6f37c8ae5dce74
I20260925 09:57:17.817919 11082 tablet_bootstrap.cc:492] T ef6fefaa76d44fe798201aab2f21b9c7 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:17.810240 10384 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 28d60531970a4ab58ce5ef99cb3a96bf (table default.Table_ce09d3a8d8e84973894f33b18bbfdb22 [id=6851500c7dba44f1bf353bb79bafd97a]). Expected version 2 got 0
I20260925 09:57:17.822961 11082 ts_tablet_manager.cc:1397] T ef6fefaa76d44fe798201aab2f21b9c7 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.084s user 0.012s sys 0.002s
I20260925 09:57:17.825495 10666 tablet_service.cc:1467] Processing CreateTablet for tablet aba253f22afa4200926f31a1b43e3970 (DEFAULT_TABLE table=default.Table_bc338e8ca7174647948301512850ce00 [id=7d130f66062344458770b2b943b4485a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:17.826431 10666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aba253f22afa4200926f31a1b43e3970. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:17.835760 10963 tablet_service.cc:1467] Processing CreateTablet for tablet aba253f22afa4200926f31a1b43e3970 (DEFAULT_TABLE table=default.Table_bc338e8ca7174647948301512850ce00 [id=7d130f66062344458770b2b943b4485a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:17.836766 10963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aba253f22afa4200926f31a1b43e3970. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:17.840765 10658 tablet_service.cc:1467] Processing CreateTablet for tablet df006163f72743b594baf1a3efd1c771 (DEFAULT_TABLE table=default.Table_136e1173ab5c41c5bc5610f6b0c259a0 [id=d46929fb3bed4ffba6eea0d548488d8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:17.841452 10953 tablet_service.cc:1467] Processing CreateTablet for tablet df006163f72743b594baf1a3efd1c771 (DEFAULT_TABLE table=default.Table_136e1173ab5c41c5bc5610f6b0c259a0 [id=d46929fb3bed4ffba6eea0d548488d8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:17.842589 10953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df006163f72743b594baf1a3efd1c771. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:17.842633 10661 tablet_service.cc:1467] Processing CreateTablet for tablet 9e697e7146e34841bd64660b45ebf231 (DEFAULT_TABLE table=default.Table_f0ebee2143d14ff8b4f798e61ec58bdf [id=8eb261fd77d74c98913f891ff2d4d897]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:17.844601 10661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e697e7146e34841bd64660b45ebf231. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:17.857723 10384 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 26fae3fbfe374f348a1729c21ec751b9 (table default.Table_0011387b45014de4b24b466f4cb907f7 [id=e3f7aa5e988c4b6c98be5cd3d465245d]). Expected version 1 got 0
I20260925 09:57:17.859824 10658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df006163f72743b594baf1a3efd1c771. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:17.860436 11084 tablet_bootstrap.cc:492] T d9e454f82a4e4c429b58f32926319b7c P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:17.862433 10384 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet ef6fefaa76d44fe798201aab2f21b9c7 (table default.Table_46e4c497898b454c85b7c2a299ebf3d9 [id=80b48dd413e3446eb7ac65e5f957b5dc]). Expected version 2 got 0
I20260925 09:57:17.865804 11084 tablet_bootstrap.cc:654] T d9e454f82a4e4c429b58f32926319b7c P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:17.868183 10959 tablet_service.cc:1467] Processing CreateTablet for tablet 2d087d3e35814c90a03eccd90eeea86c (DEFAULT_TABLE table=default.Table_bcf18c926c6e4a62b31f3bffe9b0037e [id=53df4f0ffd794afab1aa17fcfb4d56bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:17.869266 10959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d087d3e35814c90a03eccd90eeea86c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:17.857052 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7360a869f3be45a884a0379f4ef78e85" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:17.873970 10918 leader_election.cc:341] T 7360a869f3be45a884a0379f4ef78e85 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:17.874352 10918 leader_election.cc:302] T 7360a869f3be45a884a0379f4ef78e85 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:17.875262 11543 raft_consensus.cc:2726] T 7360a869f3be45a884a0379f4ef78e85 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:17.876245 11084 tablet_bootstrap.cc:492] T d9e454f82a4e4c429b58f32926319b7c P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:17.876600 11084 ts_tablet_manager.cc:1397] T d9e454f82a4e4c429b58f32926319b7c P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.004s
I20260925 09:57:17.878672 11084 raft_consensus.cc:348] T d9e454f82a4e4c429b58f32926319b7c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.861012 11082 raft_consensus.cc:348] T ef6fefaa76d44fe798201aab2f21b9c7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.869138 10664 tablet_service.cc:1467] Processing CreateTablet for tablet 2d087d3e35814c90a03eccd90eeea86c (DEFAULT_TABLE table=default.Table_bcf18c926c6e4a62b31f3bffe9b0037e [id=53df4f0ffd794afab1aa17fcfb4d56bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:17.880448 11082 raft_consensus.cc:374] T ef6fefaa76d44fe798201aab2f21b9c7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:17.880713 11082 raft_consensus.cc:729] T ef6fefaa76d44fe798201aab2f21b9c7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:17.881013 10664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d087d3e35814c90a03eccd90eeea86c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:17.886206 11579 ts_tablet_manager.cc:1918] T 1969ac471b6f471ebe8e048a96c22837 P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 09:57:17.886669 10393 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet c07c6205d1284a6fbb369c010a226da7 (table default.Table_5a6ecccec7f44c31bbfe8d64580ec33b [id=2d6bc0e2aaee4b1483b01fc678505bb0]). Expected version 1 got 0
I20260925 09:57:17.887871 10393 catalog_manager.cc:5495] T c16e096038084b2b9148390b6e773294 P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:17.889811 10393 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 28d60531970a4ab58ce5ef99cb3a96bf (table default.Table_ce09d3a8d8e84973894f33b18bbfdb22 [id=6851500c7dba44f1bf353bb79bafd97a]). Expected version 2 got 0
I20260925 09:57:17.879263 11084 raft_consensus.cc:374] T d9e454f82a4e4c429b58f32926319b7c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:17.894526 11084 raft_consensus.cc:729] T d9e454f82a4e4c429b58f32926319b7c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:17.898345 10665 tablet_service.cc:1467] Processing CreateTablet for tablet c2110d4ada2d45c2b07259374465a8df (DEFAULT_TABLE table=default.Table_256aab1fb8b742d6b04709d930f59bce [id=8307778a24ee442b8f7e89c09652309e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:17.900014 10393 catalog_manager.cc:5495] T 47a9fb9392244e1d89f8af2bfb42edd6 P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:17.900895 10668 tablet_service.cc:1467] Processing CreateTablet for tablet 3b9cdad4f9144daea800553abd378e0b (DEFAULT_TABLE table=default.Table_591aa6b2533844f1904ae4982a7f01ba [id=859095b05f56488e9687cfc7c6b2a5fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:17.898882 11082 consensus_queue.cc:260] T ef6fefaa76d44fe798201aab2f21b9c7 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.902746 10953 tablet_service.cc:1467] Processing CreateTablet for tablet c2110d4ada2d45c2b07259374465a8df (DEFAULT_TABLE table=default.Table_256aab1fb8b742d6b04709d930f59bce [id=8307778a24ee442b8f7e89c09652309e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:17.901777 10668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b9cdad4f9144daea800553abd378e0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:17.906994 11082 ts_tablet_manager.cc:1428] T ef6fefaa76d44fe798201aab2f21b9c7 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.083s user 0.006s sys 0.001s
I20260925 09:57:17.918242 10962 tablet_service.cc:1467] Processing CreateTablet for tablet 9e697e7146e34841bd64660b45ebf231 (DEFAULT_TABLE table=default.Table_f0ebee2143d14ff8b4f798e61ec58bdf [id=8eb261fd77d74c98913f891ff2d4d897]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:17.919166 10962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e697e7146e34841bd64660b45ebf231. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:17.928598 11082 tablet_bootstrap.cc:492] T 752105c1514f4c1db2a11b95b413b4f6 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:17.932156 10953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2110d4ada2d45c2b07259374465a8df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:17.895117 11084 consensus_queue.cc:260] T d9e454f82a4e4c429b58f32926319b7c P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.954012 10960 tablet_service.cc:1467] Processing CreateTablet for tablet 3ff4cf556eee494d8cd590884466f09f (DEFAULT_TABLE table=default.Table_66b8a45b14bf441d80ab32ee756de608 [id=2aeb0fb9ff8c416aba7b24d74354fcb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:17.955099 10960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ff4cf556eee494d8cd590884466f09f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:17.965435 11084 ts_tablet_manager.cc:1428] T d9e454f82a4e4c429b58f32926319b7c P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.089s user 0.007s sys 0.000s
I20260925 09:57:17.951174 10668 tablet_service.cc:1467] Processing CreateTablet for tablet 3ff4cf556eee494d8cd590884466f09f (DEFAULT_TABLE table=default.Table_66b8a45b14bf441d80ab32ee756de608 [id=2aeb0fb9ff8c416aba7b24d74354fcb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:17.968678 10959 tablet_service.cc:1467] Processing CreateTablet for tablet 3b9cdad4f9144daea800553abd378e0b (DEFAULT_TABLE table=default.Table_591aa6b2533844f1904ae4982a7f01ba [id=859095b05f56488e9687cfc7c6b2a5fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:17.972563 11084 tablet_bootstrap.cc:492] T df006163f72743b594baf1a3efd1c771 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:17.972491 10391 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 28d60531970a4ab58ce5ef99cb3a96bf (table default.Table_ce09d3a8d8e84973894f33b18bbfdb22 [id=6851500c7dba44f1bf353bb79bafd97a]). Expected version 2 got 0
I20260925 09:57:17.977483 10391 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet c07c6205d1284a6fbb369c010a226da7 (table default.Table_5a6ecccec7f44c31bbfe8d64580ec33b [id=2d6bc0e2aaee4b1483b01fc678505bb0]). Expected version 1 got 0
I20260925 09:57:17.909708 10665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2110d4ada2d45c2b07259374465a8df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:17.916790 11585 consensus_queue.cc:1035] T 28d60531970a4ab58ce5ef99cb3a96bf P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:17.978499 10959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b9cdad4f9144daea800553abd378e0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:17.979609 11586 consensus_queue.cc:1035] T 28d60531970a4ab58ce5ef99cb3a96bf P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:17.896180 10817 tablet_service.cc:1467] Processing CreateTablet for tablet aba253f22afa4200926f31a1b43e3970 (DEFAULT_TABLE table=default.Table_bc338e8ca7174647948301512850ce00 [id=7d130f66062344458770b2b943b4485a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:17.981371 10817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aba253f22afa4200926f31a1b43e3970. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:17.898245 10819 tablet_service.cc:1467] Processing CreateTablet for tablet df006163f72743b594baf1a3efd1c771 (DEFAULT_TABLE table=default.Table_136e1173ab5c41c5bc5610f6b0c259a0 [id=d46929fb3bed4ffba6eea0d548488d8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:17.984059 10819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df006163f72743b594baf1a3efd1c771. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:17.984155 11543 raft_consensus.cc:482] T 93e1b5837aff48d7907890c40dfcd0d0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:17.984809 11543 raft_consensus.cc:504] T 93e1b5837aff48d7907890c40dfcd0d0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:17.899853 10818 tablet_service.cc:1467] Processing CreateTablet for tablet 9e697e7146e34841bd64660b45ebf231 (DEFAULT_TABLE table=default.Table_f0ebee2143d14ff8b4f798e61ec58bdf [id=8eb261fd77d74c98913f891ff2d4d897]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:17.987677 10682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93e1b5837aff48d7907890c40dfcd0d0" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:17.987959 10818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e697e7146e34841bd64660b45ebf231. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:17.988508 11543 leader_election.cc:288] T 93e1b5837aff48d7907890c40dfcd0d0 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
W20260925 09:57:17.989519 10919 leader_election.cc:341] T 93e1b5837aff48d7907890c40dfcd0d0 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:17.903816 10812 tablet_service.cc:1467] Processing CreateTablet for tablet 2d087d3e35814c90a03eccd90eeea86c (DEFAULT_TABLE table=default.Table_bcf18c926c6e4a62b31f3bffe9b0037e [id=53df4f0ffd794afab1aa17fcfb4d56bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:17.992631 10668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ff4cf556eee494d8cd590884466f09f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:17.992771 10812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d087d3e35814c90a03eccd90eeea86c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:17.906960 10813 tablet_service.cc:1467] Processing CreateTablet for tablet c2110d4ada2d45c2b07259374465a8df (DEFAULT_TABLE table=default.Table_256aab1fb8b742d6b04709d930f59bce [id=8307778a24ee442b8f7e89c09652309e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:17.996023 10813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2110d4ada2d45c2b07259374465a8df. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:17.998643 10379 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 -> Peer b8be057e0b6d40a3a4b90f2b1f4a0fa2 (127.10.9.60:36985): Couldn't send request to peer b8be057e0b6d40a3a4b90f2b1f4a0fa2. Status: Network error: Client connection negotiation failed: client connection to 127.10.9.60:36985: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260925 09:57:18.001024 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93e1b5837aff48d7907890c40dfcd0d0" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:17.989244 11084 tablet_bootstrap.cc:654] T df006163f72743b594baf1a3efd1c771 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:17.913137 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d55154ceb064f22bd6f37c8ae5dce74" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:18.004920 10835 raft_consensus.cc:2454] T 2d55154ceb064f22bd6f37c8ae5dce74 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:18.006871 10664 tablet_service.cc:1467] Processing CreateTablet for tablet 60fdc299865446c89d3e5374d573029a (DEFAULT_TABLE table=default.Table_32e7bdf7eaf147ea90740b6b3cfb2fd3 [id=be059d72d5db4915ba767959c6a1c7b2]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 09:57:18.012530 10918 leader_election.cc:341] T 93e1b5837aff48d7907890c40dfcd0d0 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:18.012895 10918 leader_election.cc:302] T 93e1b5837aff48d7907890c40dfcd0d0 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:18.013634 11543 raft_consensus.cc:2726] T 93e1b5837aff48d7907890c40dfcd0d0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:18.018085 10664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60fdc299865446c89d3e5374d573029a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:18.017462 11084 tablet_bootstrap.cc:492] T df006163f72743b594baf1a3efd1c771 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:18.024618 11084 ts_tablet_manager.cc:1397] T df006163f72743b594baf1a3efd1c771 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.052s user 0.009s sys 0.000s
I20260925 09:57:18.026671 11084 raft_consensus.cc:348] T df006163f72743b594baf1a3efd1c771 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:18.013633 11082 tablet_bootstrap.cc:654] T 752105c1514f4c1db2a11b95b413b4f6 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:18.027288 11084 raft_consensus.cc:374] T df006163f72743b594baf1a3efd1c771 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:18.027531 11084 raft_consensus.cc:729] T df006163f72743b594baf1a3efd1c771 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:18.031562 11084 consensus_queue.cc:260] T df006163f72743b594baf1a3efd1c771 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:18.040966 11084 ts_tablet_manager.cc:1428] T df006163f72743b594baf1a3efd1c771 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.016s user 0.004s sys 0.000s
I20260925 09:57:18.041791 11084 tablet_bootstrap.cc:492] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:18.037142 10389 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 28d60531970a4ab58ce5ef99cb3a96bf (table default.Table_ce09d3a8d8e84973894f33b18bbfdb22 [id=6851500c7dba44f1bf353bb79bafd97a]). Expected version 2 got 0
I20260925 09:57:18.048815 10389 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet c07c6205d1284a6fbb369c010a226da7 (table default.Table_5a6ecccec7f44c31bbfe8d64580ec33b [id=2d6bc0e2aaee4b1483b01fc678505bb0]). Expected version 1 got 0
I20260925 09:57:18.054744 11082 tablet_bootstrap.cc:492] T 752105c1514f4c1db2a11b95b413b4f6 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:18.055375 11082 ts_tablet_manager.cc:1397] T 752105c1514f4c1db2a11b95b413b4f6 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.127s user 0.010s sys 0.002s
I20260925 09:57:18.060068 11084 tablet_bootstrap.cc:654] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:18.063783 11082 raft_consensus.cc:348] T 752105c1514f4c1db2a11b95b413b4f6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:18.069651 10389 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet ef6fefaa76d44fe798201aab2f21b9c7 (table default.Table_46e4c497898b454c85b7c2a299ebf3d9 [id=80b48dd413e3446eb7ac65e5f957b5dc]). Expected version 2 got 0
I20260925 09:57:18.070446 10389 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 26fae3fbfe374f348a1729c21ec751b9 (table default.Table_0011387b45014de4b24b466f4cb907f7 [id=e3f7aa5e988c4b6c98be5cd3d465245d]). Expected version 1 got 0
I20260925 09:57:18.073770 11084 tablet_bootstrap.cc:492] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:18.074062 11082 raft_consensus.cc:374] T 752105c1514f4c1db2a11b95b413b4f6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:18.074138 11084 ts_tablet_manager.cc:1397] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.032s user 0.009s sys 0.000s
I20260925 09:57:18.075006 10389 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 3d652a76d96f4dbeafde15f3c16534a6 (table default.Table_be0f658aa41f4188bc16db37e96f4ad1 [id=73083c1560cb4db5a12964387cb49b73]). Expected version 1 got 0
I20260925 09:57:18.075460 11082 raft_consensus.cc:729] T 752105c1514f4c1db2a11b95b413b4f6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:17.914064 11085 raft_consensus.cc:348] T ef6fefaa76d44fe798201aab2f21b9c7 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:18.080235 11085 raft_consensus.cc:374] T ef6fefaa76d44fe798201aab2f21b9c7 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:18.080508 11085 raft_consensus.cc:729] T ef6fefaa76d44fe798201aab2f21b9c7 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
W20260925 09:57:18.081756 10390 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b555eeba505740988c704df5c734e7c0 consensus state has no leader
I20260925 09:57:18.082259 10390 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:35140 (request call id 169) took 1672 ms. Trace:
I20260925 09:57:18.082527 10390 rpcz_store.cc:276] 0925 09:57:16.410018 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:16.710825 (+300807us) service_pool.cc:227] Handling call
0925 09:57:17.301195 (+590370us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:57:17.301830 (+ 635us) catalog_manager.cc:3616] Apply alter schema
0925 09:57:17.302370 (+ 540us) catalog_manager.cc:3639] Acquired catalog manager lock
0925 09:57:17.308095 (+ 5725us) catalog_manager.cc:3786] Updating metadata on disk
0925 09:57:17.490341 (+182246us) catalog_manager.cc:3817] Committing alterations to in-memory state
0925 09:57:18.082204 (+591863us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:17.462700 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:17.463036 (+ 336us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:17.463057 (+ 21us) write_op.cc:435] Acquired schema lock
0925 09:57:17.463069 (+ 12us) tablet.cc:660] Decoding operations
0925 09:57:17.463294 (+ 225us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:17.463326 (+ 32us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:17.463348 (+ 22us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:17.481764 (+ 18416us) tablet.cc:699] Row locks acquired
0925 09:57:17.481781 (+ 17us) write_op.cc:260] PREPARE: finished
0925 09:57:17.481878 (+ 97us) write_op.cc:270] Start()
0925 09:57:17.481981 (+ 103us) write_op.cc:276] Timestamp: P: 1790330237481813 usec, L: 0
0925 09:57:17.481997 (+ 16us) op_driver.cc:331] REPLICATION: starting
0925 09:57:17.482442 (+ 445us) log.cc:842] Serialized 405 byte log entry
0925 09:57:17.488744 (+ 6302us) op_driver.cc:444] REPLICATION: finished
0925 09:57:17.488984 (+ 240us) write_op.cc:301] APPLY: starting
0925 09:57:17.489034 (+ 50us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:17.489115 (+ 81us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:17.489205 (+ 90us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:17.489355 (+ 150us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:17.489472 (+ 117us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:17.489515 (+ 43us) write_op.cc:312] APPLY: finished
0925 09:57:17.489659 (+ 144us) log.cc:842] Serialized 23 byte log entry
0925 09:57:17.489797 (+ 138us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:17.489851 (+ 54us) write_op.cc:454] Released schema lock
0925 09:57:17.489882 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:17.490033 (+ 151us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:17.490056 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":122,"apply.run_cpu_time_us":1224,"apply.run_wall_time_us":1224,"num_ops":1,"prepare.queue_time_us":1869,"prepare.run_cpu_time_us":1801,"prepare.run_wall_time_us":21482,"raft.queue_time_us":71,"raft.run_cpu_time_us":490,"raft.run_wall_time_us":490,"replication_time_us":6674}]]}
I20260925 09:57:18.078224 11082 consensus_queue.cc:260] T 752105c1514f4c1db2a11b95b413b4f6 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:18.088333 10384 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: default.Table_3a00773e32a34a41bead0b4116e4c6df [id=7889081642604700bbc6d253ab105260] alter complete (version 1)
I20260925 09:57:18.077121 11084 raft_consensus.cc:348] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:18.090101 11084 raft_consensus.cc:374] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:18.090360 11084 raft_consensus.cc:729] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:18.090924 11084 consensus_queue.cc:260] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:18.095361 11084 ts_tablet_manager.cc:1428] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.021s user 0.004s sys 0.003s
I20260925 09:57:18.096151 11084 tablet_bootstrap.cc:492] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:18.096498 11579 log.cc:1196] T 1969ac471b6f471ebe8e048a96c22837 P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/1969ac471b6f471ebe8e048a96c22837
I20260925 09:57:18.098796 11579 ts_tablet_manager.cc:1939] T 1969ac471b6f471ebe8e048a96c22837 P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:18.109841 11084 tablet_bootstrap.cc:654] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:18.119596 11543 raft_consensus.cc:482] T d9e454f82a4e4c429b58f32926319b7c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:18.120030 11543 raft_consensus.cc:504] T d9e454f82a4e4c429b58f32926319b7c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:18.120119 11085 consensus_queue.cc:260] T ef6fefaa76d44fe798201aab2f21b9c7 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:18.122174 11543 leader_election.cc:288] T d9e454f82a4e4c429b58f32926319b7c P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:18.124768 10682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9e454f82a4e4c429b58f32926319b7c" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:18.126206 10919 leader_election.cc:341] T d9e454f82a4e4c429b58f32926319b7c P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:18.128906 10918 leader_election.cc:341] T d9e454f82a4e4c429b58f32926319b7c P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Not found: Tablet not found: d9e454f82a4e4c429b58f32926319b7c
I20260925 09:57:18.129300 10918 leader_election.cc:302] T d9e454f82a4e4c429b58f32926319b7c P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:18.129932 11543 raft_consensus.cc:2726] T d9e454f82a4e4c429b58f32926319b7c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:18.135257 10664 tablet_service.cc:1514] Processing DeleteTablet for tablet 833fac37485141f6a18104eaea907b0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:14 UTC) from {username='slave'} at 127.0.0.1:37876
W20260925 09:57:18.141868 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 833fac37485141f6a18104eaea907b0e because the tablet was not found. No further retry: Not found: Tablet not found: 833fac37485141f6a18104eaea907b0e
I20260925 09:57:18.148519 10668 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e882284a014455192375daad391efaf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:09 UTC) from {username='slave'} at 127.0.0.1:37876
W20260925 09:57:18.152643 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 0e882284a014455192375daad391efaf because the tablet was not found. No further retry: Not found: Tablet not found: 0e882284a014455192375daad391efaf
I20260925 09:57:18.154983 11543 raft_consensus.cc:482] T ef6fefaa76d44fe798201aab2f21b9c7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:18.155722 11543 raft_consensus.cc:504] T ef6fefaa76d44fe798201aab2f21b9c7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:18.159442 10392 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.1:56563 (request call id 25) took 3684 ms (client timeout 1000 ms). Trace:
W20260925 09:57:18.163576 10393 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.1:56563 (request call id 27) took 1129 ms (client timeout 1000 ms). Trace:
I20260925 09:57:18.165092 10963 tablet_service.cc:1467] Processing CreateTablet for tablet 60fdc299865446c89d3e5374d573029a (DEFAULT_TABLE table=default.Table_32e7bdf7eaf147ea90740b6b3cfb2fd3 [id=be059d72d5db4915ba767959c6a1c7b2]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 09:57:18.168650 10388 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.1:56563 (request call id 26) took 2448 ms (client timeout 1000 ms). Trace:
I20260925 09:57:18.169802 10811 tablet_service.cc:1467] Processing CreateTablet for tablet 3b9cdad4f9144daea800553abd378e0b (DEFAULT_TABLE table=default.Table_591aa6b2533844f1904ae4982a7f01ba [id=859095b05f56488e9687cfc7c6b2a5fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:18.170845 10811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b9cdad4f9144daea800553abd378e0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:18.180624 10815 tablet_service.cc:1467] Processing CreateTablet for tablet 60fdc299865446c89d3e5374d573029a (DEFAULT_TABLE table=default.Table_32e7bdf7eaf147ea90740b6b3cfb2fd3 [id=be059d72d5db4915ba767959c6a1c7b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:18.181681 10815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60fdc299865446c89d3e5374d573029a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:18.183033 11085 ts_tablet_manager.cc:1428] T ef6fefaa76d44fe798201aab2f21b9c7 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.471s user 0.005s sys 0.000s
I20260925 09:57:18.183869 11085 tablet_bootstrap.cc:492] T 42ec275c905343c4be1b72dc2bd0f6e3 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:18.187251 10959 tablet_service.cc:1514] Processing DeleteTablet for tablet 91c09118b43b41eca8f7040323ba76f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:15 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:18.187798 10963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60fdc299865446c89d3e5374d573029a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:18.194020 10958 tablet_service.cc:1514] Processing DeleteTablet for tablet 1969ac471b6f471ebe8e048a96c22837 with delete_type TABLET_DATA_DELETED (replaced by tablet 977a59175e5b43eeb6f23b938afd00a3) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:18.205914 10658 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c4e8790282a49fd849a34ca11bef5bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:11 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:18.206192 11565 raft_consensus.cc:482] T df006163f72743b594baf1a3efd1c771 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:18.206604 11603 tablet_replica.cc:330] T 4c4e8790282a49fd849a34ca11bef5bf P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:18.206642 11565 raft_consensus.cc:504] T df006163f72743b594baf1a3efd1c771 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:18.207458 11603 raft_consensus.cc:2229] T 4c4e8790282a49fd849a34ca11bef5bf P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:18.208314 11603 raft_consensus.cc:2258] T 4c4e8790282a49fd849a34ca11bef5bf P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:18.208400 10661 tablet_service.cc:1514] Processing DeleteTablet for tablet fd73d0d52350444499abae34e8c02fab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:15 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:18.209003 10666 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e882284a014455192375daad391efaf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:09 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:18.210243 11603 ts_tablet_manager.cc:1905] T 4c4e8790282a49fd849a34ca11bef5bf P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:18.213534 11585 raft_consensus.cc:482] T 3c31682bf400494f82ee3ce5e6eec441 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:18.213783 11586 consensus_queue.cc:1035] T 3be0eccc65564bfba3d6e5c398c61fb6 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:18.213930 11585 raft_consensus.cc:504] T 3c31682bf400494f82ee3ce5e6eec441 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:18.215585 10658 tablet_service.cc:1514] Processing DeleteTablet for tablet 91c09118b43b41eca8f7040323ba76f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:15 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:18.215727 11585 leader_election.cc:288] T 3c31682bf400494f82ee3ce5e6eec441 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:18.218578 11553 raft_consensus.cc:482] T ef6fefaa76d44fe798201aab2f21b9c7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:18.218997 10682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df006163f72743b594baf1a3efd1c771" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:18.219060 11553 raft_consensus.cc:504] T ef6fefaa76d44fe798201aab2f21b9c7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:18.221257 10664 tablet_service.cc:1514] Processing DeleteTablet for tablet 1969ac471b6f471ebe8e048a96c22837 with delete_type TABLET_DATA_DELETED (replaced by tablet 977a59175e5b43eeb6f23b938afd00a3) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:18.222894 11553 leader_election.cc:288] T ef6fefaa76d44fe798201aab2f21b9c7 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:18.223867 10980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c31682bf400494f82ee3ce5e6eec441" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:18.224975 10980 raft_consensus.cc:2379] T 3c31682bf400494f82ee3ce5e6eec441 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b8850818e2ee42a082802d0095c08d21 in current term 1: Already voted for candidate c43461c4ee854ae1ac52b873ccada318 in this term.
I20260925 09:57:18.232988 11565 leader_election.cc:288] T df006163f72743b594baf1a3efd1c771 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:18.242579 11084 tablet_bootstrap.cc:492] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:18.242993 11084 ts_tablet_manager.cc:1397] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.147s user 0.003s sys 0.020s
I20260925 09:57:18.243888 11543 leader_election.cc:288] T ef6fefaa76d44fe798201aab2f21b9c7 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:18.245051 11084 raft_consensus.cc:348] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:18.245663 11084 raft_consensus.cc:374] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:18.245887 11084 raft_consensus.cc:729] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:18.246400 11084 consensus_queue.cc:260] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:18.247689 10810 tablet_service.cc:1467] Processing CreateTablet for tablet 3ff4cf556eee494d8cd590884466f09f (DEFAULT_TABLE table=default.Table_66b8a45b14bf441d80ab32ee756de608 [id=2aeb0fb9ff8c416aba7b24d74354fcb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:18.237840 10980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef6fefaa76d44fe798201aab2f21b9c7" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:18.250322 10980 raft_consensus.cc:2454] T ef6fefaa76d44fe798201aab2f21b9c7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:18.260864 10619 leader_election.cc:302] T ef6fefaa76d44fe798201aab2f21b9c7 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:18.261783 11553 raft_consensus.cc:2781] T ef6fefaa76d44fe798201aab2f21b9c7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:18.265407 11553 raft_consensus.cc:482] T ef6fefaa76d44fe798201aab2f21b9c7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:18.265815 11553 raft_consensus.cc:3037] T ef6fefaa76d44fe798201aab2f21b9c7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:18.271126 11084 ts_tablet_manager.cc:1428] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.028s user 0.005s sys 0.001s
I20260925 09:57:18.272048 11084 tablet_bootstrap.cc:492] T 9e697e7146e34841bd64660b45ebf231 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:18.279315 10659 tablet_service.cc:1467] Processing CreateTablet for tablet 6161774f61ae48ab9764a05f2188a80b (DEFAULT_TABLE table=default.Table_15b07a280b634333985637e0f6746764 [id=7ee526720b2442119e0fc487b7c4fcd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:18.279973 11537 raft_consensus.cc:482] T ef6fefaa76d44fe798201aab2f21b9c7 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:18.280483 10810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ff4cf556eee494d8cd590884466f09f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:18.280429 11537 raft_consensus.cc:504] T ef6fefaa76d44fe798201aab2f21b9c7 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:18.281801 10659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6161774f61ae48ab9764a05f2188a80b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:18.282919 10664 tablet_service.cc:1467] Processing CreateTablet for tablet fe91b39f91274a7ca61e9618d8cbb475 (DEFAULT_TABLE table=default.Table_a4437c88d32f4288a6d9581edb106440 [id=288fff5d48da4efe82583b6a86de2084]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:18.283830 10664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe91b39f91274a7ca61e9618d8cbb475. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:18.267241 11603 ts_tablet_manager.cc:1918] T 4c4e8790282a49fd849a34ca11bef5bf P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:18.285081 11603 log.cc:1196] T 4c4e8790282a49fd849a34ca11bef5bf P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/4c4e8790282a49fd849a34ca11bef5bf
I20260925 09:57:18.285992 11603 ts_tablet_manager.cc:1939] T 4c4e8790282a49fd849a34ca11bef5bf P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:18.286763 10666 tablet_service.cc:1467] Processing CreateTablet for tablet 977a59175e5b43eeb6f23b938afd00a3 (DEFAULT_TABLE table=default.Table_ff4e7021d36b43f8b51f19960e707ac8 [id=f17a11d223204599afb24e2e9d1d9d9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:18.287614 10666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 977a59175e5b43eeb6f23b938afd00a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:18.287969 10658 tablet_service.cc:1467] Processing CreateTablet for tablet c7cd1d6a603642afa2c8c737079961b5 (DEFAULT_TABLE table=default.Table_0507f47042b9445abaf8671c979f8994 [id=55860c5dba9f4c818cfa6d523041ec5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:18.288100 11082 ts_tablet_manager.cc:1428] T 752105c1514f4c1db2a11b95b413b4f6 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.232s user 0.007s sys 0.000s
I20260925 09:57:18.288818 10658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7cd1d6a603642afa2c8c737079961b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:18.291617 11084 tablet_bootstrap.cc:654] T 9e697e7146e34841bd64660b45ebf231 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:18.295421 11603 tablet_replica.cc:330] T 91c09118b43b41eca8f7040323ba76f7 P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:18.299836 11603 raft_consensus.cc:2229] T 91c09118b43b41eca8f7040323ba76f7 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:18.304509 11603 raft_consensus.cc:2258] T 91c09118b43b41eca8f7040323ba76f7 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:18.308123 11603 ts_tablet_manager.cc:1905] T 91c09118b43b41eca8f7040323ba76f7 P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:18.312074 10808 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e882284a014455192375daad391efaf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:09 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:18.315680 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef6fefaa76d44fe798201aab2f21b9c7" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:18.316282 10836 raft_consensus.cc:2454] T ef6fefaa76d44fe798201aab2f21b9c7 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be97ab5d75934a648d4ab76341193ec0 in term 0.
I20260925 09:57:18.319088 11085 tablet_bootstrap.cc:654] T 42ec275c905343c4be1b72dc2bd0f6e3 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:18.315228 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df006163f72743b594baf1a3efd1c771" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:18.303203 10388 rpcz_store.cc:269] 0925 09:57:15.719995 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:15.886159 (+166164us) service_pool.cc:227] Handling call
0925 09:57:18.166435 (+2280276us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b4c000040c0. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabee853210 00007fabee863fb2 00007fabee864512 00007fabee82979b 00007fabee85673c 00007fabee856bfc 00007fabf01e7d5b 00007fabf2a2879b 00007fabf29d1330 00007fabf29d7083 00007fabf2ada4fe
0925 09:57:18.168597 (+ 2162us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:17.687625 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:17.687958 (+ 333us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:17.687982 (+ 24us) write_op.cc:435] Acquired schema lock
0925 09:57:17.687994 (+ 12us) tablet.cc:660] Decoding operations
0925 09:57:17.688273 (+ 279us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:17.688313 (+ 40us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:17.688335 (+ 22us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:17.688609 (+ 274us) tablet.cc:699] Row locks acquired
0925 09:57:17.688622 (+ 13us) write_op.cc:260] PREPARE: finished
0925 09:57:17.688735 (+ 113us) write_op.cc:270] Start()
0925 09:57:17.688811 (+ 76us) write_op.cc:276] Timestamp: P: 1790330237688693 usec, L: 0
0925 09:57:17.688826 (+ 15us) op_driver.cc:331] REPLICATION: starting
0925 09:57:17.689291 (+ 465us) log.cc:842] Serialized 895 byte log entry
0925 09:57:17.714659 (+ 25368us) op_driver.cc:444] REPLICATION: finished
0925 09:57:17.715051 (+ 392us) write_op.cc:301] APPLY: starting
0925 09:57:17.715102 (+ 51us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:17.715201 (+ 99us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:17.715285 (+ 84us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:17.715544 (+ 259us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:17.715690 (+ 146us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 09:57:17.715731 (+ 41us) write_op.cc:312] APPLY: finished
0925 09:57:17.715901 (+ 170us) log.cc:842] Serialized 29 byte log entry
0925 09:57:17.716042 (+ 141us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:17.716106 (+ 64us) write_op.cc:454] Released schema lock
0925 09:57:17.716136 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:17.716299 (+ 163us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:17.716322 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":15,"spinlock_wait_cycles":24356608,"child_traces":[["op",{"apply.queue_time_us":231,"apply.run_cpu_time_us":1492,"apply.run_wall_time_us":1492,"num_ops":2,"prepare.queue_time_us":217,"prepare.run_cpu_time_us":2049,"prepare.run_wall_time_us":70783,"raft.queue_time_us":14305,"raft.run_cpu_time_us":594,"raft.run_wall_time_us":594,"replication_time_us":25767}]]}
I20260925 09:57:18.328375 10817 tablet_service.cc:1514] Processing DeleteTablet for tablet 833fac37485141f6a18104eaea907b0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:14 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:18.329249 11579 tablet_replica.cc:330] T 833fac37485141f6a18104eaea907b0e P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:18.330246 10818 tablet_service.cc:1514] Processing DeleteTablet for tablet 91c09118b43b41eca8f7040323ba76f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:15 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:18.330363 10918 leader_election.cc:302] T ef6fefaa76d44fe798201aab2f21b9c7 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:18.330948 10819 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d55154ceb064f22bd6f37c8ae5dce74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:12 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:18.331105 11543 raft_consensus.cc:2781] T ef6fefaa76d44fe798201aab2f21b9c7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260925 09:57:18.331199 10918 leader_election.cc:341] T df006163f72743b594baf1a3efd1c771 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:18.331477 11543 raft_consensus.cc:482] T ef6fefaa76d44fe798201aab2f21b9c7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:18.331771 11543 raft_consensus.cc:3037] T ef6fefaa76d44fe798201aab2f21b9c7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:18.332544 10815 tablet_service.cc:1467] Processing CreateTablet for tablet 6161774f61ae48ab9764a05f2188a80b (DEFAULT_TABLE table=default.Table_15b07a280b634333985637e0f6746764 [id=7ee526720b2442119e0fc487b7c4fcd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:18.333480 10815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6161774f61ae48ab9764a05f2188a80b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:18.336339 10807 tablet_service.cc:1467] Processing CreateTablet for tablet fe91b39f91274a7ca61e9618d8cbb475 (DEFAULT_TABLE table=default.Table_a4437c88d32f4288a6d9581edb106440 [id=288fff5d48da4efe82583b6a86de2084]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:18.337409 10807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe91b39f91274a7ca61e9618d8cbb475. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:18.338191 10682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef6fefaa76d44fe798201aab2f21b9c7" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:18.338436 11543 raft_consensus.cc:504] T ef6fefaa76d44fe798201aab2f21b9c7 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:18.339424 10919 leader_election.cc:341] T df006163f72743b594baf1a3efd1c771 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:18.340029 10919 leader_election.cc:302] T df006163f72743b594baf1a3efd1c771 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:18.340848 11543 leader_election.cc:288] T ef6fefaa76d44fe798201aab2f21b9c7 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:18.343950 10806 tablet_service.cc:1467] Processing CreateTablet for tablet c7cd1d6a603642afa2c8c737079961b5 (DEFAULT_TABLE table=default.Table_0507f47042b9445abaf8671c979f8994 [id=55860c5dba9f4c818cfa6d523041ec5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:18.345337 10806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7cd1d6a603642afa2c8c737079961b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:18.348665 10621 leader_election.cc:302] T 2d55154ceb064f22bd6f37c8ae5dce74 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters: be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:18.349349 11585 raft_consensus.cc:2781] T 2d55154ceb064f22bd6f37c8ae5dce74 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:18.349668 11585 raft_consensus.cc:482] T 2d55154ceb064f22bd6f37c8ae5dce74 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:18.350121 11585 raft_consensus.cc:3037] T 2d55154ceb064f22bd6f37c8ae5dce74 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:18.339655 10813 tablet_service.cc:1514] Processing DeleteTablet for tablet a0976e0c01234b38a7ab934dc1b289c4 with delete_type TABLET_DATA_DELETED (replaced by tablet 2d087d3e35814c90a03eccd90eeea86c) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:18.353505 10809 tablet_service.cc:1514] Processing DeleteTablet for tablet fd73d0d52350444499abae34e8c02fab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:15 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:18.354899 10805 tablet_service.cc:1467] Processing CreateTablet for tablet 977a59175e5b43eeb6f23b938afd00a3 (DEFAULT_TABLE table=default.Table_ff4e7021d36b43f8b51f19960e707ac8 [id=f17a11d223204599afb24e2e9d1d9d9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:18.356920 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef6fefaa76d44fe798201aab2f21b9c7" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:18.357228 11537 leader_election.cc:288] T ef6fefaa76d44fe798201aab2f21b9c7 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:18.357453 10835 raft_consensus.cc:3037] T ef6fefaa76d44fe798201aab2f21b9c7 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:18.343114 10808 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c4e8790282a49fd849a34ca11bef5bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:11 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:18.360026 11585 raft_consensus.cc:504] T 2d55154ceb064f22bd6f37c8ae5dce74 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:18.364732 10619 leader_election.cc:341] T 2d55154ceb064f22bd6f37c8ae5dce74 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 2d55154ceb064f22bd6f37c8ae5dce74
I20260925 09:57:18.365262 11585 leader_election.cc:288] T 2d55154ceb064f22bd6f37c8ae5dce74 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:18.377732 10819 tablet_service.cc:1467] Processing CreateTablet for tablet 2358acdaa5104e3eacbab8bf182b8f30 (DEFAULT_TABLE table=default.Table_698f813273574f79887bdaac95b977ce [id=05a28766e4c74534b33c9bae2c8438d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:18.378784 10819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2358acdaa5104e3eacbab8bf182b8f30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:18.358605 11579 raft_consensus.cc:2229] T 833fac37485141f6a18104eaea907b0e P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:18.382547 11579 raft_consensus.cc:2258] T 833fac37485141f6a18104eaea907b0e P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260925 09:57:18.233299 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 91c09118b43b41eca8f7040323ba76f7 because the tablet was not found. No further retry: Not found: Tablet not found: 91c09118b43b41eca8f7040323ba76f7
W20260925 09:57:18.385334 10377 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 152055us
W20260925 09:57:18.389474 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 1969ac471b6f471ebe8e048a96c22837 because the tablet was not found. No further retry: Not found: Tablet not found: 1969ac471b6f471ebe8e048a96c22837
I20260925 09:57:18.277331 10386 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:35140 (request call id 160) took 2750 ms. Trace:
I20260925 09:57:18.294828 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 4c4e8790282a49fd849a34ca11bef5bf (table default.Table_beb33698ec744e42b4f6ac2fdd8fe2b9 [id=9c778da25cde488a953f712100b2fbb2]) successfully deleted
W20260925 09:57:18.390916 10380 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 96094.2us
W20260925 09:57:18.391861 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet fd73d0d52350444499abae34e8c02fab because the tablet was not found. No further retry: Not found: Tablet not found: fd73d0d52350444499abae34e8c02fab
W20260925 09:57:18.392454 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 0e882284a014455192375daad391efaf because the tablet was not found. No further retry: Not found: Tablet not found: 0e882284a014455192375daad391efaf
I20260925 09:57:18.395147 10666 tablet_service.cc:1467] Processing CreateTablet for tablet 2358acdaa5104e3eacbab8bf182b8f30 (DEFAULT_TABLE table=default.Table_698f813273574f79887bdaac95b977ce [id=05a28766e4c74534b33c9bae2c8438d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:18.396160 10666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2358acdaa5104e3eacbab8bf182b8f30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:18.397354 11603 ts_tablet_manager.cc:1918] T 91c09118b43b41eca8f7040323ba76f7 P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:18.397776 11603 log.cc:1196] T 91c09118b43b41eca8f7040323ba76f7 P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/91c09118b43b41eca8f7040323ba76f7
I20260925 09:57:18.399482 10805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 977a59175e5b43eeb6f23b938afd00a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:18.404275 11603 ts_tablet_manager.cc:1939] T 91c09118b43b41eca8f7040323ba76f7 P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
W20260925 09:57:18.157311 10387 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.2:40761 (request call id 23) took 2432 ms (client timeout 1000 ms). Trace:
W20260925 09:57:18.409382 10385 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.3:41469 (request call id 16) took 3080 ms (client timeout 1000 ms). Trace:
I20260925 09:57:18.409572 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 91c09118b43b41eca8f7040323ba76f7 (table default.Table_33b40a0d6ca44f15be47ca34bab6a093 [id=453beec106a64428b163c7db8132fc21]) successfully deleted
W20260925 09:57:18.409819 10385 rpcz_store.cc:269] 0925 09:57:15.329403 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:15.394174 (+ 64771us) service_pool.cc:227] Handling call
0925 09:57:18.409301 (+3015127us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":5672,"reported_tablets":30}
W20260925 09:57:18.410372 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 1969ac471b6f471ebe8e048a96c22837 because the tablet was not found. No further retry: Not found: Tablet not found: 1969ac471b6f471ebe8e048a96c22837
W20260925 09:57:18.408020 10387 rpcz_store.cc:269] 0925 09:57:15.725058 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:15.893387 (+168329us) service_pool.cc:227] Handling call
0925 09:57:18.157115 (+2263728us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":21410,"reported_tablets":16}
I20260925 09:57:18.414773 10835 raft_consensus.cc:2454] T ef6fefaa76d44fe798201aab2f21b9c7 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be97ab5d75934a648d4ab76341193ec0 in term 1.
I20260925 09:57:18.301200 11553 raft_consensus.cc:504] T ef6fefaa76d44fe798201aab2f21b9c7 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:18.416143 10682 raft_consensus.cc:2379] T ef6fefaa76d44fe798201aab2f21b9c7 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be97ab5d75934a648d4ab76341193ec0 in current term 1: Already voted for candidate b8850818e2ee42a082802d0095c08d21 in this term.
I20260925 09:57:18.416102 10918 leader_election.cc:302] T ef6fefaa76d44fe798201aab2f21b9c7 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:18.416316 10387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_d1b168dcbc624b88980354edd6eff43a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:18.417779 11579 ts_tablet_manager.cc:1905] T 833fac37485141f6a18104eaea907b0e P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:18.390724 10386 rpcz_store.cc:276] 0925 09:57:15.526791 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:15.585589 (+ 58798us) service_pool.cc:227] Handling call
0925 09:57:15.599142 (+ 13553us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:57:15.648416 (+ 49274us) catalog_manager.cc:3616] Apply alter schema
0925 09:57:15.883551 (+235135us) catalog_manager.cc:3639] Acquired catalog manager lock
0925 09:57:15.883847 (+ 296us) catalog_manager.cc:3786] Updating metadata on disk
0925 09:57:15.939870 (+ 56023us) catalog_manager.cc:3817] Committing alterations to in-memory state
0925 09:57:18.277244 (+2337374us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:15.917996 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:15.918387 (+ 391us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:15.918411 (+ 24us) write_op.cc:435] Acquired schema lock
0925 09:57:15.918423 (+ 12us) tablet.cc:660] Decoding operations
0925 09:57:15.918689 (+ 266us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:15.918725 (+ 36us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:15.918748 (+ 23us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:15.918894 (+ 146us) tablet.cc:699] Row locks acquired
0925 09:57:15.918907 (+ 13us) write_op.cc:260] PREPARE: finished
0925 09:57:15.919013 (+ 106us) write_op.cc:270] Start()
0925 09:57:15.919093 (+ 80us) write_op.cc:276] Timestamp: P: 1790330235918947 usec, L: 0
0925 09:57:15.919109 (+ 16us) op_driver.cc:331] REPLICATION: starting
0925 09:57:15.919621 (+ 512us) log.cc:842] Serialized 405 byte log entry
0925 09:57:15.933078 (+ 13457us) op_driver.cc:444] REPLICATION: finished
0925 09:57:15.933351 (+ 273us) write_op.cc:301] APPLY: starting
0925 09:57:15.933414 (+ 63us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:15.933514 (+ 100us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:15.933613 (+ 99us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:15.933787 (+ 174us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:15.933938 (+ 151us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:15.933996 (+ 58us) write_op.cc:312] APPLY: finished
0925 09:57:15.934171 (+ 175us) log.cc:842] Serialized 23 byte log entry
0925 09:57:15.934311 (+ 140us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:15.934373 (+ 62us) write_op.cc:454] Released schema lock
0925 09:57:15.934410 (+ 37us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:15.934582 (+ 172us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:15.934614 (+ 32us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":142,"apply.run_cpu_time_us":1412,"apply.run_wall_time_us":1412,"num_ops":1,"prepare.queue_time_us":32744,"prepare.run_cpu_time_us":1877,"prepare.run_wall_time_us":1879,"raft.queue_time_us":1239,"raft.run_cpu_time_us":587,"raft.run_wall_time_us":587,"replication_time_us":13900}]]}
I20260925 09:57:18.419313 11553 leader_election.cc:288] T ef6fefaa76d44fe798201aab2f21b9c7 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:18.420046 10386 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_ff4e7021d36b43f8b51f19960e707ac8" } modify_external_catalogs: true
I20260925 09:57:18.420594 10386 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_ff4e7021d36b43f8b51f19960e707ac8" } modify_external_catalogs: true
W20260925 09:57:18.159852 10392 rpcz_store.cc:269] 0925 09:57:14.474909 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:14.848205 (+373296us) service_pool.cc:227] Handling call
0925 09:57:14.919918 (+ 71713us) spinlock_profiling.cc:243] Waited 68.9 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2ad9c3e 00007fabf0610fb4 00007fabf0610f42 00007fabf0610e71 00007fabf0610dec 00007fabf06100a2 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:18.159350 (+3239432us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:15.559684 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:15.560550 (+ 866us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:15.560582 (+ 32us) write_op.cc:435] Acquired schema lock
0925 09:57:15.560600 (+ 18us) tablet.cc:660] Decoding operations
0925 09:57:15.560954 (+ 354us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:15.560986 (+ 32us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:15.561007 (+ 21us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:15.561291 (+ 284us) tablet.cc:699] Row locks acquired
0925 09:57:15.561310 (+ 19us) write_op.cc:260] PREPARE: finished
0925 09:57:15.561417 (+ 107us) write_op.cc:270] Start()
0925 09:57:15.561493 (+ 76us) write_op.cc:276] Timestamp: P: 1790330235561379 usec, L: 0
0925 09:57:15.561508 (+ 15us) op_driver.cc:331] REPLICATION: starting
0925 09:57:15.562162 (+ 654us) log.cc:842] Serialized 895 byte log entry
0925 09:57:15.790579 (+228417us) op_driver.cc:444] REPLICATION: finished
0925 09:57:15.794158 (+ 3579us) write_op.cc:301] APPLY: starting
0925 09:57:15.794300 (+ 142us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:15.794513 (+ 213us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:15.794664 (+ 151us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:15.795034 (+ 370us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:15.795406 (+ 372us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 09:57:15.795472 (+ 66us) write_op.cc:312] APPLY: finished
0925 09:57:15.795673 (+ 201us) log.cc:842] Serialized 29 byte log entry
0925 09:57:15.799118 (+ 3445us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:15.799216 (+ 98us) write_op.cc:454] Released schema lock
0925 09:57:15.799250 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:15.807012 (+ 7762us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:15.807097 (+ 85us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4420,"reported_tablets":14,"spinlock_wait_cycles":158503168,"child_traces":[["op",{"apply.queue_time_us":3341,"apply.run_cpu_time_us":2475,"apply.run_wall_time_us":13625,"mutex_wait_us":80264,"num_ops":2,"prepare.queue_time_us":89563,"prepare.run_cpu_time_us":2214,"prepare.run_wall_time_us":4236,"raft.queue_time_us":84,"raft.run_cpu_time_us":10,"raft.run_wall_time_us":12,"replication_time_us":228991,"thread_start_us":75710,"threads_started":1}]]}
W20260925 09:57:18.164160 10393 rpcz_store.cc:269] 0925 09:57:17.034109 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:17.486487 (+452378us) service_pool.cc:227] Handling call
0925 09:57:17.961439 (+474952us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:17.961464 (+ 25us) write_op.cc:454] Released schema lock
0925 09:57:18.163493 (+202029us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:17.935628 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:17.936255 (+ 627us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:17.936292 (+ 37us) write_op.cc:435] Acquired schema lock
0925 09:57:17.936313 (+ 21us) tablet.cc:660] Decoding operations
0925 09:57:17.937160 (+ 847us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:17.937240 (+ 80us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:17.937277 (+ 37us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:17.937654 (+ 377us) tablet.cc:699] Row locks acquired
0925 09:57:17.937675 (+ 21us) write_op.cc:260] PREPARE: finished
0925 09:57:17.937826 (+ 151us) write_op.cc:270] Start()
0925 09:57:17.937940 (+ 114us) write_op.cc:276] Timestamp: P: 1790330237937749 usec, L: 0
0925 09:57:17.937964 (+ 24us) op_driver.cc:331] REPLICATION: starting
0925 09:57:17.938648 (+ 684us) log.cc:842] Serialized 895 byte log entry
0925 09:57:17.948276 (+ 9628us) op_driver.cc:444] REPLICATION: finished
0925 09:57:17.948850 (+ 574us) write_op.cc:301] APPLY: starting
0925 09:57:17.948905 (+ 55us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:17.949009 (+ 104us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:17.949092 (+ 83us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:17.949512 (+ 420us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:17.949646 (+ 134us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 09:57:17.949687 (+ 41us) write_op.cc:312] APPLY: finished
0925 09:57:17.949869 (+ 182us) log.cc:842] Serialized 29 byte log entry
0925 09:57:17.949999 (+ 130us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:17.950063 (+ 64us) write_op.cc:454] Released schema lock
0925 09:57:17.950097 (+ 34us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":17,"child_traces":[["op",{"apply.queue_time_us":429,"apply.run_cpu_time_us":1427,"apply.run_wall_time_us":1426,"num_ops":2,"prepare.queue_time_us":8445,"prepare.run_cpu_time_us":2913,"prepare.run_wall_time_us":25774,"raft.queue_time_us":85,"raft.run_cpu_time_us":634,"raft.run_wall_time_us":634,"replication_time_us":10241}]]}
I20260925 09:57:18.349926 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 1969ac471b6f471ebe8e048a96c22837 (table default.Table_ff4e7021d36b43f8b51f19960e707ac8 [id=f17a11d223204599afb24e2e9d1d9d9d]) successfully deleted
W20260925 09:57:18.422875 10379 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 72966us
W20260925 09:57:18.426905 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 0e882284a014455192375daad391efaf because the tablet was not found. No further retry: Not found: Tablet not found: 0e882284a014455192375daad391efaf
I20260925 09:57:18.427000 11537 raft_consensus.cc:482] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:18.428259 10393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_08049956307d45478571dcca6a46356f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:18.435647 10393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_08049956307d45478571dcca6a46356f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:18.420501 10980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef6fefaa76d44fe798201aab2f21b9c7" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:18.437983 10958 tablet_service.cc:1467] Processing CreateTablet for tablet c7cd1d6a603642afa2c8c737079961b5 (DEFAULT_TABLE table=default.Table_0507f47042b9445abaf8671c979f8994 [id=55860c5dba9f4c818cfa6d523041ec5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:18.444787 10958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7cd1d6a603642afa2c8c737079961b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:18.439589 10962 tablet_service.cc:1514] Processing DeleteTablet for tablet 752105c1514f4c1db2a11b95b413b4f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:14 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:18.427474 11537 raft_consensus.cc:504] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:18.450832 11537 leader_election.cc:288] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:18.451481 11537 raft_consensus.cc:482] T 752105c1514f4c1db2a11b95b413b4f6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260925 09:57:18.453712 10816 consensus_meta.cc:287] T a44d9f2f7e3e48aeb0f31fa29ae1988a P c43461c4ee854ae1ac52b873ccada318: Time spent flushing consensus metadata: real 0.564s user 0.007s sys 0.001s
W20260925 09:57:18.429306 10379 catalog_manager.cc:4914] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): alter failed for tablet 1969ac471b6f471ebe8e048a96c22837 (table default.Table_ff4e7021d36b43f8b51f19960e707ac8 [id=f17a11d223204599afb24e2e9d1d9d9d]),no further retry: Not found: Tablet not found: 1969ac471b6f471ebe8e048a96c22837
I20260925 09:57:18.440266 10954 tablet_service.cc:1514] Processing DeleteTablet for tablet fd73d0d52350444499abae34e8c02fab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:15 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:18.489377 11604 tablet_replica.cc:330] T 752105c1514f4c1db2a11b95b413b4f6 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:18.439898 10960 tablet_service.cc:1467] Processing CreateTablet for tablet 2358acdaa5104e3eacbab8bf182b8f30 (DEFAULT_TABLE table=default.Table_698f813273574f79887bdaac95b977ce [id=05a28766e4c74534b33c9bae2c8438d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:18.490765 10960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2358acdaa5104e3eacbab8bf182b8f30. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:18.493129 10379 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 28d60531970a4ab58ce5ef99cb3a96bf (table default.Table_ce09d3a8d8e84973894f33b18bbfdb22 [id=6851500c7dba44f1bf353bb79bafd97a], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 28d60531970a4ab58ce5ef99cb3a96bf consensus state has no leader
I20260925 09:57:18.457557 10961 tablet_service.cc:1467] Processing CreateTablet for tablet fe91b39f91274a7ca61e9618d8cbb475 (DEFAULT_TABLE table=default.Table_a4437c88d32f4288a6d9581edb106440 [id=288fff5d48da4efe82583b6a86de2084]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:18.498715 10952 tablet_service.cc:1514] Processing DeleteTablet for tablet 41df75fe4b8e4856bdbb5ead3cf4fd24 with delete_type TABLET_DATA_DELETED (replaced by tablet c7cd1d6a603642afa2c8c737079961b5) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:18.439060 10953 tablet_service.cc:1467] Processing CreateTablet for tablet 977a59175e5b43eeb6f23b938afd00a3 (DEFAULT_TABLE table=default.Table_ff4e7021d36b43f8b51f19960e707ac8 [id=f17a11d223204599afb24e2e9d1d9d9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:18.499460 10961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe91b39f91274a7ca61e9618d8cbb475. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:18.500202 10953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 977a59175e5b43eeb6f23b938afd00a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:18.509635 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef6fefaa76d44fe798201aab2f21b9c7" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:18.411818 11565 raft_consensus.cc:482] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:18.439939 10956 tablet_service.cc:1514] Processing DeleteTablet for tablet a0976e0c01234b38a7ab934dc1b289c4 with delete_type TABLET_DATA_DELETED (replaced by tablet 2d087d3e35814c90a03eccd90eeea86c) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:18.442734 10957 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d55154ceb064f22bd6f37c8ae5dce74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:12 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:18.425796 11605 raft_consensus.cc:2781] T ef6fefaa76d44fe798201aab2f21b9c7 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election won for term 1
W20260925 09:57:18.412317 10919 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 69177.1us
W20260925 09:57:18.372552 10384 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.3:41469 (request call id 17) took 1975 ms (client timeout 1000 ms). Trace:
W20260925 09:57:18.182862 10391 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.2:40761 (request call id 24) took 1334 ms (client timeout 1000 ms). Trace:
W20260925 09:57:18.522069 10387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d1b168dcbc624b88980354edd6eff43a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:18.431012 10392 master_service.cc:946] ReplaceTablet: received request to replace tablet c16e096038084b2b9148390b6e773294 from {username='slave'} at 127.0.0.1:35140
W20260925 09:57:18.522423 10391 rpcz_store.cc:269] 0925 09:57:16.848350 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:17.457175 (+608825us) service_pool.cc:227] Handling call
0925 09:57:18.182762 (+725587us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":477,"reported_tablets":21}
I20260925 09:57:18.368831 10390 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 0a2bbe141e564a96be05de86f2993234 (table default.Table_988762e7d6264b1cb64927ff31711470 [id=aed8a2ddc51545ad9cec4d272dcf3daf]). Expected version 2 got 1
I20260925 09:57:18.523905 10391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_5bb8cffc4b434151b44aa2a11097fa19"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:18.523943 10390 catalog_manager.cc:5495] T 3c31682bf400494f82ee3ce5e6eec441 P c43461c4ee854ae1ac52b873ccada318 reported cstate change: term changed from 0 to 1, leader changed from <none> to c43461c4ee854ae1ac52b873ccada318 (127.10.9.2). New cstate: current_term: 1 leader_uuid: "c43461c4ee854ae1ac52b873ccada318" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
W20260925 09:57:18.525339 10391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5bb8cffc4b434151b44aa2a11097fa19 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:18.525906 10390 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 28d60531970a4ab58ce5ef99cb3a96bf (table default.Table_ce09d3a8d8e84973894f33b18bbfdb22 [id=6851500c7dba44f1bf353bb79bafd97a]). Expected version 2 got 0
I20260925 09:57:18.526933 10390 catalog_manager.cc:5495] T 47a9fb9392244e1d89f8af2bfb42edd6 P c43461c4ee854ae1ac52b873ccada318 reported cstate change: term changed from 1 to 2, leader changed from b8850818e2ee42a082802d0095c08d21 (127.10.9.1) to c43461c4ee854ae1ac52b873ccada318 (127.10.9.2). New cstate: current_term: 2 leader_uuid: "c43461c4ee854ae1ac52b873ccada318" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:18.535900 10390 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet c07c6205d1284a6fbb369c010a226da7 (table default.Table_5a6ecccec7f44c31bbfe8d64580ec33b [id=2d6bc0e2aaee4b1483b01fc678505bb0]). Expected version 1 got 0
I20260925 09:57:18.538746 10390 catalog_manager.cc:5495] T ef22f64036684dce9d7c69c37722f468 P c43461c4ee854ae1ac52b873ccada318 reported cstate change: term changed from 0 to 1, leader changed from <none> to c43461c4ee854ae1ac52b873ccada318 (127.10.9.2). New cstate: current_term: 1 leader_uuid: "c43461c4ee854ae1ac52b873ccada318" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:18.507174 11082 tablet_bootstrap.cc:492] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:18.535593 11605 raft_consensus.cc:686] T ef6fefaa76d44fe798201aab2f21b9c7 P be97ab5d75934a648d4ab76341193ec0 [term 1 LEADER]: Becoming Leader. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Running, Role: LEADER
I20260925 09:57:18.411971 11543 raft_consensus.cc:2726] T df006163f72743b594baf1a3efd1c771 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:18.451962 11537 raft_consensus.cc:504] T 752105c1514f4c1db2a11b95b413b4f6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:18.549306 11604 raft_consensus.cc:2229] T 752105c1514f4c1db2a11b95b413b4f6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:18.566829 11604 raft_consensus.cc:2258] T 752105c1514f4c1db2a11b95b413b4f6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:18.442365 10955 tablet_service.cc:1514] Processing DeleteTablet for tablet 833fac37485141f6a18104eaea907b0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:14 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:18.551894 11565 raft_consensus.cc:504] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:18.568841 11604 ts_tablet_manager.cc:1905] T 752105c1514f4c1db2a11b95b413b4f6 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 09:57:18.558625 11612 log.cc:924] Time spent T 3be0eccc65564bfba3d6e5c398c61fb6 P be97ab5d75934a648d4ab76341193ec0: Append to log took a long time: real 0.177s user 0.000s sys 0.002s
I20260925 09:57:18.411124 10385 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_eb065d5979c64931abf91d82b8623a0a" } modify_external_catalogs: true
I20260925 09:57:18.577004 11605 consensus_queue.cc:237] T ef6fefaa76d44fe798201aab2f21b9c7 P be97ab5d75934a648d4ab76341193ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:18.578327 10682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef6fefaa76d44fe798201aab2f21b9c7" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:18.579100 10682 raft_consensus.cc:2379] T ef6fefaa76d44fe798201aab2f21b9c7 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate be97ab5d75934a648d4ab76341193ec0 in current term 1: Already voted for candidate b8850818e2ee42a082802d0095c08d21 in this term.
I20260925 09:57:18.579221 11565 leader_election.cc:288] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:18.581820 11608 tablet.cc:1722] T 0a2bbe141e564a96be05de86f2993234 P b8850818e2ee42a082802d0095c08d21: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 09:57:18.585891 11084 tablet_bootstrap.cc:492] T 9e697e7146e34841bd64660b45ebf231 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:18.586284 11084 ts_tablet_manager.cc:1397] T 9e697e7146e34841bd64660b45ebf231 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.314s user 0.014s sys 0.038s
I20260925 09:57:18.588331 10682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a21909e2e5794f57971f4e42795235c7" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:18.441064 10959 tablet_service.cc:1467] Processing CreateTablet for tablet 6161774f61ae48ab9764a05f2188a80b (DEFAULT_TABLE table=default.Table_15b07a280b634333985637e0f6746764 [id=7ee526720b2442119e0fc487b7c4fcd4]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 09:57:18.590134 10919 leader_election.cc:341] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:18.590713 10959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6161774f61ae48ab9764a05f2188a80b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:18.591892 11084 raft_consensus.cc:348] T 9e697e7146e34841bd64660b45ebf231 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:18.525732 10384 rpcz_store.cc:269] 0925 09:57:16.397312 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:16.520911 (+123599us) service_pool.cc:227] Handling call
0925 09:57:18.372454 (+1851543us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:17.907453 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:17.907785 (+ 332us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:17.907806 (+ 21us) write_op.cc:435] Acquired schema lock
0925 09:57:17.907818 (+ 12us) tablet.cc:660] Decoding operations
0925 09:57:17.908120 (+ 302us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:17.908160 (+ 40us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:17.908190 (+ 30us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:17.908319 (+ 129us) tablet.cc:699] Row locks acquired
0925 09:57:17.908332 (+ 13us) write_op.cc:260] PREPARE: finished
0925 09:57:17.908427 (+ 95us) write_op.cc:270] Start()
0925 09:57:17.908502 (+ 75us) write_op.cc:276] Timestamp: P: 1790330237908368 usec, L: 0
0925 09:57:17.908518 (+ 16us) op_driver.cc:331] REPLICATION: starting
0925 09:57:17.908962 (+ 444us) log.cc:842] Serialized 356 byte log entry
0925 09:57:17.918999 (+ 10037us) op_driver.cc:444] REPLICATION: finished
0925 09:57:17.919286 (+ 287us) write_op.cc:301] APPLY: starting
0925 09:57:17.919345 (+ 59us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:17.919468 (+ 123us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:17.919570 (+ 102us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:17.919758 (+ 188us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:17.919914 (+ 156us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:17.919961 (+ 47us) write_op.cc:312] APPLY: finished
0925 09:57:17.920133 (+ 172us) log.cc:842] Serialized 23 byte log entry
0925 09:57:17.920276 (+ 143us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:17.920344 (+ 68us) write_op.cc:454] Released schema lock
0925 09:57:17.920376 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:17.920553 (+ 177us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:17.920580 (+ 27us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":32,"child_traces":[["op",{"apply.queue_time_us":150,"apply.run_cpu_time_us":1456,"apply.run_wall_time_us":1456,"num_ops":1,"prepare.queue_time_us":14155,"prepare.run_cpu_time_us":1769,"prepare.run_wall_time_us":4410,"raft.queue_time_us":172,"raft.run_cpu_time_us":619,"raft.run_wall_time_us":6177,"replication_time_us":10411}]]}
I20260925 09:57:18.592532 11084 raft_consensus.cc:374] T 9e697e7146e34841bd64660b45ebf231 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:18.592762 11084 raft_consensus.cc:729] T 9e697e7146e34841bd64660b45ebf231 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:18.593329 11084 consensus_queue.cc:260] T 9e697e7146e34841bd64660b45ebf231 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:18.604426 11537 leader_election.cc:288] T 752105c1514f4c1db2a11b95b413b4f6 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:18.586374 11609 tablet.cc:1722] T 0a2bbe141e564a96be05de86f2993234 P be97ab5d75934a648d4ab76341193ec0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 09:57:18.613097 10386 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 977a59175e5b43eeb6f23b938afd00a3
I20260925 09:57:18.614462 10386 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:35140 (request call id 172) took 1763 ms. Trace:
I20260925 09:57:18.614842 10386 rpcz_store.cc:276] 0925 09:57:16.851093 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:18.419722 (+1568629us) service_pool.cc:227] Handling call
0925 09:57:18.421044 (+ 1322us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:57:18.468560 (+ 47516us) catalog_manager.cc:2752] Modifying in-memory table state
0925 09:57:18.468712 (+ 152us) catalog_manager.cc:2761] Locking tablets
0925 09:57:18.469375 (+ 663us) catalog_manager.cc:2775] Removing table and tablets from system table
0925 09:57:18.503147 (+ 33772us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:18.503171 (+ 24us) write_op.cc:454] Released schema lock
0925 09:57:18.503415 (+ 244us) catalog_manager.cc:2793] Removing table from by-name map
0925 09:57:18.612897 (+109482us) catalog_manager.cc:2809] Committing in-memory state
0925 09:57:18.613011 (+ 114us) catalog_manager.cc:2813] Aborting table tasks
0925 09:57:18.614410 (+ 1399us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:18.475684 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:18.476098 (+ 414us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:18.476129 (+ 31us) write_op.cc:435] Acquired schema lock
0925 09:57:18.476148 (+ 19us) tablet.cc:660] Decoding operations
0925 09:57:18.476500 (+ 352us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:18.476549 (+ 49us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:18.476581 (+ 32us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:18.476873 (+ 292us) tablet.cc:699] Row locks acquired
0925 09:57:18.476886 (+ 13us) write_op.cc:260] PREPARE: finished
0925 09:57:18.476995 (+ 109us) write_op.cc:270] Start()
0925 09:57:18.477076 (+ 81us) write_op.cc:276] Timestamp: P: 1790330238476926 usec, L: 0
0925 09:57:18.477091 (+ 15us) op_driver.cc:331] REPLICATION: starting
0925 09:57:18.477546 (+ 455us) log.cc:842] Serialized 753 byte log entry
0925 09:57:18.485734 (+ 8188us) op_driver.cc:444] REPLICATION: finished
0925 09:57:18.486153 (+ 419us) write_op.cc:301] APPLY: starting
0925 09:57:18.486291 (+ 138us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:18.486439 (+ 148us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:18.486530 (+ 91us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:18.486764 (+ 234us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:18.486902 (+ 138us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 09:57:18.486962 (+ 60us) write_op.cc:312] APPLY: finished
0925 09:57:18.487148 (+ 186us) log.cc:842] Serialized 29 byte log entry
0925 09:57:18.487294 (+ 146us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:18.487365 (+ 71us) write_op.cc:454] Released schema lock
0925 09:57:18.487924 (+ 559us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":279,"apply.run_cpu_time_us":1457,"apply.run_wall_time_us":1948,"num_ops":2,"prepare.queue_time_us":4260,"prepare.run_cpu_time_us":2160,"prepare.run_wall_time_us":2160,"raft.queue_time_us":785,"raft.run_cpu_time_us":942,"raft.run_wall_time_us":957,"replication_time_us":8568,"spinlock_wait_cycles":469248}]]}
I20260925 09:57:18.619837 10960 tablet_service.cc:1514] Processing DeleteTablet for tablet 977a59175e5b43eeb6f23b938afd00a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:18 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:18.631598 11084 ts_tablet_manager.cc:1428] T 9e697e7146e34841bd64660b45ebf231 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.045s user 0.006s sys 0.001s
I20260925 09:57:18.619833 10393 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35140 (request call id 173) took 1707 ms. Trace:
I20260925 09:57:18.640210 11084 tablet_bootstrap.cc:492] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:18.638116 10393 rpcz_store.cc:276] 0925 09:57:16.912403 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:18.427788 (+1515385us) service_pool.cc:227] Handling call
0925 09:57:18.437738 (+ 9950us) catalog_manager.cc:2075] Acquired catalog manager lock
0925 09:57:18.466725 (+ 28987us) catalog_manager.cc:2121] Created new table and tablet info
0925 09:57:18.507832 (+ 41107us) catalog_manager.cc:2181] Wrote table and tablets to system table
0925 09:57:18.618597 (+110765us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0925 09:57:18.619776 (+ 1179us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:18.494785 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:18.495173 (+ 388us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:18.495199 (+ 26us) write_op.cc:435] Acquired schema lock
0925 09:57:18.495226 (+ 27us) tablet.cc:660] Decoding operations
0925 09:57:18.495529 (+ 303us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:18.495567 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:18.495588 (+ 21us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:18.495762 (+ 174us) tablet.cc:699] Row locks acquired
0925 09:57:18.495776 (+ 14us) write_op.cc:260] PREPARE: finished
0925 09:57:18.495878 (+ 102us) write_op.cc:270] Start()
0925 09:57:18.495951 (+ 73us) write_op.cc:276] Timestamp: P: 1790330238495808 usec, L: 0
0925 09:57:18.495965 (+ 14us) op_driver.cc:331] REPLICATION: starting
0925 09:57:18.496428 (+ 463us) log.cc:842] Serialized 433 byte log entry
0925 09:57:18.505166 (+ 8738us) op_driver.cc:444] REPLICATION: finished
0925 09:57:18.505548 (+ 382us) write_op.cc:301] APPLY: starting
0925 09:57:18.505600 (+ 52us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:18.505697 (+ 97us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:18.505782 (+ 85us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:18.506158 (+ 376us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:18.506295 (+ 137us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 09:57:18.506337 (+ 42us) write_op.cc:312] APPLY: finished
0925 09:57:18.506509 (+ 172us) log.cc:842] Serialized 29 byte log entry
0925 09:57:18.506675 (+ 166us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:18.506740 (+ 65us) write_op.cc:454] Released schema lock
0925 09:57:18.506770 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:18.506924 (+ 154us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:18.506946 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":231,"apply.run_cpu_time_us":1557,"apply.run_wall_time_us":1557,"num_ops":2,"prepare.queue_time_us":103,"prepare.run_cpu_time_us":1926,"prepare.run_wall_time_us":8750,"raft.queue_time_us":70,"raft.run_cpu_time_us":14,"raft.run_wall_time_us":15,"replication_time_us":9135}]]}
I20260925 09:57:18.642891 10393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_820f0ea3147b431b94658411fadaf4bb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:18.619333 10385 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_eb065d5979c64931abf91d82b8623a0a" } modify_external_catalogs: true
W20260925 09:57:18.649089 10393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_820f0ea3147b431b94658411fadaf4bb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:18.651188 11082 tablet_bootstrap.cc:654] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:18.649935 10386 master_service.cc:946] ReplaceTablet: received request to replace tablet c07c6205d1284a6fbb369c010a226da7 from {username='slave'} at 127.0.0.1:35140
I20260925 09:57:18.660423 10658 tablet_service.cc:1514] Processing DeleteTablet for tablet 977a59175e5b43eeb6f23b938afd00a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:18 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:18.661029 11084 tablet_bootstrap.cc:654] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:18.661945 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 977a59175e5b43eeb6f23b938afd00a3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 977a59175e5b43eeb6f23b938afd00a3 already in progress: creating tablet
I20260925 09:57:18.668303 10391 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_5a6ecccec7f44c31bbfe8d64580ec33b" } new_table_name: "default.Table_8e186a99ccba425b9b588041d2364a40" modify_external_catalogs: true
I20260925 09:57:18.681523 11084 tablet_bootstrap.cc:492] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:18.682008 11084 ts_tablet_manager.cc:1397] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.042s user 0.009s sys 0.000s
I20260925 09:57:18.684288 11084 raft_consensus.cc:348] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:18.684890 11084 raft_consensus.cc:374] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:18.685113 11084 raft_consensus.cc:729] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:18.685678 11084 consensus_queue.cc:260] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:18.687258 10387 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35140 (request call id 171) took 2271 ms. Trace:
I20260925 09:57:18.688378 10385 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet a44d9f2f7e3e48aeb0f31fa29ae1988a
I20260925 09:57:18.690418 10385 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:35140 (request call id 170) took 2277 ms. Trace:
I20260925 09:57:18.690721 10961 tablet_service.cc:1514] Processing DeleteTablet for tablet a44d9f2f7e3e48aeb0f31fa29ae1988a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:18 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:18.691548 10659 tablet_service.cc:1514] Processing DeleteTablet for tablet a44d9f2f7e3e48aeb0f31fa29ae1988a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:18 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:18.694195 11082 tablet_bootstrap.cc:492] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:18.694571 11082 ts_tablet_manager.cc:1397] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.188s user 0.012s sys 0.000s
W20260925 09:57:18.573137 10775 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 269068us
I20260925 09:57:18.687616 10387 rpcz_store.cc:276] 0925 09:57:16.415782 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:18.415873 (+2000091us) service_pool.cc:227] Handling call
0925 09:57:18.620171 (+204298us) catalog_manager.cc:2075] Acquired catalog manager lock
0925 09:57:18.621649 (+ 1478us) catalog_manager.cc:2121] Created new table and tablet info
0925 09:57:18.686538 (+ 64889us) catalog_manager.cc:2181] Wrote table and tablets to system table
0925 09:57:18.686980 (+ 442us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0925 09:57:18.687206 (+ 226us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:18.639547 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:18.639901 (+ 354us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:18.639921 (+ 20us) write_op.cc:435] Acquired schema lock
0925 09:57:18.639934 (+ 13us) tablet.cc:660] Decoding operations
0925 09:57:18.640182 (+ 248us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:18.640224 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:18.640245 (+ 21us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:18.640425 (+ 180us) tablet.cc:699] Row locks acquired
0925 09:57:18.640440 (+ 15us) write_op.cc:260] PREPARE: finished
0925 09:57:18.640547 (+ 107us) write_op.cc:270] Start()
0925 09:57:18.640623 (+ 76us) write_op.cc:276] Timestamp: P: 1790330238640507 usec, L: 0
0925 09:57:18.640638 (+ 15us) op_driver.cc:331] REPLICATION: starting
0925 09:57:18.641089 (+ 451us) log.cc:842] Serialized 433 byte log entry
0925 09:57:18.680802 (+ 39713us) op_driver.cc:444] REPLICATION: finished
0925 09:57:18.683642 (+ 2840us) write_op.cc:301] APPLY: starting
0925 09:57:18.683701 (+ 59us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:18.683831 (+ 130us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:18.683952 (+ 121us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:18.684496 (+ 544us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:18.684664 (+ 168us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 09:57:18.684715 (+ 51us) write_op.cc:312] APPLY: finished
0925 09:57:18.684902 (+ 187us) log.cc:842] Serialized 29 byte log entry
0925 09:57:18.685044 (+ 142us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:18.685106 (+ 62us) write_op.cc:454] Released schema lock
0925 09:57:18.685136 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:18.685314 (+ 178us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:18.685335 (+ 21us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2675,"apply.run_cpu_time_us":1874,"apply.run_wall_time_us":1874,"num_ops":2,"prepare.queue_time_us":3854,"prepare.run_cpu_time_us":1800,"prepare.run_wall_time_us":3504,"raft.queue_time_us":83,"raft.run_cpu_time_us":15,"raft.run_wall_time_us":15,"replication_time_us":40088}]]}
W20260925 09:57:18.693009 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet a44d9f2f7e3e48aeb0f31fa29ae1988a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a44d9f2f7e3e48aeb0f31fa29ae1988a already in progress: creating tablet
I20260925 09:57:18.698810 11604 ts_tablet_manager.cc:1918] T 752105c1514f4c1db2a11b95b413b4f6 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:18.703608 11084 ts_tablet_manager.cc:1428] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.021s user 0.007s sys 0.000s
I20260925 09:57:18.704203 11604 log.cc:1196] T 752105c1514f4c1db2a11b95b413b4f6 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/752105c1514f4c1db2a11b95b413b4f6
I20260925 09:57:18.707243 11604 ts_tablet_manager.cc:1939] T 752105c1514f4c1db2a11b95b413b4f6 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:18.707794 11082 raft_consensus.cc:348] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:18.707455 11084 tablet_bootstrap.cc:492] T c2110d4ada2d45c2b07259374465a8df P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:18.710247 11082 raft_consensus.cc:374] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:18.710912 11082 raft_consensus.cc:729] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:18.715649 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 752105c1514f4c1db2a11b95b413b4f6 (table default.Table_8c5b820f0db34e3bbea2d404afb1909a [id=fc979ab5541c41ef82a95e11f5a455ab]) successfully deleted
I20260925 09:57:18.716619 11082 consensus_queue.cc:260] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:18.722415 10685 raft_consensus.cc:3037] T ef22f64036684dce9d7c69c37722f468 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:18.722500 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "752105c1514f4c1db2a11b95b413b4f6" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:18.723136 10684 raft_consensus.cc:2454] T 752105c1514f4c1db2a11b95b413b4f6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
I20260925 09:57:18.720853 10678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef6fefaa76d44fe798201aab2f21b9c7" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:18.724447 10678 raft_consensus.cc:2379] T ef6fefaa76d44fe798201aab2f21b9c7 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c43461c4ee854ae1ac52b873ccada318 in current term 1: Already voted for candidate b8850818e2ee42a082802d0095c08d21 in this term.
I20260925 09:57:18.724884 11082 ts_tablet_manager.cc:1428] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.030s user 0.005s sys 0.000s
I20260925 09:57:18.721459 10688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41df75fe4b8e4856bdbb5ead3cf4fd24" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:18.725359 10688 raft_consensus.cc:2454] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
W20260925 09:57:18.726174 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet fd73d0d52350444499abae34e8c02fab because the tablet was not found. No further retry: Not found: Tablet not found: fd73d0d52350444499abae34e8c02fab
W20260925 09:57:18.727653 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 41df75fe4b8e4856bdbb5ead3cf4fd24 because the tablet was not found. No further retry: Not found: Tablet not found: 41df75fe4b8e4856bdbb5ead3cf4fd24
I20260925 09:57:18.722302 10682 raft_consensus.cc:1261] T 47a9fb9392244e1d89f8af2bfb42edd6 P b8850818e2ee42a082802d0095c08d21 [term 2 FOLLOWER]: Refusing update from remote peer c43461c4ee854ae1ac52b873ccada318: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260925 09:57:18.727983 10682 pending_rounds.cc:76] T 47a9fb9392244e1d89f8af2bfb42edd6 P b8850818e2ee42a082802d0095c08d21: Aborting all ops after (but not including) 0
I20260925 09:57:18.728243 10682 pending_rounds.cc:97] T 47a9fb9392244e1d89f8af2bfb42edd6 P b8850818e2ee42a082802d0095c08d21: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260925 09:57:18.728499 10682 raft_consensus.cc:2866] T 47a9fb9392244e1d89f8af2bfb42edd6 P b8850818e2ee42a082802d0095c08d21 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260925 09:57:18.729526 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet a0976e0c01234b38a7ab934dc1b289c4 because the tablet was not found. No further retry: Not found: Tablet not found: a0976e0c01234b38a7ab934dc1b289c4
W20260925 09:57:18.731364 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 2d55154ceb064f22bd6f37c8ae5dce74 because the tablet was not found. No further retry: Not found: Tablet not found: 2d55154ceb064f22bd6f37c8ae5dce74
W20260925 09:57:18.736604 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 833fac37485141f6a18104eaea907b0e because the tablet was not found. No further retry: Not found: Tablet not found: 833fac37485141f6a18104eaea907b0e
I20260925 09:57:18.739216 11084 tablet_bootstrap.cc:654] T c2110d4ada2d45c2b07259374465a8df P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:18.698871 10385 rpcz_store.cc:276] 0925 09:57:16.412714 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:18.410751 (+1998037us) service_pool.cc:227] Handling call
0925 09:57:18.643766 (+233015us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:57:18.644269 (+ 503us) catalog_manager.cc:2752] Modifying in-memory table state
0925 09:57:18.644383 (+ 114us) catalog_manager.cc:2761] Locking tablets
0925 09:57:18.644869 (+ 486us) catalog_manager.cc:2775] Removing table and tablets from system table
0925 09:57:18.687982 (+ 43113us) catalog_manager.cc:2793] Removing table from by-name map
0925 09:57:18.688189 (+ 207us) catalog_manager.cc:2809] Committing in-memory state
0925 09:57:18.688290 (+ 101us) catalog_manager.cc:2813] Aborting table tasks
0925 09:57:18.690369 (+ 2079us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:18.646471 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:18.646799 (+ 328us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:18.646819 (+ 20us) write_op.cc:435] Acquired schema lock
0925 09:57:18.646832 (+ 13us) tablet.cc:660] Decoding operations
0925 09:57:18.647097 (+ 265us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:18.647129 (+ 32us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:18.647149 (+ 20us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:18.647409 (+ 260us) tablet.cc:699] Row locks acquired
0925 09:57:18.647424 (+ 15us) write_op.cc:260] PREPARE: finished
0925 09:57:18.647517 (+ 93us) write_op.cc:270] Start()
0925 09:57:18.647588 (+ 71us) write_op.cc:276] Timestamp: P: 1790330238647481 usec, L: 0
0925 09:57:18.647603 (+ 15us) op_driver.cc:331] REPLICATION: starting
0925 09:57:18.648006 (+ 403us) log.cc:842] Serialized 722 byte log entry
0925 09:57:18.681007 (+ 33001us) op_driver.cc:444] REPLICATION: finished
0925 09:57:18.685630 (+ 4623us) write_op.cc:301] APPLY: starting
0925 09:57:18.685674 (+ 44us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:18.685766 (+ 92us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:18.685856 (+ 90us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:18.686065 (+ 209us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:18.686199 (+ 134us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 09:57:18.686237 (+ 38us) write_op.cc:312] APPLY: finished
0925 09:57:18.686396 (+ 159us) log.cc:842] Serialized 29 byte log entry
0925 09:57:18.686507 (+ 111us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:18.686557 (+ 50us) write_op.cc:454] Released schema lock
0925 09:57:18.686591 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:18.686750 (+ 159us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:18.686774 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":4497,"apply.run_cpu_time_us":1292,"apply.run_wall_time_us":1292,"num_ops":2,"prepare.queue_time_us":95,"prepare.run_cpu_time_us":1758,"prepare.run_wall_time_us":1758,"raft.queue_time_us":69,"raft.run_cpu_time_us":17,"raft.run_wall_time_us":17,"replication_time_us":33339,"threads_started":1}]]}
W20260925 09:57:18.752542 10377 catalog_manager.cc:4821] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 977a59175e5b43eeb6f23b938afd00a3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 977a59175e5b43eeb6f23b938afd00a3 already in progress: creating tablet
W20260925 09:57:18.763118 10377 catalog_manager.cc:4821] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet a44d9f2f7e3e48aeb0f31fa29ae1988a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a44d9f2f7e3e48aeb0f31fa29ae1988a already in progress: creating tablet
I20260925 09:57:18.778486 10775 leader_election.cc:302] T 752105c1514f4c1db2a11b95b413b4f6 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:18.779549 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c31682bf400494f82ee3ce5e6eec441" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:18.789531 11537 raft_consensus.cc:2781] T 752105c1514f4c1db2a11b95b413b4f6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:18.789971 11537 raft_consensus.cc:482] T 752105c1514f4c1db2a11b95b413b4f6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:18.790354 11537 raft_consensus.cc:3037] T 752105c1514f4c1db2a11b95b413b4f6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:18.793814 11084 tablet_bootstrap.cc:492] T c2110d4ada2d45c2b07259374465a8df P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:18.794410 11084 ts_tablet_manager.cc:1397] T c2110d4ada2d45c2b07259374465a8df P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.090s user 0.009s sys 0.000s
I20260925 09:57:18.794812 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef6fefaa76d44fe798201aab2f21b9c7" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:18.795578 10831 raft_consensus.cc:2379] T ef6fefaa76d44fe798201aab2f21b9c7 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b8850818e2ee42a082802d0095c08d21 in current term 1: Already voted for candidate be97ab5d75934a648d4ab76341193ec0 in this term.
I20260925 09:57:18.796926 10775 leader_election.cc:302] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:18.797561 11084 raft_consensus.cc:348] T c2110d4ada2d45c2b07259374465a8df P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:18.799908 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d55154ceb064f22bd6f37c8ae5dce74" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:18.810932 11085 tablet_bootstrap.cc:492] T 42ec275c905343c4be1b72dc2bd0f6e3 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:18.814327 10831 raft_consensus.cc:3037] T 2d55154ceb064f22bd6f37c8ae5dce74 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:18.814435 11085 ts_tablet_manager.cc:1397] T 42ec275c905343c4be1b72dc2bd0f6e3 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.631s user 0.034s sys 0.007s
I20260925 09:57:18.816661 11085 raft_consensus.cc:348] T 42ec275c905343c4be1b72dc2bd0f6e3 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:18.817291 11085 raft_consensus.cc:374] T 42ec275c905343c4be1b72dc2bd0f6e3 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:18.817519 11085 raft_consensus.cc:729] T 42ec275c905343c4be1b72dc2bd0f6e3 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:18.818045 11085 consensus_queue.cc:260] T 42ec275c905343c4be1b72dc2bd0f6e3 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:18.821269 11085 ts_tablet_manager.cc:1428] T 42ec275c905343c4be1b72dc2bd0f6e3 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260925 09:57:18.822222 11085 tablet_bootstrap.cc:492] T 689ab2b6cf1640129f6db8b113cf2fc6 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:18.825825 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef6fefaa76d44fe798201aab2f21b9c7" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:18.826835 10836 raft_consensus.cc:2379] T ef6fefaa76d44fe798201aab2f21b9c7 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b8850818e2ee42a082802d0095c08d21 in current term 1: Already voted for candidate be97ab5d75934a648d4ab76341193ec0 in this term.
I20260925 09:57:18.845239 10393 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35140 (request call id 176) took 1148 ms. Trace:
I20260925 09:57:18.848883 10384 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_da740072c87649f9855a1d7c5c277619"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:18.850283 10384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_da740072c87649f9855a1d7c5c277619 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:18.798492 11084 raft_consensus.cc:374] T c2110d4ada2d45c2b07259374465a8df P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:18.852504 11084 raft_consensus.cc:729] T c2110d4ada2d45c2b07259374465a8df P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:18.853163 11084 consensus_queue.cc:260] T c2110d4ada2d45c2b07259374465a8df P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:18.856076 11084 ts_tablet_manager.cc:1428] T c2110d4ada2d45c2b07259374465a8df P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.061s user 0.008s sys 0.000s
I20260925 09:57:18.856925 11605 raft_consensus.cc:482] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:18.857437 11605 raft_consensus.cc:504] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:18.859340 11605 leader_election.cc:288] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:18.860510 10387 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_988762e7d6264b1cb64927ff31711470" } new_table_name: "default.Table_b5c66e748a8741f78316bd5264f0f186" modify_external_catalogs: true
I20260925 09:57:18.862154 10685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aba253f22afa4200926f31a1b43e3970" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:18.863408 11084 tablet_bootstrap.cc:492] T 3b9cdad4f9144daea800553abd378e0b P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
W20260925 09:57:18.863868 10919 leader_election.cc:341] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:18.864784 11526 raft_consensus.cc:2781] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:18.865208 11526 raft_consensus.cc:482] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:18.865512 11526 raft_consensus.cc:3037] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:57:18.871491 10775 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 74682.2us
I20260925 09:57:18.876839 11610 consensus_queue.cc:1035] T 47a9fb9392244e1d89f8af2bfb42edd6 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260925 09:57:18.879606 10831 raft_consensus.cc:2454] T 2d55154ceb064f22bd6f37c8ae5dce74 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:18.885685 11610 consensus_queue.cc:1035] T ef22f64036684dce9d7c69c37722f468 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:18.887130 11084 tablet_bootstrap.cc:654] T 3b9cdad4f9144daea800553abd378e0b P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:18.897819 10392 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet c16e096038084b2b9148390b6e773294
I20260925 09:57:18.899315 10392 catalog_manager.cc:6565] ReplaceTablet: tablet c16e096038084b2b9148390b6e773294 deleted and replaced by tablet d5956be0da6441c79723e72f12dfe54a
I20260925 09:57:18.899729 10392 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:35140 (request call id 174) took 1362 ms. Trace:
I20260925 09:57:18.905573 10959 tablet_service.cc:1514] Processing DeleteTablet for tablet c16e096038084b2b9148390b6e773294 with delete_type TABLET_DATA_DELETED (replaced by tablet d5956be0da6441c79723e72f12dfe54a) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:18.900048 10392 rpcz_store.cc:276] 0925 09:57:17.537632 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:18.430877 (+893245us) service_pool.cc:227] Handling call
0925 09:57:18.899651 (+468774us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:18.777901 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:18.778434 (+ 533us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:18.778468 (+ 34us) write_op.cc:435] Acquired schema lock
0925 09:57:18.778488 (+ 20us) tablet.cc:660] Decoding operations
0925 09:57:18.779125 (+ 637us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:18.779186 (+ 61us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:18.779220 (+ 34us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:18.779488 (+ 268us) tablet.cc:699] Row locks acquired
0925 09:57:18.779507 (+ 19us) write_op.cc:260] PREPARE: finished
0925 09:57:18.779637 (+ 130us) write_op.cc:270] Start()
0925 09:57:18.779724 (+ 87us) write_op.cc:276] Timestamp: P: 1790330238779581 usec, L: 0
0925 09:57:18.779740 (+ 16us) op_driver.cc:331] REPLICATION: starting
0925 09:57:18.780267 (+ 527us) log.cc:842] Serialized 653 byte log entry
0925 09:57:18.891941 (+111674us) op_driver.cc:444] REPLICATION: finished
0925 09:57:18.892770 (+ 829us) write_op.cc:301] APPLY: starting
0925 09:57:18.892846 (+ 76us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:18.892954 (+ 108us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:18.893063 (+ 109us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:18.893522 (+ 459us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:18.893769 (+ 247us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:18.893820 (+ 51us) write_op.cc:312] APPLY: finished
0925 09:57:18.894026 (+ 206us) log.cc:842] Serialized 29 byte log entry
0925 09:57:18.894180 (+ 154us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:18.894252 (+ 72us) write_op.cc:454] Released schema lock
0925 09:57:18.894281 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:18.894455 (+ 174us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:18.894477 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":647,"apply.run_cpu_time_us":1905,"apply.run_wall_time_us":1905,"num_ops":2,"prepare.queue_time_us":26536,"prepare.run_cpu_time_us":2672,"prepare.run_wall_time_us":45766,"raft.queue_time_us":140,"raft.run_cpu_time_us":13,"raft.run_wall_time_us":14,"replication_time_us":112125}]]}
I20260925 09:57:18.911550 11084 tablet_bootstrap.cc:492] T 3b9cdad4f9144daea800553abd378e0b P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:18.913555 11085 tablet_bootstrap.cc:654] T 689ab2b6cf1640129f6db8b113cf2fc6 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:18.839490 10772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 276623us
I20260925 09:57:18.914871 11084 ts_tablet_manager.cc:1397] T 3b9cdad4f9144daea800553abd378e0b P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.052s user 0.009s sys 0.003s
I20260925 09:57:18.918097 11084 raft_consensus.cc:348] T 3b9cdad4f9144daea800553abd378e0b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:18.919046 11084 raft_consensus.cc:374] T 3b9cdad4f9144daea800553abd378e0b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:18.920792 11605 raft_consensus.cc:482] T c2110d4ada2d45c2b07259374465a8df P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:18.921370 11605 raft_consensus.cc:504] T c2110d4ada2d45c2b07259374465a8df P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:18.925208 11605 leader_election.cc:288] T c2110d4ada2d45c2b07259374465a8df P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:18.927251 10386 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet c07c6205d1284a6fbb369c010a226da7
I20260925 09:57:18.928805 10386 catalog_manager.cc:6565] ReplaceTablet: tablet c07c6205d1284a6fbb369c010a226da7 deleted and replaced by tablet 4e3120347c8e4a3eab57d6562ae40e8d
I20260925 09:57:18.929212 10386 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:35140 (request call id 177) took 1072 ms. Trace:
I20260925 09:57:18.929754 10959 tablet_service.cc:1514] Processing DeleteTablet for tablet c07c6205d1284a6fbb369c010a226da7 with delete_type TABLET_DATA_DELETED (replaced by tablet 4e3120347c8e4a3eab57d6562ae40e8d) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:18.933698 10388 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 0a2bbe141e564a96be05de86f2993234 (table default.Table_988762e7d6264b1cb64927ff31711470 [id=aed8a2ddc51545ad9cec4d272dcf3daf]). Expected version 2 got 1
I20260925 09:57:18.935434 11604 tablet_replica.cc:330] T c16e096038084b2b9148390b6e773294 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:18.936522 11604 raft_consensus.cc:2229] T c16e096038084b2b9148390b6e773294 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:18.937042 11604 raft_consensus.cc:2258] T c16e096038084b2b9148390b6e773294 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:18.937879 10388 catalog_manager.cc:5495] T 3be0eccc65564bfba3d6e5c398c61fb6 P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:18.939647 10388 catalog_manager.cc:5495] T 28d60531970a4ab58ce5ef99cb3a96bf P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: HEALTHY } } }
I20260925 09:57:18.940877 10388 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 28d60531970a4ab58ce5ef99cb3a96bf (table default.Table_ce09d3a8d8e84973894f33b18bbfdb22 [id=6851500c7dba44f1bf353bb79bafd97a]). Expected version 2 got 0
I20260925 09:57:18.941440 11526 raft_consensus.cc:504] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:18.942173 10388 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet ef6fefaa76d44fe798201aab2f21b9c7 (table default.Table_46e4c497898b454c85b7c2a299ebf3d9 [id=80b48dd413e3446eb7ac65e5f957b5dc]). Expected version 2 got 0
I20260925 09:57:18.939340 11604 ts_tablet_manager.cc:1905] T c16e096038084b2b9148390b6e773294 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:18.929533 10386 rpcz_store.cc:276] 0925 09:57:17.856636 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:18.649814 (+793178us) service_pool.cc:227] Handling call
0925 09:57:18.929133 (+279319us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:18.823797 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:18.824279 (+ 482us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:18.824311 (+ 32us) write_op.cc:435] Acquired schema lock
0925 09:57:18.824330 (+ 19us) tablet.cc:660] Decoding operations
0925 09:57:18.824702 (+ 372us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:18.824737 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:18.824757 (+ 20us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:18.824946 (+ 189us) tablet.cc:699] Row locks acquired
0925 09:57:18.824960 (+ 14us) write_op.cc:260] PREPARE: finished
0925 09:57:18.825063 (+ 103us) write_op.cc:270] Start()
0925 09:57:18.825152 (+ 89us) write_op.cc:276] Timestamp: P: 1790330238825009 usec, L: 0
0925 09:57:18.825168 (+ 16us) op_driver.cc:331] REPLICATION: starting
0925 09:57:18.825708 (+ 540us) log.cc:842] Serialized 653 byte log entry
0925 09:57:18.892141 (+ 66433us) op_driver.cc:444] REPLICATION: finished
0925 09:57:18.894811 (+ 2670us) write_op.cc:301] APPLY: starting
0925 09:57:18.894857 (+ 46us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:18.894957 (+ 100us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:18.895034 (+ 77us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:18.895318 (+ 284us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:18.925876 (+ 30558us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:18.925951 (+ 75us) write_op.cc:312] APPLY: finished
0925 09:57:18.926204 (+ 253us) log.cc:842] Serialized 29 byte log entry
0925 09:57:18.926396 (+ 192us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:18.926480 (+ 84us) write_op.cc:454] Released schema lock
0925 09:57:18.926531 (+ 51us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:18.926816 (+ 285us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:18.926886 (+ 70us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2549,"apply.run_cpu_time_us":2092,"apply.run_wall_time_us":32303,"num_ops":2,"prepare.queue_time_us":61282,"prepare.run_cpu_time_us":2159,"prepare.run_wall_time_us":22388,"replication_time_us":66911}]]}
I20260925 09:57:18.945472 11526 leader_election.cc:288] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
W20260925 09:57:18.942831 10772 leader_election.cc:341] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 41df75fe4b8e4856bdbb5ead3cf4fd24
I20260925 09:57:18.919379 11084 raft_consensus.cc:729] T 3b9cdad4f9144daea800553abd378e0b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:18.954161 11084 consensus_queue.cc:260] T 3b9cdad4f9144daea800553abd378e0b P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:18.956197 11082 tablet_bootstrap.cc:492] T 89a143a5623e4ef38027381a56c95227 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:18.956565 11084 ts_tablet_manager.cc:1428] T 3b9cdad4f9144daea800553abd378e0b P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.041s user 0.007s sys 0.000s
I20260925 09:57:18.958473 10772 leader_election.cc:302] T ef6fefaa76d44fe798201aab2f21b9c7 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:18.963577 11084 tablet_bootstrap.cc:492] T a44d9f2f7e3e48aeb0f31fa29ae1988a P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:18.964648 11082 tablet_bootstrap.cc:654] T 89a143a5623e4ef38027381a56c95227 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:18.965032 10981 raft_consensus.cc:1261] T 47a9fb9392244e1d89f8af2bfb42edd6 P be97ab5d75934a648d4ab76341193ec0 [term 2 FOLLOWER]: Refusing update from remote peer c43461c4ee854ae1ac52b873ccada318: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260925 09:57:18.980995 11084 tablet_bootstrap.cc:654] T a44d9f2f7e3e48aeb0f31fa29ae1988a P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:18.948553 11579 ts_tablet_manager.cc:1918] T 833fac37485141f6a18104eaea907b0e P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:18.990203 11537 raft_consensus.cc:504] T 752105c1514f4c1db2a11b95b413b4f6 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:18.991942 11082 tablet_bootstrap.cc:492] T 89a143a5623e4ef38027381a56c95227 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:18.992395 11082 ts_tablet_manager.cc:1397] T 89a143a5623e4ef38027381a56c95227 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.036s user 0.008s sys 0.006s
I20260925 09:57:18.992739 10386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_14d12f6871c847daa860c3a0cfee4231"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:18.994417 10386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_14d12f6871c847daa860c3a0cfee4231 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:18.994300 11082 raft_consensus.cc:348] T 89a143a5623e4ef38027381a56c95227 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:18.994863 11082 raft_consensus.cc:374] T 89a143a5623e4ef38027381a56c95227 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:18.995078 11082 raft_consensus.cc:729] T 89a143a5623e4ef38027381a56c95227 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:19.000378 10384 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_1e7379199917430eb9e027fc7bb2a4f9" } modify_external_catalogs: true
I20260925 09:57:19.000962 10384 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_1e7379199917430eb9e027fc7bb2a4f9" } modify_external_catalogs: true
I20260925 09:57:18.984550 11526 raft_consensus.cc:2726] T ef6fefaa76d44fe798201aab2f21b9c7 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:19.010653 11579 log.cc:1196] T 833fac37485141f6a18104eaea907b0e P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/833fac37485141f6a18104eaea907b0e
I20260925 09:57:19.017311 10392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_489e2c74ed6249ce9cf54541e0a491cd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:19.018802 10392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_489e2c74ed6249ce9cf54541e0a491cd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:19.019742 11579 ts_tablet_manager.cc:1939] T 833fac37485141f6a18104eaea907b0e P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:19.021095 11579 tablet_replica.cc:330] T 91c09118b43b41eca8f7040323ba76f7 P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:19.022063 11579 raft_consensus.cc:2229] T 91c09118b43b41eca8f7040323ba76f7 P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:19.022820 11579 pending_rounds.cc:62] T 91c09118b43b41eca8f7040323ba76f7 P c43461c4ee854ae1ac52b873ccada318: Trying to abort 1 pending ops.
I20260925 09:57:18.995886 11082 consensus_queue.cc:260] T 89a143a5623e4ef38027381a56c95227 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:18.845703 10393 rpcz_store.cc:276] 0925 09:57:17.697217 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:18.642434 (+945217us) service_pool.cc:227] Handling call
0925 09:57:18.655155 (+ 12721us) catalog_manager.cc:2075] Acquired catalog manager lock
0925 09:57:18.675102 (+ 19947us) catalog_manager.cc:2121] Created new table and tablet info
0925 09:57:18.844443 (+169341us) catalog_manager.cc:2181] Wrote table and tablets to system table
0925 09:57:18.844868 (+ 425us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0925 09:57:18.845144 (+ 276us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:18.685124 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:18.685539 (+ 415us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:18.685575 (+ 36us) write_op.cc:435] Acquired schema lock
0925 09:57:18.685587 (+ 12us) tablet.cc:660] Decoding operations
0925 09:57:18.685847 (+ 260us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:18.685881 (+ 34us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:18.685900 (+ 19us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:18.686079 (+ 179us) tablet.cc:699] Row locks acquired
0925 09:57:18.686092 (+ 13us) write_op.cc:260] PREPARE: finished
0925 09:57:18.686188 (+ 96us) write_op.cc:270] Start()
0925 09:57:18.686341 (+ 153us) write_op.cc:276] Timestamp: P: 1790330238686150 usec, L: 0
0925 09:57:18.686359 (+ 18us) op_driver.cc:331] REPLICATION: starting
0925 09:57:18.709227 (+ 22868us) log.cc:842] Serialized 433 byte log entry
0925 09:57:18.709819 (+ 592us) spinlock_profiling.cc:243] Waited 22.4 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90471 00007fabf0b07198 00007fabf0f68ade 00007fabf0f682b8 00007fabf0f6cba1 00007fabf0f6cb59 00007fabf0f6cae9 00007fabf0f6cab1 00007fabf0f6bdad 0000000000580784 00000000005805b9
0925 09:57:18.727142 (+ 17323us) op_driver.cc:444] REPLICATION: finished
0925 09:57:18.727879 (+ 737us) write_op.cc:301] APPLY: starting
0925 09:57:18.727934 (+ 55us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:18.730057 (+ 2123us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:18.730082 (+ 25us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:18.730490 (+ 408us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:18.730629 (+ 139us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 09:57:18.730672 (+ 43us) write_op.cc:312] APPLY: finished
0925 09:57:18.730860 (+ 188us) log.cc:842] Serialized 29 byte log entry
0925 09:57:18.731009 (+ 149us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:18.731078 (+ 69us) write_op.cc:454] Released schema lock
0925 09:57:18.731112 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:18.731300 (+ 188us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:18.731331 (+ 31us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":567,"apply.run_cpu_time_us":1630,"apply.run_wall_time_us":3630,"num_ops":2,"prepare.queue_time_us":8856,"prepare.run_cpu_time_us":2538,"prepare.run_wall_time_us":24963,"raft.queue_time_us":9293,"raft.run_cpu_time_us":960,"raft.run_wall_time_us":1370,"replication_time_us":40698,"spinlock_wait_cycles":51451264}]]}
I20260925 09:57:19.029521 10393 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_bc69fdbbaf5b4a4e8cdfe195eefb16ab" } modify_external_catalogs: true
I20260925 09:57:19.030154 10393 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_bc69fdbbaf5b4a4e8cdfe195eefb16ab" } modify_external_catalogs: true
I20260925 09:57:19.054949 11084 tablet_bootstrap.cc:492] T a44d9f2f7e3e48aeb0f31fa29ae1988a P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:19.055369 11084 ts_tablet_manager.cc:1397] T a44d9f2f7e3e48aeb0f31fa29ae1988a P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.098s user 0.019s sys 0.021s
I20260925 09:57:19.058197 11084 raft_consensus.cc:348] T a44d9f2f7e3e48aeb0f31fa29ae1988a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.058947 11084 raft_consensus.cc:374] T a44d9f2f7e3e48aeb0f31fa29ae1988a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:19.059264 11084 raft_consensus.cc:729] T a44d9f2f7e3e48aeb0f31fa29ae1988a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:19.059975 11084 consensus_queue.cc:260] T a44d9f2f7e3e48aeb0f31fa29ae1988a P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.062305 11084 ts_tablet_manager.cc:1428] T a44d9f2f7e3e48aeb0f31fa29ae1988a P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20260925 09:57:19.076061 11084 tablet_bootstrap.cc:492] T 55d14d4d8abe4ec9ac9aeef0d027e804 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:19.079223 11579 pending_rounds.cc:68] T 91c09118b43b41eca8f7040323ba76f7 P c43461c4ee854ae1ac52b873ccada318: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192652953710592 op_type: NO_OP noop_request { }
I20260925 09:57:19.037510 10807 tablet_service.cc:1514] Processing DeleteTablet for tablet 977a59175e5b43eeb6f23b938afd00a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:18 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:19.042392 11537 leader_election.cc:288] T 752105c1514f4c1db2a11b95b413b4f6 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:19.043660 10805 tablet_service.cc:1514] Processing DeleteTablet for tablet a44d9f2f7e3e48aeb0f31fa29ae1988a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:18 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:19.044477 10810 tablet_service.cc:1514] Processing DeleteTablet for tablet c16e096038084b2b9148390b6e773294 with delete_type TABLET_DATA_DELETED (replaced by tablet d5956be0da6441c79723e72f12dfe54a) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:19.045434 10819 tablet_service.cc:1514] Processing DeleteTablet for tablet c07c6205d1284a6fbb369c010a226da7 with delete_type TABLET_DATA_DELETED (replaced by tablet 4e3120347c8e4a3eab57d6562ae40e8d) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:19.052462 11085 tablet_bootstrap.cc:492] T 689ab2b6cf1640129f6db8b113cf2fc6 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:19.063714 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2110d4ada2d45c2b07259374465a8df" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:19.068159 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aba253f22afa4200926f31a1b43e3970" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:19.084724 11085 ts_tablet_manager.cc:1397] T 689ab2b6cf1640129f6db8b113cf2fc6 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.263s user 0.010s sys 0.000s
I20260925 09:57:19.085625 10621 leader_election.cc:302] T 3c31682bf400494f82ee3ce5e6eec441 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8850818e2ee42a082802d0095c08d21; no voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:19.086562 11579 raft_consensus.cc:2866] T 91c09118b43b41eca8f7040323ba76f7 P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:57:19.036520 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a21909e2e5794f57971f4e42795235c7" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:19.078850 10772 leader_election.cc:341] T 752105c1514f4c1db2a11b95b413b4f6 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 752105c1514f4c1db2a11b95b413b4f6
I20260925 09:57:19.088153 11579 raft_consensus.cc:2258] T 91c09118b43b41eca8f7040323ba76f7 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:19.094889 10659 tablet_service.cc:1514] Processing DeleteTablet for tablet c16e096038084b2b9148390b6e773294 with delete_type TABLET_DATA_DELETED (replaced by tablet d5956be0da6441c79723e72f12dfe54a) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:19.095525 11579 ts_tablet_manager.cc:1905] T 91c09118b43b41eca8f7040323ba76f7 P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:19.096056 10666 tablet_service.cc:1514] Processing DeleteTablet for tablet c07c6205d1284a6fbb369c010a226da7 with delete_type TABLET_DATA_DELETED (replaced by tablet 4e3120347c8e4a3eab57d6562ae40e8d) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:19.097194 10678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "752105c1514f4c1db2a11b95b413b4f6" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:19.098022 10678 raft_consensus.cc:3037] T 752105c1514f4c1db2a11b95b413b4f6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:19.106657 10678 raft_consensus.cc:2454] T 752105c1514f4c1db2a11b95b413b4f6 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:19.110810 10679 raft_consensus.cc:3037] T 3c31682bf400494f82ee3ce5e6eec441 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:19.137042 11082 ts_tablet_manager.cc:1428] T 89a143a5623e4ef38027381a56c95227 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.144s user 0.001s sys 0.004s
I20260925 09:57:19.137837 10688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41df75fe4b8e4856bdbb5ead3cf4fd24" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:19.138581 10688 raft_consensus.cc:3037] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:19.138932 11082 tablet_bootstrap.cc:492] T 42ec275c905343c4be1b72dc2bd0f6e3 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:19.139462 11084 tablet_bootstrap.cc:654] T 55d14d4d8abe4ec9ac9aeef0d027e804 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:19.134872 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2110d4ada2d45c2b07259374465a8df" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:19.145020 11082 tablet_bootstrap.cc:654] T 42ec275c905343c4be1b72dc2bd0f6e3 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:19.145390 10919 leader_election.cc:341] T c2110d4ada2d45c2b07259374465a8df P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:19.157335 11627 consensus_queue.cc:1035] T 3be0eccc65564bfba3d6e5c398c61fb6 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:19.158560 11585 raft_consensus.cc:2726] T 3c31682bf400494f82ee3ce5e6eec441 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:19.158865 11603 tablet_replica.cc:330] T c16e096038084b2b9148390b6e773294 P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:19.160209 11604 ts_tablet_manager.cc:1918] T c16e096038084b2b9148390b6e773294 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:19.160686 11604 log.cc:1196] T c16e096038084b2b9148390b6e773294 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/c16e096038084b2b9148390b6e773294
I20260925 09:57:19.162230 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 833fac37485141f6a18104eaea907b0e (table default.Table_8e0c5caa8df14066af83c49a50a172bf [id=fc3b3c18f00c42559612468a21e96694]) successfully deleted
I20260925 09:57:19.164358 11603 raft_consensus.cc:2229] T c16e096038084b2b9148390b6e773294 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:19.165369 11603 raft_consensus.cc:2258] T c16e096038084b2b9148390b6e773294 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:19.167538 11604 ts_tablet_manager.cc:1939] T c16e096038084b2b9148390b6e773294 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:19.168170 11603 ts_tablet_manager.cc:1905] T c16e096038084b2b9148390b6e773294 P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:19.173367 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet c16e096038084b2b9148390b6e773294 (table default.Table_70f0b47292de423097986b9a1c54f854 [id=8f2eaee8273640fb9ef73ccbe51782f1]) successfully deleted
I20260925 09:57:19.170854 11084 tablet_bootstrap.cc:492] T 55d14d4d8abe4ec9ac9aeef0d027e804 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:19.174168 11084 ts_tablet_manager.cc:1397] T 55d14d4d8abe4ec9ac9aeef0d027e804 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.098s user 0.009s sys 0.001s
I20260925 09:57:19.177304 11605 raft_consensus.cc:482] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:19.177726 11605 raft_consensus.cc:504] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.179445 11605 leader_election.cc:288] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:19.096575 11526 consensus_queue.cc:1035] T 47a9fb9392244e1d89f8af2bfb42edd6 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:19.098845 11610 consensus_queue.cc:1035] T ef22f64036684dce9d7c69c37722f468 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:19.121598 10775 leader_election.cc:302] T 752105c1514f4c1db2a11b95b413b4f6 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:19.183936 10682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d087d3e35814c90a03eccd90eeea86c" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:19.184598 10775 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 63109.2us
W20260925 09:57:19.185132 10919 leader_election.cc:341] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:19.185321 11610 raft_consensus.cc:2781] T 752105c1514f4c1db2a11b95b413b4f6 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:19.185732 11610 raft_consensus.cc:686] T 752105c1514f4c1db2a11b95b413b4f6 P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Becoming Leader. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Running, Role: LEADER
I20260925 09:57:19.186457 11610 consensus_queue.cc:237] T 752105c1514f4c1db2a11b95b413b4f6 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.187652 11616 raft_consensus.cc:482] T 42ec275c905343c4be1b72dc2bd0f6e3 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:19.187216 10688 raft_consensus.cc:2454] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:19.188077 11616 raft_consensus.cc:504] T 42ec275c905343c4be1b72dc2bd0f6e3 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.189934 11616 leader_election.cc:288] T 42ec275c905343c4be1b72dc2bd0f6e3 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:19.193799 11616 consensus_queue.cc:1035] T 3c31682bf400494f82ee3ce5e6eec441 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 09:57:19.197579 10621 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 64767.9us
I20260925 09:57:19.176594 11084 raft_consensus.cc:348] T 55d14d4d8abe4ec9ac9aeef0d027e804 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.198701 11084 raft_consensus.cc:374] T 55d14d4d8abe4ec9ac9aeef0d027e804 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:19.199151 11084 raft_consensus.cc:729] T 55d14d4d8abe4ec9ac9aeef0d027e804 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:19.200438 11604 tablet_replica.cc:330] T c07c6205d1284a6fbb369c010a226da7 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:19.202164 11604 raft_consensus.cc:2229] T c07c6205d1284a6fbb369c010a226da7 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:19.221647 11603 ts_tablet_manager.cc:1918] T c16e096038084b2b9148390b6e773294 P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:19.222469 11603 log.cc:1196] T c16e096038084b2b9148390b6e773294 P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/c16e096038084b2b9148390b6e773294
I20260925 09:57:19.223949 11603 ts_tablet_manager.cc:1939] T c16e096038084b2b9148390b6e773294 P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:19.231855 10666 tablet_service.cc:1467] Processing CreateTablet for tablet 5246648245e04e26be2abf1dbf706688 (DEFAULT_TABLE table=default.Table_820f0ea3147b431b94658411fadaf4bb [id=eb7ff3ee12dc47f69726666b9caf533f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:19.234192 10659 tablet_service.cc:1467] Processing CreateTablet for tablet e684798155d445bfb325a1b7c5c48753 (DEFAULT_TABLE table=default.Table_5bb8cffc4b434151b44aa2a11097fa19 [id=ab3c143dc00c43e78531514a114bfd16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:19.092954 11085 raft_consensus.cc:348] T 689ab2b6cf1640129f6db8b113cf2fc6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.236008 11085 raft_consensus.cc:374] T 689ab2b6cf1640129f6db8b113cf2fc6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:19.236327 11085 raft_consensus.cc:729] T 689ab2b6cf1640129f6db8b113cf2fc6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:19.239730 11604 raft_consensus.cc:2258] T c07c6205d1284a6fbb369c010a226da7 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:19.238161 10959 tablet_service.cc:1467] Processing CreateTablet for tablet 5246648245e04e26be2abf1dbf706688 (DEFAULT_TABLE table=default.Table_820f0ea3147b431b94658411fadaf4bb [id=eb7ff3ee12dc47f69726666b9caf533f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:19.240821 10959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5246648245e04e26be2abf1dbf706688. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:19.241724 11604 ts_tablet_manager.cc:1905] T c07c6205d1284a6fbb369c010a226da7 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:19.199770 11084 consensus_queue.cc:260] T 55d14d4d8abe4ec9ac9aeef0d027e804 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.243633 10621 leader_election.cc:302] T ef6fefaa76d44fe798201aab2f21b9c7 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8850818e2ee42a082802d0095c08d21; no voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:19.239519 10961 tablet_service.cc:1467] Processing CreateTablet for tablet e684798155d445bfb325a1b7c5c48753 (DEFAULT_TABLE table=default.Table_5bb8cffc4b434151b44aa2a11097fa19 [id=ab3c143dc00c43e78531514a114bfd16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:19.244760 10961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e684798155d445bfb325a1b7c5c48753. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:19.244753 10621 leader_election.cc:302] T 2d55154ceb064f22bd6f37c8ae5dce74 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters: be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:19.246276 11630 raft_consensus.cc:2726] T ef6fefaa76d44fe798201aab2f21b9c7 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260925 09:57:19.247047 11605 raft_consensus.cc:482] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:19.246935 11085 consensus_queue.cc:260] T 689ab2b6cf1640129f6db8b113cf2fc6 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.247449 11605 raft_consensus.cc:504] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.248939 11085 ts_tablet_manager.cc:1428] T 689ab2b6cf1640129f6db8b113cf2fc6 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.164s user 0.005s sys 0.000s
I20260925 09:57:19.249152 11605 leader_election.cc:288] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:19.249720 11085 tablet_bootstrap.cc:492] T 89a143a5623e4ef38027381a56c95227 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:19.250317 11084 ts_tablet_manager.cc:1428] T 55d14d4d8abe4ec9ac9aeef0d027e804 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.076s user 0.005s sys 0.000s
W20260925 09:57:19.180347 10772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 80668.1us
I20260925 09:57:19.240896 10775 leader_election.cc:302] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:19.252211 11616 raft_consensus.cc:2781] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:19.252717 11616 raft_consensus.cc:686] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Becoming Leader. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Running, Role: LEADER
I20260925 09:57:19.253492 11616 consensus_queue.cc:237] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:19.257380 10772 leader_election.cc:341] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 41df75fe4b8e4856bdbb5ead3cf4fd24
I20260925 09:57:19.257529 10658 tablet_service.cc:1467] Processing CreateTablet for tablet 417058493c6547c69e72347cef373292 (DEFAULT_TABLE table=default.Table_d1b168dcbc624b88980354edd6eff43a [id=8165aba086104ad4ab318b0128fdf3a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:19.258610 11084 tablet_bootstrap.cc:492] T 60fdc299865446c89d3e5374d573029a P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:19.258440 11537 consensus_queue.cc:1035] T 3c31682bf400494f82ee3ce5e6eec441 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:19.261730 10819 tablet_service.cc:1467] Processing CreateTablet for tablet 5246648245e04e26be2abf1dbf706688 (DEFAULT_TABLE table=default.Table_820f0ea3147b431b94658411fadaf4bb [id=eb7ff3ee12dc47f69726666b9caf533f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:19.261708 10960 tablet_service.cc:1467] Processing CreateTablet for tablet 417058493c6547c69e72347cef373292 (DEFAULT_TABLE table=default.Table_d1b168dcbc624b88980354edd6eff43a [id=8165aba086104ad4ab318b0128fdf3a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:19.262646 10819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5246648245e04e26be2abf1dbf706688. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:19.262696 10960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 417058493c6547c69e72347cef373292. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:19.269640 10664 tablet_service.cc:1467] Processing CreateTablet for tablet 5608c5f9b65b4da7b2fbb56417c92d88 (DEFAULT_TABLE table=default.Table_08049956307d45478571dcca6a46356f [id=e46d114e5d484b66bb15eb0416b5ce0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:19.272720 11082 tablet_bootstrap.cc:492] T 42ec275c905343c4be1b72dc2bd0f6e3 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:19.273132 11082 ts_tablet_manager.cc:1397] T 42ec275c905343c4be1b72dc2bd0f6e3 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.134s user 0.010s sys 0.011s
I20260925 09:57:19.275259 10953 tablet_service.cc:1467] Processing CreateTablet for tablet 5608c5f9b65b4da7b2fbb56417c92d88 (DEFAULT_TABLE table=default.Table_08049956307d45478571dcca6a46356f [id=e46d114e5d484b66bb15eb0416b5ce0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:19.276311 10953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5608c5f9b65b4da7b2fbb56417c92d88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:19.281579 10685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42ec275c905343c4be1b72dc2bd0f6e3" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:19.283636 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89a143a5623e4ef38027381a56c95227" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:19.284281 10687 raft_consensus.cc:2454] T 89a143a5623e4ef38027381a56c95227 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be97ab5d75934a648d4ab76341193ec0 in term 0.
I20260925 09:57:19.286346 10659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e684798155d445bfb325a1b7c5c48753. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:19.288614 10772 leader_election.cc:341] T 752105c1514f4c1db2a11b95b413b4f6 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 752105c1514f4c1db2a11b95b413b4f6
W20260925 09:57:19.289634 10918 leader_election.cc:341] T c2110d4ada2d45c2b07259374465a8df P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:19.289992 10918 leader_election.cc:302] T c2110d4ada2d45c2b07259374465a8df P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:19.290810 10666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5246648245e04e26be2abf1dbf706688. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:19.290953 11605 raft_consensus.cc:2726] T c2110d4ada2d45c2b07259374465a8df P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:19.292057 10918 leader_election.cc:341] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:19.292377 10918 leader_election.cc:302] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:19.292997 11084 tablet_bootstrap.cc:654] T 60fdc299865446c89d3e5374d573029a P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:19.293166 11605 raft_consensus.cc:2726] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:19.294174 10918 leader_election.cc:341] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:19.294508 10918 leader_election.cc:302] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:19.295148 11605 raft_consensus.cc:2726] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:19.296269 10664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5608c5f9b65b4da7b2fbb56417c92d88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:19.300524 10810 tablet_service.cc:1467] Processing CreateTablet for tablet e684798155d445bfb325a1b7c5c48753 (DEFAULT_TABLE table=default.Table_5bb8cffc4b434151b44aa2a11097fa19 [id=ab3c143dc00c43e78531514a114bfd16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:19.301247 10658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 417058493c6547c69e72347cef373292. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:19.301555 10810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e684798155d445bfb325a1b7c5c48753. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:19.304214 10919 leader_election.cc:302] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:19.305431 11605 raft_consensus.cc:2781] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:19.305887 11605 raft_consensus.cc:482] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:19.306291 11605 raft_consensus.cc:3037] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:19.312561 10805 tablet_service.cc:1467] Processing CreateTablet for tablet 417058493c6547c69e72347cef373292 (DEFAULT_TABLE table=default.Table_d1b168dcbc624b88980354edd6eff43a [id=8165aba086104ad4ab318b0128fdf3a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:19.313580 10805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 417058493c6547c69e72347cef373292. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:19.320464 10807 tablet_service.cc:1467] Processing CreateTablet for tablet 5608c5f9b65b4da7b2fbb56417c92d88 (DEFAULT_TABLE table=default.Table_08049956307d45478571dcca6a46356f [id=e46d114e5d484b66bb15eb0416b5ce0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:19.321444 10807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5608c5f9b65b4da7b2fbb56417c92d88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:19.339465 11605 raft_consensus.cc:504] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.351329 11084 tablet_bootstrap.cc:492] T 60fdc299865446c89d3e5374d573029a P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:19.358494 11084 ts_tablet_manager.cc:1397] T 60fdc299865446c89d3e5374d573029a P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.100s user 0.004s sys 0.005s
I20260925 09:57:19.366976 10983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42ec275c905343c4be1b72dc2bd0f6e3" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:19.367636 10983 raft_consensus.cc:2454] T 42ec275c905343c4be1b72dc2bd0f6e3 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
I20260925 09:57:19.368337 11084 raft_consensus.cc:348] T 60fdc299865446c89d3e5374d573029a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.369167 11084 raft_consensus.cc:374] T 60fdc299865446c89d3e5374d573029a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:19.369467 11084 raft_consensus.cc:729] T 60fdc299865446c89d3e5374d573029a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:19.370056 11604 ts_tablet_manager.cc:1918] T c07c6205d1284a6fbb369c010a226da7 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:19.370589 11604 log.cc:1196] T c07c6205d1284a6fbb369c010a226da7 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/c07c6205d1284a6fbb369c010a226da7
I20260925 09:57:19.370323 11084 consensus_queue.cc:260] T 60fdc299865446c89d3e5374d573029a P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.371786 11604 ts_tablet_manager.cc:1939] T c07c6205d1284a6fbb369c010a226da7 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:19.380115 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet c07c6205d1284a6fbb369c010a226da7 (table default.Table_5a6ecccec7f44c31bbfe8d64580ec33b [id=2d6bc0e2aaee4b1483b01fc678505bb0]) successfully deleted
I20260925 09:57:19.383279 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d087d3e35814c90a03eccd90eeea86c" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:19.384718 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89a143a5623e4ef38027381a56c95227" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:19.372648 11084 ts_tablet_manager.cc:1428] T 60fdc299865446c89d3e5374d573029a P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.014s user 0.005s sys 0.000s
I20260925 09:57:19.392427 11084 tablet_bootstrap.cc:492] T 977a59175e5b43eeb6f23b938afd00a3 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:19.393412 11605 leader_election.cc:288] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:19.398862 11084 tablet_bootstrap.cc:654] T 977a59175e5b43eeb6f23b938afd00a3 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:19.406628 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89a143a5623e4ef38027381a56c95227" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
W20260925 09:57:19.408116 10918 leader_election.cc:341] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:19.408531 10918 leader_election.cc:302] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:19.408793 10772 leader_election.cc:302] T 42ec275c905343c4be1b72dc2bd0f6e3 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:19.409233 11565 raft_consensus.cc:2726] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:19.409745 10918 leader_election.cc:341] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:19.409881 11616 raft_consensus.cc:2781] T 42ec275c905343c4be1b72dc2bd0f6e3 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:19.410324 11616 raft_consensus.cc:482] T 42ec275c905343c4be1b72dc2bd0f6e3 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260925 09:57:19.410761 10918 leader_election.cc:341] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:19.410835 11616 raft_consensus.cc:3037] T 42ec275c905343c4be1b72dc2bd0f6e3 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:19.418542 10385 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 28d60531970a4ab58ce5ef99cb3a96bf (table default.Table_ce09d3a8d8e84973894f33b18bbfdb22 [id=6851500c7dba44f1bf353bb79bafd97a]). Expected version 2 got 0
I20260925 09:57:19.424628 11634 raft_consensus.cc:2781] T 2d55154ceb064f22bd6f37c8ae5dce74 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:19.425122 11634 raft_consensus.cc:686] T 2d55154ceb064f22bd6f37c8ae5dce74 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:19.425884 11634 consensus_queue.cc:237] T 2d55154ceb064f22bd6f37c8ae5dce74 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.428153 11082 raft_consensus.cc:348] T 42ec275c905343c4be1b72dc2bd0f6e3 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.428812 11082 raft_consensus.cc:374] T 42ec275c905343c4be1b72dc2bd0f6e3 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:19.429077 11082 raft_consensus.cc:729] T 42ec275c905343c4be1b72dc2bd0f6e3 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:19.429649 11082 consensus_queue.cc:260] T 42ec275c905343c4be1b72dc2bd0f6e3 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.432188 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89a143a5623e4ef38027381a56c95227" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:19.432703 10687 raft_consensus.cc:3037] T 89a143a5623e4ef38027381a56c95227 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:57:19.434195 10621 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 187713us
I20260925 09:57:19.434648 11082 ts_tablet_manager.cc:1428] T 42ec275c905343c4be1b72dc2bd0f6e3 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.161s user 0.007s sys 0.001s
I20260925 09:57:19.435509 11082 tablet_bootstrap.cc:492] T 689ab2b6cf1640129f6db8b113cf2fc6 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
W20260925 09:57:19.452293 10772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 43619.6us
I20260925 09:57:19.457017 11603 tablet_replica.cc:330] T c07c6205d1284a6fbb369c010a226da7 P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:19.462249 11603 raft_consensus.cc:2229] T c07c6205d1284a6fbb369c010a226da7 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:19.466722 11603 raft_consensus.cc:2258] T c07c6205d1284a6fbb369c010a226da7 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 09:57:19.471334 10775 leader_election.cc:341] T 42ec275c905343c4be1b72dc2bd0f6e3 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:19.473925 11603 ts_tablet_manager.cc:1905] T c07c6205d1284a6fbb369c010a226da7 P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:19.478034 11084 tablet_bootstrap.cc:492] T 977a59175e5b43eeb6f23b938afd00a3 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:19.478559 11084 ts_tablet_manager.cc:1397] T 977a59175e5b43eeb6f23b938afd00a3 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.086s user 0.014s sys 0.013s
I20260925 09:57:19.482383 11084 raft_consensus.cc:348] T 977a59175e5b43eeb6f23b938afd00a3 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.483171 11084 raft_consensus.cc:374] T 977a59175e5b43eeb6f23b938afd00a3 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:19.483489 11084 raft_consensus.cc:729] T 977a59175e5b43eeb6f23b938afd00a3 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:19.475593 11082 tablet_bootstrap.cc:654] T 689ab2b6cf1640129f6db8b113cf2fc6 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:19.465672 10687 raft_consensus.cc:2454] T 89a143a5623e4ef38027381a56c95227 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be97ab5d75934a648d4ab76341193ec0 in term 1.
I20260925 09:57:19.490520 10919 leader_election.cc:302] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:19.491228 11084 consensus_queue.cc:260] T 977a59175e5b43eeb6f23b938afd00a3 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.491976 11655 raft_consensus.cc:2781] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:19.492777 11655 raft_consensus.cc:686] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0 [term 1 LEADER]: Becoming Leader. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Running, Role: LEADER
I20260925 09:57:19.493628 11655 consensus_queue.cc:237] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.505296 11084 ts_tablet_manager.cc:1428] T 977a59175e5b43eeb6f23b938afd00a3 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.026s user 0.009s sys 0.000s
I20260925 09:57:19.455785 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet c16e096038084b2b9148390b6e773294 (table default.Table_70f0b47292de423097986b9a1c54f854 [id=8f2eaee8273640fb9ef73ccbe51782f1]) successfully deleted
W20260925 09:57:19.513886 10380 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 58117.6us
I20260925 09:57:19.517338 11084 tablet_bootstrap.cc:492] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:19.517781 10385 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 3d652a76d96f4dbeafde15f3c16534a6 (table default.Table_be0f658aa41f4188bc16db37e96f4ad1 [id=73083c1560cb4db5a12964387cb49b73]). Expected version 1 got 0
I20260925 09:57:19.519068 10385 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet b555eeba505740988c704df5c734e7c0 (table default.Table_48f368d811494119be0dc534e2fdcfbd [id=3e32026d175b4b92a3ee036f17e54b9d]). Expected version 1 got 0
I20260925 09:57:19.524286 11084 tablet_bootstrap.cc:654] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:19.534966 11616 raft_consensus.cc:504] T 42ec275c905343c4be1b72dc2bd0f6e3 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.536774 11616 leader_election.cc:288] T 42ec275c905343c4be1b72dc2bd0f6e3 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:19.542279 10385 catalog_manager.cc:5495] T ef6fefaa76d44fe798201aab2f21b9c7 P be97ab5d75934a648d4ab76341193ec0 reported cstate change: term changed from 0 to 1, leader changed from <none> to be97ab5d75934a648d4ab76341193ec0 (127.10.9.3). New cstate: current_term: 1 leader_uuid: "be97ab5d75934a648d4ab76341193ec0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: HEALTHY } } }
I20260925 09:57:19.548691 10983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42ec275c905343c4be1b72dc2bd0f6e3" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:19.549325 10983 raft_consensus.cc:3037] T 42ec275c905343c4be1b72dc2bd0f6e3 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:19.551008 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42ec275c905343c4be1b72dc2bd0f6e3" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:19.551613 10686 raft_consensus.cc:3037] T 42ec275c905343c4be1b72dc2bd0f6e3 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:19.554348 11084 tablet_bootstrap.cc:492] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:19.554728 11084 ts_tablet_manager.cc:1397] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.049s user 0.013s sys 0.003s
I20260925 09:57:19.557137 11084 raft_consensus.cc:348] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.557957 11084 raft_consensus.cc:374] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:19.558207 11084 raft_consensus.cc:729] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:19.559613 11565 raft_consensus.cc:482] T 9e697e7146e34841bd64660b45ebf231 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:19.560009 11565 raft_consensus.cc:504] T 9e697e7146e34841bd64660b45ebf231 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.561800 11565 leader_election.cc:288] T 9e697e7146e34841bd64660b45ebf231 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:19.563530 10682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e697e7146e34841bd64660b45ebf231" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:19.558869 11084 consensus_queue.cc:260] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:19.571018 10919 leader_election.cc:341] T 9e697e7146e34841bd64660b45ebf231 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:19.574167 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e697e7146e34841bd64660b45ebf231" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:19.581383 10918 leader_election.cc:341] T 9e697e7146e34841bd64660b45ebf231 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:19.581229 10686 raft_consensus.cc:2454] T 42ec275c905343c4be1b72dc2bd0f6e3 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:19.581840 10918 leader_election.cc:302] T 9e697e7146e34841bd64660b45ebf231 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:19.584264 10983 raft_consensus.cc:2454] T 42ec275c905343c4be1b72dc2bd0f6e3 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:19.586025 11082 tablet_bootstrap.cc:492] T 689ab2b6cf1640129f6db8b113cf2fc6 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:19.586414 11082 ts_tablet_manager.cc:1397] T 689ab2b6cf1640129f6db8b113cf2fc6 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.151s user 0.006s sys 0.011s
I20260925 09:57:19.588876 11082 raft_consensus.cc:348] T 689ab2b6cf1640129f6db8b113cf2fc6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.590888 11082 raft_consensus.cc:374] T 689ab2b6cf1640129f6db8b113cf2fc6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:19.591436 11082 raft_consensus.cc:729] T 689ab2b6cf1640129f6db8b113cf2fc6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:19.592111 10385 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet ef6fefaa76d44fe798201aab2f21b9c7 (table default.Table_46e4c497898b454c85b7c2a299ebf3d9 [id=80b48dd413e3446eb7ac65e5f957b5dc]). Expected version 2 got 0
I20260925 09:57:19.592789 10385 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 26fae3fbfe374f348a1729c21ec751b9 (table default.Table_0011387b45014de4b24b466f4cb907f7 [id=e3f7aa5e988c4b6c98be5cd3d465245d]). Expected version 1 got 0
I20260925 09:57:19.595079 11082 consensus_queue.cc:260] T 689ab2b6cf1640129f6db8b113cf2fc6 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.596983 11565 raft_consensus.cc:482] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:19.597419 11565 raft_consensus.cc:504] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.599265 11565 leader_election.cc:288] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:19.600606 11082 ts_tablet_manager.cc:1428] T 689ab2b6cf1640129f6db8b113cf2fc6 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.014s user 0.006s sys 0.002s
I20260925 09:57:19.600767 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ff4cf556eee494d8cd590884466f09f" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:19.601994 10919 leader_election.cc:341] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:19.602682 11082 tablet_bootstrap.cc:492] T 359090244c4748e8a2d45e3c77e89498 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:19.610805 11082 tablet_bootstrap.cc:654] T 359090244c4748e8a2d45e3c77e89498 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:19.589864 11655 raft_consensus.cc:2726] T 9e697e7146e34841bd64660b45ebf231 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:19.619210 11084 ts_tablet_manager.cc:1428] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.064s user 0.002s sys 0.005s
I20260925 09:57:19.624892 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ff4cf556eee494d8cd590884466f09f" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:19.626593 11082 tablet_bootstrap.cc:492] T 359090244c4748e8a2d45e3c77e89498 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:19.626754 11610 raft_consensus.cc:482] T 689ab2b6cf1640129f6db8b113cf2fc6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:19.627177 11610 raft_consensus.cc:504] T 689ab2b6cf1640129f6db8b113cf2fc6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.626963 11082 ts_tablet_manager.cc:1397] T 359090244c4748e8a2d45e3c77e89498 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.025s user 0.009s sys 0.000s
I20260925 09:57:19.629876 10775 leader_election.cc:302] T 42ec275c905343c4be1b72dc2bd0f6e3 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:19.635545 11084 tablet_bootstrap.cc:492] T 2358acdaa5104e3eacbab8bf182b8f30 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:19.634748 11082 raft_consensus.cc:348] T 359090244c4748e8a2d45e3c77e89498 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.636240 11082 raft_consensus.cc:374] T 359090244c4748e8a2d45e3c77e89498 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:19.636480 11082 raft_consensus.cc:729] T 359090244c4748e8a2d45e3c77e89498 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:19.637071 11082 consensus_queue.cc:260] T 359090244c4748e8a2d45e3c77e89498 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.638577 11603 ts_tablet_manager.cc:1918] T c07c6205d1284a6fbb369c010a226da7 P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:19.639803 11603 log.cc:1196] T c07c6205d1284a6fbb369c010a226da7 P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/c07c6205d1284a6fbb369c010a226da7
I20260925 09:57:19.640993 11603 ts_tablet_manager.cc:1939] T c07c6205d1284a6fbb369c010a226da7 P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:19.642344 11655 raft_consensus.cc:482] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:19.642875 11655 raft_consensus.cc:504] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.645525 11655 leader_election.cc:288] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:19.641795 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "689ab2b6cf1640129f6db8b113cf2fc6" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:19.645720 10688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "359090244c4748e8a2d45e3c77e89498" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:19.646451 10688 raft_consensus.cc:2454] T 359090244c4748e8a2d45e3c77e89498 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be97ab5d75934a648d4ab76341193ec0 in term 0.
I20260925 09:57:19.648448 10686 raft_consensus.cc:2454] T 689ab2b6cf1640129f6db8b113cf2fc6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
I20260925 09:57:19.647744 10919 leader_election.cc:302] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:19.649274 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet c07c6205d1284a6fbb369c010a226da7 (table default.Table_5a6ecccec7f44c31bbfe8d64580ec33b [id=2d6bc0e2aaee4b1483b01fc678505bb0]) successfully deleted
I20260925 09:57:19.649537 11565 raft_consensus.cc:2781] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:19.649842 11565 raft_consensus.cc:482] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:19.650148 11565 raft_consensus.cc:3037] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:19.645251 11082 ts_tablet_manager.cc:1428] T 359090244c4748e8a2d45e3c77e89498 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.017s user 0.004s sys 0.000s
I20260925 09:57:19.653414 11082 tablet_bootstrap.cc:492] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:19.654755 11605 raft_consensus.cc:482] T 3b9cdad4f9144daea800553abd378e0b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:19.655169 11605 raft_consensus.cc:504] T 3b9cdad4f9144daea800553abd378e0b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.657215 11605 leader_election.cc:288] T 3b9cdad4f9144daea800553abd378e0b P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:19.657589 11565 raft_consensus.cc:504] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.659655 11082 tablet_bootstrap.cc:654] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:19.629118 11610 leader_election.cc:288] T 689ab2b6cf1640129f6db8b113cf2fc6 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:19.662254 10775 leader_election.cc:302] T 689ab2b6cf1640129f6db8b113cf2fc6 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:19.648995 11616 raft_consensus.cc:2781] T 42ec275c905343c4be1b72dc2bd0f6e3 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:19.672840 11082 tablet_bootstrap.cc:492] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:19.674101 11082 ts_tablet_manager.cc:1397] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.021s user 0.005s sys 0.003s
I20260925 09:57:19.674762 11085 tablet_bootstrap.cc:654] T 89a143a5623e4ef38027381a56c95227 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:19.677052 11082 raft_consensus.cc:348] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.677767 11082 raft_consensus.cc:374] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:19.678015 11082 raft_consensus.cc:729] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:19.681778 11082 consensus_queue.cc:260] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.683792 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b9cdad4f9144daea800553abd378e0b" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:19.686755 11082 ts_tablet_manager.cc:1428] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.012s user 0.001s sys 0.005s
I20260925 09:57:19.687606 11082 tablet_bootstrap.cc:492] T 197f61525b0c4f9da441d98701d231a3 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
W20260925 09:57:19.685164 10919 leader_election.cc:341] T 3b9cdad4f9144daea800553abd378e0b P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:19.699823 11082 tablet_bootstrap.cc:654] T 197f61525b0c4f9da441d98701d231a3 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:19.704854 11084 tablet_bootstrap.cc:654] T 2358acdaa5104e3eacbab8bf182b8f30 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:19.705596 11565 leader_election.cc:288] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:19.706542 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "359090244c4748e8a2d45e3c77e89498" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:19.707427 10686 raft_consensus.cc:3037] T 359090244c4748e8a2d45e3c77e89498 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:19.709892 11082 tablet_bootstrap.cc:492] T 197f61525b0c4f9da441d98701d231a3 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:19.710253 11082 ts_tablet_manager.cc:1397] T 197f61525b0c4f9da441d98701d231a3 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.023s user 0.005s sys 0.003s
I20260925 09:57:19.712358 11082 raft_consensus.cc:348] T 197f61525b0c4f9da441d98701d231a3 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.712957 11082 raft_consensus.cc:374] T 197f61525b0c4f9da441d98701d231a3 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:19.713209 11082 raft_consensus.cc:729] T 197f61525b0c4f9da441d98701d231a3 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:19.713744 11082 consensus_queue.cc:260] T 197f61525b0c4f9da441d98701d231a3 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.720468 11670 raft_consensus.cc:2781] T 689ab2b6cf1640129f6db8b113cf2fc6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:19.720849 11670 raft_consensus.cc:482] T 689ab2b6cf1640129f6db8b113cf2fc6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:19.721153 11670 raft_consensus.cc:3037] T 689ab2b6cf1640129f6db8b113cf2fc6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:19.727933 11082 ts_tablet_manager.cc:1428] T 197f61525b0c4f9da441d98701d231a3 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.017s user 0.006s sys 0.000s
I20260925 09:57:19.728842 11082 tablet_bootstrap.cc:492] T cf35f39bcd6a41128acb2a4936fcc316 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:19.735509 11082 tablet_bootstrap.cc:654] T cf35f39bcd6a41128acb2a4936fcc316 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:19.743285 11655 consensus_queue.cc:1035] T ef6fefaa76d44fe798201aab2f21b9c7 P be97ab5d75934a648d4ab76341193ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:19.747493 11082 tablet_bootstrap.cc:492] T cf35f39bcd6a41128acb2a4936fcc316 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:19.747861 11082 ts_tablet_manager.cc:1397] T cf35f39bcd6a41128acb2a4936fcc316 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.000s
I20260925 09:57:19.755861 11565 raft_consensus.cc:482] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:19.750914 11082 raft_consensus.cc:348] T cf35f39bcd6a41128acb2a4936fcc316 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.759167 11082 raft_consensus.cc:374] T cf35f39bcd6a41128acb2a4936fcc316 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:19.759459 11082 raft_consensus.cc:729] T cf35f39bcd6a41128acb2a4936fcc316 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:19.761493 11082 consensus_queue.cc:260] T cf35f39bcd6a41128acb2a4936fcc316 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.763461 11082 ts_tablet_manager.cc:1428] T cf35f39bcd6a41128acb2a4936fcc316 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.015s user 0.007s sys 0.000s
I20260925 09:57:19.764432 11082 tablet_bootstrap.cc:492] T b555eeba505740988c704df5c734e7c0 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:19.767683 11565 raft_consensus.cc:504] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.773681 11084 tablet_bootstrap.cc:492] T 2358acdaa5104e3eacbab8bf182b8f30 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:19.774088 11084 ts_tablet_manager.cc:1397] T 2358acdaa5104e3eacbab8bf182b8f30 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.141s user 0.007s sys 0.002s
I20260925 09:57:19.776611 11082 tablet_bootstrap.cc:654] T b555eeba505740988c704df5c734e7c0 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:19.769776 11579 ts_tablet_manager.cc:1918] T 91c09118b43b41eca8f7040323ba76f7 P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:19.783628 11579 log.cc:1196] T 91c09118b43b41eca8f7040323ba76f7 P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/91c09118b43b41eca8f7040323ba76f7
I20260925 09:57:19.784202 11664 tablet.cc:1722] T 0a2bbe141e564a96be05de86f2993234 P c43461c4ee854ae1ac52b873ccada318: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 09:57:19.772428 11616 raft_consensus.cc:686] T 42ec275c905343c4be1b72dc2bd0f6e3 P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Becoming Leader. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Running, Role: LEADER
I20260925 09:57:19.777599 10686 raft_consensus.cc:2454] T 359090244c4748e8a2d45e3c77e89498 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be97ab5d75934a648d4ab76341193ec0 in term 1.
I20260925 09:57:19.794912 11565 leader_election.cc:288] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:19.791256 10688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "689ab2b6cf1640129f6db8b113cf2fc6" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:19.795816 10688 raft_consensus.cc:2454] T 689ab2b6cf1640129f6db8b113cf2fc6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be97ab5d75934a648d4ab76341193ec0 in term 0.
W20260925 09:57:19.799371 10772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 167088us
I20260925 09:57:19.800699 10919 leader_election.cc:302] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:19.804549 11655 raft_consensus.cc:2781] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:19.805136 11655 raft_consensus.cc:482] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:19.805677 11655 raft_consensus.cc:3037] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:19.807571 11085 tablet_bootstrap.cc:492] T 89a143a5623e4ef38027381a56c95227 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:19.807971 11085 ts_tablet_manager.cc:1397] T 89a143a5623e4ef38027381a56c95227 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.558s user 0.011s sys 0.004s
I20260925 09:57:19.802706 11579 ts_tablet_manager.cc:1939] T 91c09118b43b41eca8f7040323ba76f7 P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:19.817471 10919 leader_election.cc:302] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:19.818362 11605 raft_consensus.cc:2781] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:19.818830 11605 raft_consensus.cc:686] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [term 1 LEADER]: Becoming Leader. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Running, Role: LEADER
I20260925 09:57:19.819715 11605 consensus_queue.cc:237] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.829656 11085 raft_consensus.cc:348] T 89a143a5623e4ef38027381a56c95227 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.830253 11085 raft_consensus.cc:374] T 89a143a5623e4ef38027381a56c95227 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:19.830489 11085 raft_consensus.cc:729] T 89a143a5623e4ef38027381a56c95227 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:19.831068 11085 consensus_queue.cc:260] T 89a143a5623e4ef38027381a56c95227 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.832638 11085 ts_tablet_manager.cc:1428] T 89a143a5623e4ef38027381a56c95227 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.024s user 0.006s sys 0.000s
I20260925 09:57:19.835379 11085 tablet_bootstrap.cc:492] T 26fae3fbfe374f348a1729c21ec751b9 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:19.839756 11084 raft_consensus.cc:348] T 2358acdaa5104e3eacbab8bf182b8f30 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.840389 11084 raft_consensus.cc:374] T 2358acdaa5104e3eacbab8bf182b8f30 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:19.840615 11084 raft_consensus.cc:729] T 2358acdaa5104e3eacbab8bf182b8f30 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:19.843194 11084 consensus_queue.cc:260] T 2358acdaa5104e3eacbab8bf182b8f30 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.845058 11084 ts_tablet_manager.cc:1428] T 2358acdaa5104e3eacbab8bf182b8f30 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.071s user 0.003s sys 0.003s
W20260925 09:57:19.854913 10772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 49505.4us
W20260925 09:57:19.855705 10918 leader_election.cc:341] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:19.856053 10918 leader_election.cc:302] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:19.857817 10983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "689ab2b6cf1640129f6db8b113cf2fc6" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:19.861687 11634 raft_consensus.cc:482] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:19.862120 11634 raft_consensus.cc:504] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.864916 11565 raft_consensus.cc:2726] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:19.866137 11634 leader_election.cc:288] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:19.865917 11565 consensus_queue.cc:1035] T ef6fefaa76d44fe798201aab2f21b9c7 P be97ab5d75934a648d4ab76341193ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:19.867861 11684 raft_consensus.cc:482] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:19.868451 11684 raft_consensus.cc:504] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.872141 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26fae3fbfe374f348a1729c21ec751b9" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:19.872884 10686 raft_consensus.cc:2454] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be97ab5d75934a648d4ab76341193ec0 in term 0.
I20260925 09:57:19.874270 10919 leader_election.cc:302] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:19.874933 10981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26fae3fbfe374f348a1729c21ec751b9" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:19.875522 11605 raft_consensus.cc:2781] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:19.876029 10981 raft_consensus.cc:2454] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:19.877457 10619 leader_election.cc:302] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:19.878634 11634 raft_consensus.cc:2781] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:19.879026 11634 raft_consensus.cc:482] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:19.879379 11634 raft_consensus.cc:3037] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:19.883842 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "359090244c4748e8a2d45e3c77e89498" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:19.885010 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b9cdad4f9144daea800553abd378e0b" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:19.886149 10834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "359090244c4748e8a2d45e3c77e89498" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:19.883782 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26fae3fbfe374f348a1729c21ec751b9" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:19.888458 10621 leader_election.cc:341] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:19.889045 11634 raft_consensus.cc:504] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.892642 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26fae3fbfe374f348a1729c21ec751b9" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:19.893139 11605 raft_consensus.cc:482] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:19.893589 11605 raft_consensus.cc:3037] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:57:19.894227 10621 leader_election.cc:341] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:19.896428 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "689ab2b6cf1640129f6db8b113cf2fc6" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:19.897980 11684 leader_election.cc:288] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:19.902483 10980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26fae3fbfe374f348a1729c21ec751b9" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:19.905656 11684 raft_consensus.cc:482] T a44d9f2f7e3e48aeb0f31fa29ae1988a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:19.906167 11684 raft_consensus.cc:504] T a44d9f2f7e3e48aeb0f31fa29ae1988a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.793594 11616 consensus_queue.cc:237] T 42ec275c905343c4be1b72dc2bd0f6e3 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.909240 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a44d9f2f7e3e48aeb0f31fa29ae1988a" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:19.910714 10919 leader_election.cc:341] T a44d9f2f7e3e48aeb0f31fa29ae1988a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:19.918321 11579 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 149) took 1589 ms. Trace:
I20260925 09:57:19.918658 11579 rpcz_store.cc:276] 0925 09:57:18.328604 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:18.330097 (+ 1493us) service_pool.cc:227] Handling call
0925 09:57:19.020936 (+690839us) ts_tablet_manager.cc:1145] Deleting tablet 91c09118b43b41eca8f7040323ba76f7
0925 09:57:19.020976 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:19.578574 (+557598us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:19.755841 (+177267us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:19.918264 (+162423us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":690213}
I20260925 09:57:19.919906 11579 tablet_replica.cc:330] T 2d55154ceb064f22bd6f37c8ae5dce74 P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:19.920009 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 91c09118b43b41eca8f7040323ba76f7 (table default.Table_33b40a0d6ca44f15be47ca34bab6a093 [id=453beec106a64428b163c7db8132fc21]) successfully deleted
I20260925 09:57:19.920701 11579 raft_consensus.cc:2229] T 2d55154ceb064f22bd6f37c8ae5dce74 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:19.921247 11579 raft_consensus.cc:2258] T 2d55154ceb064f22bd6f37c8ae5dce74 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:19.922914 11579 ts_tablet_manager.cc:1905] T 2d55154ceb064f22bd6f37c8ae5dce74 P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:19.905915 11670 raft_consensus.cc:504] T 689ab2b6cf1640129f6db8b113cf2fc6 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.925441 10835 raft_consensus.cc:2379] T 689ab2b6cf1640129f6db8b113cf2fc6 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be97ab5d75934a648d4ab76341193ec0 in current term 1: Already voted for candidate c43461c4ee854ae1ac52b873ccada318 in this term.
I20260925 09:57:19.928854 11634 leader_election.cc:288] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:19.930505 11565 raft_consensus.cc:482] T 2358acdaa5104e3eacbab8bf182b8f30 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:19.930975 11565 raft_consensus.cc:504] T 2358acdaa5104e3eacbab8bf182b8f30 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.936352 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2358acdaa5104e3eacbab8bf182b8f30" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:19.936554 11684 leader_election.cc:288] T a44d9f2f7e3e48aeb0f31fa29ae1988a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
W20260925 09:57:19.938107 10919 leader_election.cc:341] T 2358acdaa5104e3eacbab8bf182b8f30 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:19.939093 11565 leader_election.cc:288] T 2358acdaa5104e3eacbab8bf182b8f30 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:19.940876 11085 tablet_bootstrap.cc:654] T 26fae3fbfe374f348a1729c21ec751b9 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:19.942904 11082 tablet_bootstrap.cc:492] T b555eeba505740988c704df5c734e7c0 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:19.943300 11082 ts_tablet_manager.cc:1397] T b555eeba505740988c704df5c734e7c0 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.179s user 0.034s sys 0.031s
W20260925 09:57:19.942936 10918 leader_election.cc:341] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:19.944336 10918 leader_election.cc:341] T 3b9cdad4f9144daea800553abd378e0b P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:19.945274 10918 leader_election.cc:302] T 3b9cdad4f9144daea800553abd378e0b P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:19.945484 11082 raft_consensus.cc:348] T b555eeba505740988c704df5c734e7c0 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.946120 11082 raft_consensus.cc:374] T b555eeba505740988c704df5c734e7c0 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:19.946375 11082 raft_consensus.cc:729] T b555eeba505740988c704df5c734e7c0 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:19.946843 11565 raft_consensus.cc:2726] T 3b9cdad4f9144daea800553abd378e0b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:19.845811 11084 tablet_bootstrap.cc:492] T fe91b39f91274a7ca61e9618d8cbb475 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:19.914906 11655 raft_consensus.cc:504] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.952370 11565 raft_consensus.cc:482] T 60fdc299865446c89d3e5374d573029a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:19.952865 11565 raft_consensus.cc:504] T 60fdc299865446c89d3e5374d573029a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.955430 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60fdc299865446c89d3e5374d573029a" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:19.956660 10919 leader_election.cc:341] T 60fdc299865446c89d3e5374d573029a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:19.957774 11565 leader_election.cc:288] T 60fdc299865446c89d3e5374d573029a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:19.959816 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "689ab2b6cf1640129f6db8b113cf2fc6" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:19.960517 10686 raft_consensus.cc:3037] T 689ab2b6cf1640129f6db8b113cf2fc6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:19.971680 10392 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 28d60531970a4ab58ce5ef99cb3a96bf (table default.Table_ce09d3a8d8e84973894f33b18bbfdb22 [id=6851500c7dba44f1bf353bb79bafd97a]). Expected version 2 got 0
I20260925 09:57:19.973266 10392 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet ef6fefaa76d44fe798201aab2f21b9c7 (table default.Table_46e4c497898b454c85b7c2a299ebf3d9 [id=80b48dd413e3446eb7ac65e5f957b5dc]). Expected version 2 got 0
I20260925 09:57:19.946921 11082 consensus_queue.cc:260] T b555eeba505740988c704df5c734e7c0 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.976246 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26fae3fbfe374f348a1729c21ec751b9" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:19.958341 10983 raft_consensus.cc:2379] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c43461c4ee854ae1ac52b873ccada318 in current term 1: Already voted for candidate be97ab5d75934a648d4ab76341193ec0 in this term.
I20260925 09:57:19.959079 11655 leader_election.cc:288] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:19.961701 11605 raft_consensus.cc:504] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:19.985311 10688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "689ab2b6cf1640129f6db8b113cf2fc6" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:19.986927 11605 leader_election.cc:288] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:19.987926 10688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26fae3fbfe374f348a1729c21ec751b9" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:19.988166 11082 ts_tablet_manager.cc:1428] T b555eeba505740988c704df5c734e7c0 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.045s user 0.005s sys 0.000s
I20260925 09:57:19.988657 10688 raft_consensus.cc:2379] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate be97ab5d75934a648d4ab76341193ec0 in current term 1: Already voted for candidate b8850818e2ee42a082802d0095c08d21 in this term.
I20260925 09:57:19.993276 11670 leader_election.cc:288] T 689ab2b6cf1640129f6db8b113cf2fc6 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:19.994145 10980 raft_consensus.cc:2379] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b8850818e2ee42a082802d0095c08d21 in current term 1: Already voted for candidate be97ab5d75934a648d4ab76341193ec0 in this term.
I20260925 09:57:19.997902 10983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "689ab2b6cf1640129f6db8b113cf2fc6" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:19.999966 10619 leader_election.cc:302] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8850818e2ee42a082802d0095c08d21; no voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:20.001991 11627 raft_consensus.cc:2726] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260925 09:57:20.008632 10686 raft_consensus.cc:2454] T 689ab2b6cf1640129f6db8b113cf2fc6 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be97ab5d75934a648d4ab76341193ec0 in term 1.
I20260925 09:57:19.998988 10983 raft_consensus.cc:2379] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c43461c4ee854ae1ac52b873ccada318 in current term 1: Already voted for candidate be97ab5d75934a648d4ab76341193ec0 in this term.
I20260925 09:57:20.016830 10772 leader_election.cc:302] T 689ab2b6cf1640129f6db8b113cf2fc6 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:20.017870 11616 raft_consensus.cc:2726] T 689ab2b6cf1640129f6db8b113cf2fc6 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260925 09:57:20.012048 10919 leader_election.cc:302] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:20.022994 11655 raft_consensus.cc:2781] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:20.023483 11655 raft_consensus.cc:686] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0 [term 1 LEADER]: Becoming Leader. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Running, Role: LEADER
I20260925 09:57:20.024210 11655 consensus_queue.cc:237] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.031028 10658 tablet_service.cc:1467] Processing CreateTablet for tablet 545ac025f2cb469da847979011a0b3ce (DEFAULT_TABLE table=default.Table_14d12f6871c847daa860c3a0cfee4231 [id=f61297f0d10c4656bd06a752c9c7596b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:20.031342 10810 tablet_service.cc:1467] Processing CreateTablet for tablet 545ac025f2cb469da847979011a0b3ce (DEFAULT_TABLE table=default.Table_14d12f6871c847daa860c3a0cfee4231 [id=f61297f0d10c4656bd06a752c9c7596b]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 09:57:19.981774 10918 leader_election.cc:341] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:20.032398 10810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 545ac025f2cb469da847979011a0b3ce. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:20.032691 10918 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 50962.9us
W20260925 09:57:20.033672 10918 leader_election.cc:341] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:20.034911 10386 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 0a2bbe141e564a96be05de86f2993234 (table default.Table_988762e7d6264b1cb64927ff31711470 [id=aed8a2ddc51545ad9cec4d272dcf3daf]). Expected version 2 got 1
I20260925 09:57:20.037201 11082 tablet_bootstrap.cc:492] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:20.037794 10961 tablet_service.cc:1467] Processing CreateTablet for tablet 545ac025f2cb469da847979011a0b3ce (DEFAULT_TABLE table=default.Table_14d12f6871c847daa860c3a0cfee4231 [id=f61297f0d10c4656bd06a752c9c7596b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:20.038651 10961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 545ac025f2cb469da847979011a0b3ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:20.007778 11605 raft_consensus.cc:482] T 55d14d4d8abe4ec9ac9aeef0d027e804 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:20.040041 10658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 545ac025f2cb469da847979011a0b3ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:20.039705 11605 raft_consensus.cc:504] T 55d14d4d8abe4ec9ac9aeef0d027e804 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.045542 11082 tablet_bootstrap.cc:654] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:20.048800 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2358acdaa5104e3eacbab8bf182b8f30" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:20.053308 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60fdc299865446c89d3e5374d573029a" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:20.055473 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "689ab2b6cf1640129f6db8b113cf2fc6" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:20.056123 10832 raft_consensus.cc:2379] T 689ab2b6cf1640129f6db8b113cf2fc6 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate be97ab5d75934a648d4ab76341193ec0 in current term 1: Already voted for candidate c43461c4ee854ae1ac52b873ccada318 in this term.
I20260925 09:57:20.057718 11627 raft_consensus.cc:482] T cf35f39bcd6a41128acb2a4936fcc316 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:20.057813 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26fae3fbfe374f348a1729c21ec751b9" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:20.058266 11085 tablet_bootstrap.cc:492] T 26fae3fbfe374f348a1729c21ec751b9 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:20.058184 11627 raft_consensus.cc:504] T cf35f39bcd6a41128acb2a4936fcc316 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:20.059281 10918 leader_election.cc:341] T 2358acdaa5104e3eacbab8bf182b8f30 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:20.059684 10918 leader_election.cc:302] T 2358acdaa5104e3eacbab8bf182b8f30 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:20.060359 11627 leader_election.cc:288] T cf35f39bcd6a41128acb2a4936fcc316 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:20.060359 11565 raft_consensus.cc:482] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:20.060765 11565 raft_consensus.cc:504] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.063972 11684 raft_consensus.cc:2726] T 2358acdaa5104e3eacbab8bf182b8f30 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:20.058682 11085 ts_tablet_manager.cc:1397] T 26fae3fbfe374f348a1729c21ec751b9 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.225s user 0.009s sys 0.003s
I20260925 09:57:20.065784 11565 leader_election.cc:288] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:20.066459 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b555eeba505740988c704df5c734e7c0" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:20.067157 10686 raft_consensus.cc:2454] T b555eeba505740988c704df5c734e7c0 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be97ab5d75934a648d4ab76341193ec0 in term 0.
I20260925 09:57:20.067999 11085 raft_consensus.cc:348] T 26fae3fbfe374f348a1729c21ec751b9 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.068384 10919 leader_election.cc:302] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:20.068706 11085 raft_consensus.cc:374] T 26fae3fbfe374f348a1729c21ec751b9 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:19.996994 11084 tablet_bootstrap.cc:654] T fe91b39f91274a7ca61e9618d8cbb475 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:20.069219 11565 raft_consensus.cc:2781] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:20.070451 11565 raft_consensus.cc:482] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:20.070808 11565 raft_consensus.cc:3037] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:20.070736 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a44d9f2f7e3e48aeb0f31fa29ae1988a" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:20.070923 11605 leader_election.cc:288] T 55d14d4d8abe4ec9ac9aeef0d027e804 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:20.071704 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55d14d4d8abe4ec9ac9aeef0d027e804" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:20.072162 10983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf35f39bcd6a41128acb2a4936fcc316" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:20.072861 10983 raft_consensus.cc:2454] T cf35f39bcd6a41128acb2a4936fcc316 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
W20260925 09:57:20.073086 10919 leader_election.cc:341] T 55d14d4d8abe4ec9ac9aeef0d027e804 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:20.073915 10619 leader_election.cc:302] T cf35f39bcd6a41128acb2a4936fcc316 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:20.074786 11634 raft_consensus.cc:2781] T cf35f39bcd6a41128acb2a4936fcc316 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:20.075100 11634 raft_consensus.cc:482] T cf35f39bcd6a41128acb2a4936fcc316 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260925 09:57:20.075341 10918 leader_election.cc:341] T 60fdc299865446c89d3e5374d573029a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:20.075490 11634 raft_consensus.cc:3037] T cf35f39bcd6a41128acb2a4936fcc316 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:20.075693 10918 leader_election.cc:302] T 60fdc299865446c89d3e5374d573029a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:20.078943 11684 raft_consensus.cc:2726] T 60fdc299865446c89d3e5374d573029a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:20.068974 11085 raft_consensus.cc:729] T 26fae3fbfe374f348a1729c21ec751b9 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:20.083976 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf35f39bcd6a41128acb2a4936fcc316" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:20.084118 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b555eeba505740988c704df5c734e7c0" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:20.086436 10621 leader_election.cc:341] T cf35f39bcd6a41128acb2a4936fcc316 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:20.088681 10918 leader_election.cc:341] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:20.089033 10918 leader_election.cc:302] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:20.090384 11579 ts_tablet_manager.cc:1918] T 2d55154ceb064f22bd6f37c8ae5dce74 P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:20.090749 11579 log.cc:1196] T 2d55154ceb064f22bd6f37c8ae5dce74 P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/2d55154ceb064f22bd6f37c8ae5dce74
I20260925 09:57:20.091153 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55d14d4d8abe4ec9ac9aeef0d027e804" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:20.094077 11565 raft_consensus.cc:504] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.096997 11084 tablet_bootstrap.cc:492] T fe91b39f91274a7ca61e9618d8cbb475 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:20.097399 11084 ts_tablet_manager.cc:1397] T fe91b39f91274a7ca61e9618d8cbb475 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.252s user 0.007s sys 0.002s
I20260925 09:57:20.097801 11684 raft_consensus.cc:2726] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260925 09:57:20.099766 10918 leader_election.cc:341] T a44d9f2f7e3e48aeb0f31fa29ae1988a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:20.100951 11634 raft_consensus.cc:504] T cf35f39bcd6a41128acb2a4936fcc316 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.105296 11634 leader_election.cc:288] T cf35f39bcd6a41128acb2a4936fcc316 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:20.107586 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b555eeba505740988c704df5c734e7c0" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:20.108162 10686 raft_consensus.cc:3037] T b555eeba505740988c704df5c734e7c0 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:20.109453 10918 leader_election.cc:302] T a44d9f2f7e3e48aeb0f31fa29ae1988a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:20.104612 10983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf35f39bcd6a41128acb2a4936fcc316" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:20.111074 10983 raft_consensus.cc:3037] T cf35f39bcd6a41128acb2a4936fcc316 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:57:20.111459 10918 leader_election.cc:341] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:20.112653 11655 raft_consensus.cc:2726] T a44d9f2f7e3e48aeb0f31fa29ae1988a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:20.112987 11565 leader_election.cc:288] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:20.091359 11579 ts_tablet_manager.cc:1939] T 2d55154ceb064f22bd6f37c8ae5dce74 P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:20.099617 11084 raft_consensus.cc:348] T fe91b39f91274a7ca61e9618d8cbb475 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.103345 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf35f39bcd6a41128acb2a4936fcc316" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:20.083971 11085 consensus_queue.cc:260] T 26fae3fbfe374f348a1729c21ec751b9 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.115669 11084 raft_consensus.cc:374] T fe91b39f91274a7ca61e9618d8cbb475 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:20.116047 11084 raft_consensus.cc:729] T fe91b39f91274a7ca61e9618d8cbb475 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
W20260925 09:57:20.117024 10621 leader_election.cc:341] T cf35f39bcd6a41128acb2a4936fcc316 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:20.119382 10686 raft_consensus.cc:2454] T b555eeba505740988c704df5c734e7c0 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be97ab5d75934a648d4ab76341193ec0 in term 1.
I20260925 09:57:20.119567 11084 consensus_queue.cc:260] T fe91b39f91274a7ca61e9618d8cbb475 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.121661 11084 ts_tablet_manager.cc:1428] T fe91b39f91274a7ca61e9618d8cbb475 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.024s user 0.007s sys 0.000s
I20260925 09:57:20.122058 11085 ts_tablet_manager.cc:1428] T 26fae3fbfe374f348a1729c21ec751b9 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.056s user 0.005s sys 0.001s
I20260925 09:57:20.122473 11084 tablet_bootstrap.cc:492] T 6161774f61ae48ab9764a05f2188a80b P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:20.122972 11085 tablet_bootstrap.cc:492] T 359090244c4748e8a2d45e3c77e89498 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:20.121611 11579 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 148) took 1793 ms. Trace:
I20260925 09:57:20.125057 11579 rpcz_store.cc:276] 0925 09:57:18.328381 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:18.330801 (+ 2420us) service_pool.cc:227] Handling call
0925 09:57:19.919758 (+1588957us) ts_tablet_manager.cc:1145] Deleting tablet 2d55154ceb064f22bd6f37c8ae5dce74
0925 09:57:19.919794 (+ 36us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:20.048852 (+129058us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:20.090144 (+ 41292us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:20.121554 (+ 31410us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":8546,"tablet-delete.queue_time_us":1588409}
I20260925 09:57:20.127359 10666 tablet_service.cc:1467] Processing CreateTablet for tablet 69a96874eea1409a9a5a06b5787fbd99 (DEFAULT_TABLE table=default.Table_da740072c87649f9855a1d7c5c277619 [id=1a66bc26bda548bbbccb11c77d222e3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:20.127723 10810 tablet_service.cc:1467] Processing CreateTablet for tablet 69a96874eea1409a9a5a06b5787fbd99 (DEFAULT_TABLE table=default.Table_da740072c87649f9855a1d7c5c277619 [id=1a66bc26bda548bbbccb11c77d222e3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:20.128335 10666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69a96874eea1409a9a5a06b5787fbd99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:20.128872 10810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69a96874eea1409a9a5a06b5787fbd99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:20.130764 10807 tablet_service.cc:1467] Processing CreateTablet for tablet d5956be0da6441c79723e72f12dfe54a (DEFAULT_TABLE table=default.Table_70f0b47292de423097986b9a1c54f854 [id=8f2eaee8273640fb9ef73ccbe51782f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:20.131577 10386 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 28d60531970a4ab58ce5ef99cb3a96bf (table default.Table_ce09d3a8d8e84973894f33b18bbfdb22 [id=6851500c7dba44f1bf353bb79bafd97a]). Expected version 2 got 0
I20260925 09:57:20.131788 10807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5956be0da6441c79723e72f12dfe54a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:20.131666 10919 leader_election.cc:302] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:20.134333 10959 tablet_service.cc:1467] Processing CreateTablet for tablet 69a96874eea1409a9a5a06b5787fbd99 (DEFAULT_TABLE table=default.Table_da740072c87649f9855a1d7c5c277619 [id=1a66bc26bda548bbbccb11c77d222e3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:20.134557 10960 tablet_service.cc:1467] Processing CreateTablet for tablet d5956be0da6441c79723e72f12dfe54a (DEFAULT_TABLE table=default.Table_70f0b47292de423097986b9a1c54f854 [id=8f2eaee8273640fb9ef73ccbe51782f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:20.135928 10664 tablet_service.cc:1467] Processing CreateTablet for tablet d5956be0da6441c79723e72f12dfe54a (DEFAULT_TABLE table=default.Table_70f0b47292de423097986b9a1c54f854 [id=8f2eaee8273640fb9ef73ccbe51782f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:20.136839 10664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5956be0da6441c79723e72f12dfe54a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:20.151527 11565 raft_consensus.cc:2781] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:20.152654 11565 raft_consensus.cc:686] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0 [term 1 LEADER]: Becoming Leader. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Running, Role: LEADER
I20260925 09:57:20.152886 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 2d55154ceb064f22bd6f37c8ae5dce74 (table default.Table_7eaf247fb74f497d80a88b67fa301cab [id=efe6d05ed9084a1c998e95e99672d4fc]) successfully deleted
I20260925 09:57:20.155884 11579 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 146) took 1828 ms. Trace:
I20260925 09:57:20.156219 11579 rpcz_store.cc:276] 0925 09:57:18.327896 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:18.334363 (+ 6467us) service_pool.cc:227] Handling call
0925 09:57:20.155612 (+1821249us) ts_tablet_manager.cc:1145] Deleting tablet fd73d0d52350444499abae34e8c02fab
0925 09:57:20.155660 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:20.155826 (+ 166us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1797794}
W20260925 09:57:20.157248 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet fd73d0d52350444499abae34e8c02fab because the tablet was not found. No further retry: Not found: Tablet not found: fd73d0d52350444499abae34e8c02fab
I20260925 09:57:20.156347 11565 consensus_queue.cc:237] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.161341 11085 tablet_bootstrap.cc:654] T 359090244c4748e8a2d45e3c77e89498 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:20.162461 10918 leader_election.cc:341] T 55d14d4d8abe4ec9ac9aeef0d027e804 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:20.162845 10918 leader_election.cc:302] T 55d14d4d8abe4ec9ac9aeef0d027e804 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:20.163700 11655 raft_consensus.cc:2726] T 55d14d4d8abe4ec9ac9aeef0d027e804 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:20.169571 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b555eeba505740988c704df5c734e7c0" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:20.171273 11670 consensus_queue.cc:1035] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:20.175661 10960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5956be0da6441c79723e72f12dfe54a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:20.178929 10959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69a96874eea1409a9a5a06b5787fbd99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:20.187606 11579 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 145) took 1860 ms. Trace:
I20260925 09:57:20.188014 11579 rpcz_store.cc:276] 0925 09:57:18.327630 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:18.334617 (+ 6987us) service_pool.cc:227] Handling call
0925 09:57:20.162759 (+1828142us) ts_tablet_manager.cc:1145] Deleting tablet a0976e0c01234b38a7ab934dc1b289c4
0925 09:57:20.162817 (+ 58us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:20.187544 (+ 24727us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1806948}
I20260925 09:57:20.188843 10386 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet ef6fefaa76d44fe798201aab2f21b9c7 (table default.Table_46e4c497898b454c85b7c2a299ebf3d9 [id=80b48dd413e3446eb7ac65e5f957b5dc]). Expected version 2 got 0
I20260925 09:57:20.194732 11579 tablet_replica.cc:330] T 4c4e8790282a49fd849a34ca11bef5bf P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
W20260925 09:57:20.197597 10918 leader_election.cc:341] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:20.198944 10659 tablet_service.cc:1467] Processing CreateTablet for tablet 7a31b999f0fd447eacff13de71aa3fff (DEFAULT_TABLE table=default.Table_489e2c74ed6249ce9cf54541e0a491cd [id=8d2c4997954141129ebbec9e72658d90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:20.200070 10659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a31b999f0fd447eacff13de71aa3fff. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:20.189136 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet a0976e0c01234b38a7ab934dc1b289c4 because the tablet was not found. No further retry: Not found: Tablet not found: a0976e0c01234b38a7ab934dc1b289c4
I20260925 09:57:20.207072 11579 raft_consensus.cc:2229] T 4c4e8790282a49fd849a34ca11bef5bf P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:20.208724 11579 raft_consensus.cc:2258] T 4c4e8790282a49fd849a34ca11bef5bf P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:20.211365 10387 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:35140 (request call id 179) took 1591 ms. Trace:
W20260925 09:57:20.212126 10391 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4e3120347c8e4a3eab57d6562ae40e8d consensus state has no leader
I20260925 09:57:20.212651 10391 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:35140 (request call id 178) took 1920 ms. Trace:
I20260925 09:57:20.213063 10391 rpcz_store.cc:276] 0925 09:57:18.292088 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:18.667863 (+375775us) service_pool.cc:227] Handling call
0925 09:57:18.669196 (+ 1333us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:57:18.677054 (+ 7858us) catalog_manager.cc:3616] Apply alter schema
0925 09:57:18.677574 (+ 520us) catalog_manager.cc:3639] Acquired catalog manager lock
0925 09:57:18.677779 (+ 205us) catalog_manager.cc:3786] Updating metadata on disk
0925 09:57:18.916077 (+238298us) catalog_manager.cc:3817] Committing alterations to in-memory state
0925 09:57:20.212599 (+1296522us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:18.710173 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:18.710516 (+ 343us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:18.710538 (+ 22us) write_op.cc:435] Acquired schema lock
0925 09:57:18.710551 (+ 13us) tablet.cc:660] Decoding operations
0925 09:57:18.710828 (+ 277us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:18.710874 (+ 46us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:18.710908 (+ 34us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:18.711092 (+ 184us) tablet.cc:699] Row locks acquired
0925 09:57:18.711106 (+ 14us) write_op.cc:260] PREPARE: finished
0925 09:57:18.711223 (+ 117us) write_op.cc:270] Start()
0925 09:57:18.711320 (+ 97us) write_op.cc:276] Timestamp: P: 1790330238711178 usec, L: 0
0925 09:57:18.713051 (+ 1731us) op_driver.cc:331] REPLICATION: starting
0925 09:57:18.713506 (+ 455us) log.cc:842] Serialized 405 byte log entry
0925 09:57:18.727377 (+ 13871us) op_driver.cc:444] REPLICATION: finished
0925 09:57:18.731636 (+ 4259us) write_op.cc:301] APPLY: starting
0925 09:57:18.731681 (+ 45us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:18.731766 (+ 85us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:18.731846 (+ 80us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:18.732011 (+ 165us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:18.732135 (+ 124us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:18.732198 (+ 63us) write_op.cc:312] APPLY: finished
0925 09:57:18.732355 (+ 157us) log.cc:842] Serialized 23 byte log entry
0925 09:57:18.732479 (+ 124us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:18.732558 (+ 79us) write_op.cc:454] Released schema lock
0925 09:57:18.732589 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:18.732848 (+ 259us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:18.732874 (+ 26us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":4116,"apply.run_cpu_time_us":1385,"apply.run_wall_time_us":1384,"mutex_wait_us":24101,"num_ops":1,"prepare.queue_time_us":31367,"prepare.run_cpu_time_us":1911,"prepare.run_wall_time_us":5659,"raft.queue_time_us":66,"raft.run_cpu_time_us":17,"raft.run_wall_time_us":17,"replication_time_us":14259}]]}
W20260925 09:57:20.222782 10390 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.2:40761 (request call id 25) took 2255 ms (client timeout 1000 ms). Trace:
I20260925 09:57:20.226096 11579 ts_tablet_manager.cc:1905] T 4c4e8790282a49fd849a34ca11bef5bf P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:20.211702 10387 rpcz_store.cc:276] 0925 09:57:18.620397 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:18.699515 (+ 79118us) service_pool.cc:227] Handling call
0925 09:57:18.860374 (+160859us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2adb7d6 00007fabf0621634 00007fabf06215c2 00007fabf06214f1 00007fabf062146c 00007fabf0620722 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:18.862773 (+ 2399us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:57:18.863641 (+ 868us) catalog_manager.cc:3616] Apply alter schema
0925 09:57:18.867450 (+ 3809us) catalog_manager.cc:3639] Acquired catalog manager lock
0925 09:57:18.867624 (+ 174us) catalog_manager.cc:3786] Updating metadata on disk
0925 09:57:19.128970 (+261346us) catalog_manager.cc:3817] Committing alterations to in-memory state
0925 09:57:20.211303 (+1082333us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:18.876062 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:18.876405 (+ 343us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:18.876426 (+ 21us) write_op.cc:435] Acquired schema lock
0925 09:57:18.876438 (+ 12us) tablet.cc:660] Decoding operations
0925 09:57:18.876663 (+ 225us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:18.876696 (+ 33us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:18.876716 (+ 20us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:18.876840 (+ 124us) tablet.cc:699] Row locks acquired
0925 09:57:18.876853 (+ 13us) write_op.cc:260] PREPARE: finished
0925 09:57:18.876946 (+ 93us) write_op.cc:270] Start()
0925 09:57:18.877014 (+ 68us) write_op.cc:276] Timestamp: P: 1790330238876891 usec, L: 0
0925 09:57:18.877029 (+ 15us) op_driver.cc:331] REPLICATION: starting
0925 09:57:18.877464 (+ 435us) log.cc:842] Serialized 405 byte log entry
0925 09:57:19.044952 (+167488us) op_driver.cc:444] REPLICATION: finished
0925 09:57:19.045886 (+ 934us) write_op.cc:301] APPLY: starting
0925 09:57:19.045944 (+ 58us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:19.070627 (+ 24683us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:19.070779 (+ 152us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:19.071047 (+ 268us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:19.071193 (+ 146us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:19.071240 (+ 47us) write_op.cc:312] APPLY: finished
0925 09:57:19.104360 (+ 33120us) log.cc:842] Serialized 23 byte log entry
0925 09:57:19.128253 (+ 23893us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:19.128328 (+ 75us) write_op.cc:454] Released schema lock
0925 09:57:19.128368 (+ 40us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:19.129285 (+ 917us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:19.129318 (+ 33us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":341167360,"child_traces":[["op",{"apply.queue_time_us":754,"apply.run_cpu_time_us":1820,"apply.run_wall_time_us":83617,"num_ops":1,"prepare.queue_time_us":81,"prepare.run_cpu_time_us":1581,"prepare.run_wall_time_us":2590,"replication_time_us":167839}]]}
I20260925 09:57:20.229602 10387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_e613b02df28b46f2b371344af4971233"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:20.231235 10387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e613b02df28b46f2b371344af4971233 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:20.224277 10810 tablet_service.cc:1467] Processing CreateTablet for tablet 7a31b999f0fd447eacff13de71aa3fff (DEFAULT_TABLE table=default.Table_489e2c74ed6249ce9cf54541e0a491cd [id=8d2c4997954141129ebbec9e72658d90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:20.235018 10810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a31b999f0fd447eacff13de71aa3fff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:20.242108 11674 raft_consensus.cc:482] T 26fae3fbfe374f348a1729c21ec751b9 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:20.242527 11674 raft_consensus.cc:504] T 26fae3fbfe374f348a1729c21ec751b9 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.244647 11674 leader_election.cc:288] T 26fae3fbfe374f348a1729c21ec751b9 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:20.247591 10805 tablet_service.cc:1514] Processing DeleteTablet for tablet 1969ac471b6f471ebe8e048a96c22837 with delete_type TABLET_DATA_DELETED (replaced by tablet 977a59175e5b43eeb6f23b938afd00a3) from {username='slave'} at 127.0.0.1:33344
W20260925 09:57:20.223081 10390 rpcz_store.cc:269] 0925 09:57:17.967436 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:18.086947 (+119511us) service_pool.cc:227] Handling call
0925 09:57:18.745744 (+658797us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:18.745779 (+ 35us) write_op.cc:454] Released schema lock
0925 09:57:20.222720 (+1476941us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:18.595587 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:18.595970 (+ 383us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:18.595992 (+ 22us) write_op.cc:435] Acquired schema lock
0925 09:57:18.596003 (+ 11us) tablet.cc:660] Decoding operations
0925 09:57:18.596418 (+ 415us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:18.596458 (+ 40us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:18.596480 (+ 22us) tablet.cc:683] Acquiring locks for 3 operations
0925 09:57:18.597493 (+ 1013us) tablet.cc:699] Row locks acquired
0925 09:57:18.597509 (+ 16us) write_op.cc:260] PREPARE: finished
0925 09:57:18.597621 (+ 112us) write_op.cc:270] Start()
0925 09:57:18.597703 (+ 82us) write_op.cc:276] Timestamp: P: 1790330238597542 usec, L: 0
0925 09:57:18.597720 (+ 17us) op_driver.cc:331] REPLICATION: starting
0925 09:57:18.598257 (+ 537us) log.cc:842] Serialized 1266 byte log entry
0925 09:57:18.613499 (+ 15242us) op_driver.cc:444] REPLICATION: finished
0925 09:57:18.614268 (+ 769us) write_op.cc:301] APPLY: starting
0925 09:57:18.614322 (+ 54us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:18.614440 (+ 118us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:18.614544 (+ 104us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:18.614881 (+ 337us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:18.615022 (+ 141us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0925 09:57:18.615073 (+ 51us) write_op.cc:312] APPLY: finished
0925 09:57:18.615259 (+ 186us) log.cc:842] Serialized 35 byte log entry
0925 09:57:18.615410 (+ 151us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:18.615482 (+ 72us) write_op.cc:454] Released schema lock
0925 09:57:18.615511 (+ 29us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":157981,"reported_tablets":25,"child_traces":[["op",{"apply.queue_time_us":612,"apply.run_cpu_time_us":1443,"apply.run_wall_time_us":1444,"num_ops":3,"prepare.queue_time_us":10552,"prepare.run_cpu_time_us":2259,"prepare.run_wall_time_us":5396,"raft.queue_time_us":148,"raft.run_cpu_time_us":707,"raft.run_wall_time_us":714,"replication_time_us":15711}]]}
I20260925 09:57:20.250726 10688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26fae3fbfe374f348a1729c21ec751b9" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:20.251633 10688 raft_consensus.cc:2379] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c43461c4ee854ae1ac52b873ccada318 in current term 1: Already voted for candidate b8850818e2ee42a082802d0095c08d21 in this term.
I20260925 09:57:20.259663 10805 tablet_service.cc:1514] Processing DeleteTablet for tablet 833fac37485141f6a18104eaea907b0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:14 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:20.260653 10805 tablet_service.cc:1514] Processing DeleteTablet for tablet 91c09118b43b41eca8f7040323ba76f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:15 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:20.261895 10805 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d55154ceb064f22bd6f37c8ae5dce74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:12 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:20.262825 10805 tablet_service.cc:1514] Processing DeleteTablet for tablet 41df75fe4b8e4856bdbb5ead3cf4fd24 with delete_type TABLET_DATA_DELETED (replaced by tablet c7cd1d6a603642afa2c8c737079961b5) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:20.263501 11082 tablet_bootstrap.cc:492] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:20.263926 11082 ts_tablet_manager.cc:1397] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.227s user 0.016s sys 0.003s
I20260925 09:57:20.266048 11082 raft_consensus.cc:348] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:20.273986 10621 consensus_peers.cc:487] T 2d55154ceb064f22bd6f37c8ae5dce74 P b8850818e2ee42a082802d0095c08d21 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2d55154ceb064f22bd6f37c8ae5dce74. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:20.274973 11082 raft_consensus.cc:374] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:20.275344 11082 raft_consensus.cc:729] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
W20260925 09:57:20.255983 10380 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 4e3120347c8e4a3eab57d6562ae40e8d (table default.Table_8e186a99ccba425b9b588041d2364a40 [id=2d6bc0e2aaee4b1483b01fc678505bb0], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 4e3120347c8e4a3eab57d6562ae40e8d consensus state has no leader
I20260925 09:57:20.278087 10390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_8b45818055ec40e386d43e2641ec6ed0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:20.280372 10390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8b45818055ec40e386d43e2641ec6ed0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:20.284125 11082 consensus_queue.cc:260] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.284543 10807 tablet_service.cc:1514] Processing DeleteTablet for tablet 752105c1514f4c1db2a11b95b413b4f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:14 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:20.289848 11082 ts_tablet_manager.cc:1428] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.026s user 0.004s sys 0.000s
I20260925 09:57:20.298121 11082 tablet_bootstrap.cc:492] T c588b0c05a1a4f3ea545066183e76502 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:20.296200 11701 tablet.cc:1722] T 28d60531970a4ab58ce5ef99cb3a96bf P b8850818e2ee42a082802d0095c08d21: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 09:57:20.303339 11082 tablet_bootstrap.cc:654] T c588b0c05a1a4f3ea545066183e76502 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:20.315521 11579 ts_tablet_manager.cc:1918] T 4c4e8790282a49fd849a34ca11bef5bf P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:20.315898 11579 log.cc:1196] T 4c4e8790282a49fd849a34ca11bef5bf P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/4c4e8790282a49fd849a34ca11bef5bf
I20260925 09:57:20.316604 11579 ts_tablet_manager.cc:1939] T 4c4e8790282a49fd849a34ca11bef5bf P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:20.316762 10387 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35140 (request call id 185) took 1202 ms. Trace:
I20260925 09:57:20.328357 11664 tablet.cc:1722] T 0a2bbe141e564a96be05de86f2993234 P c43461c4ee854ae1ac52b873ccada318: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260925 09:57:20.323545 10387 rpcz_store.cc:276] 0925 09:57:19.113926 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:20.229223 (+1115297us) service_pool.cc:227] Handling call
0925 09:57:20.232104 (+ 2881us) catalog_manager.cc:2075] Acquired catalog manager lock
0925 09:57:20.281876 (+ 49772us) catalog_manager.cc:2121] Created new table and tablet info
0925 09:57:20.316060 (+ 34184us) catalog_manager.cc:2181] Wrote table and tablets to system table
0925 09:57:20.316398 (+ 338us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0925 09:57:20.316698 (+ 300us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:20.286070 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:20.286390 (+ 320us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:20.286412 (+ 22us) write_op.cc:435] Acquired schema lock
0925 09:57:20.286424 (+ 12us) tablet.cc:660] Decoding operations
0925 09:57:20.286827 (+ 403us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:20.286866 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:20.286886 (+ 20us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:20.287069 (+ 183us) tablet.cc:699] Row locks acquired
0925 09:57:20.287082 (+ 13us) write_op.cc:260] PREPARE: finished
0925 09:57:20.287179 (+ 97us) write_op.cc:270] Start()
0925 09:57:20.287254 (+ 75us) write_op.cc:276] Timestamp: P: 1790330240287127 usec, L: 0
0925 09:57:20.287267 (+ 13us) op_driver.cc:331] REPLICATION: starting
0925 09:57:20.287865 (+ 598us) log.cc:842] Serialized 433 byte log entry
0925 09:57:20.298263 (+ 10398us) op_driver.cc:444] REPLICATION: finished
0925 09:57:20.298681 (+ 418us) write_op.cc:301] APPLY: starting
0925 09:57:20.298740 (+ 59us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:20.298839 (+ 99us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:20.298935 (+ 96us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:20.299305 (+ 370us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:20.299504 (+ 199us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 09:57:20.299548 (+ 44us) write_op.cc:312] APPLY: finished
0925 09:57:20.299723 (+ 175us) log.cc:842] Serialized 29 byte log entry
0925 09:57:20.299844 (+ 121us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:20.299907 (+ 63us) write_op.cc:454] Released schema lock
0925 09:57:20.299936 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:20.300090 (+ 154us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:20.300123 (+ 33us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":281,"apply.run_cpu_time_us":1552,"apply.run_wall_time_us":1592,"mutex_wait_us":148,"num_ops":2,"prepare.queue_time_us":2359,"prepare.run_cpu_time_us":2186,"prepare.run_wall_time_us":2323,"raft.queue_time_us":401,"raft.run_cpu_time_us":691,"raft.run_wall_time_us":3257,"replication_time_us":10917}]]}
I20260925 09:57:20.335728 11701 tablet.cc:1722] T 0a2bbe141e564a96be05de86f2993234 P b8850818e2ee42a082802d0095c08d21: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260925 09:57:20.337330 10387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_cd9d59504d7a408ba2f9b149cbe1c5fa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:20.339344 10387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cd9d59504d7a408ba2f9b149cbe1c5fa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:20.340189 11664 tablet.cc:1722] T 28d60531970a4ab58ce5ef99cb3a96bf P c43461c4ee854ae1ac52b873ccada318: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 09:57:20.340855 11674 consensus_queue.cc:1035] T 752105c1514f4c1db2a11b95b413b4f6 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:20.354777 11030 heartbeater.cc:347] Connected to a master server at 127.10.9.61:33047
I20260925 09:57:20.357869 10953 tablet_service.cc:1467] Processing CreateTablet for tablet 7a31b999f0fd447eacff13de71aa3fff (DEFAULT_TABLE table=default.Table_489e2c74ed6249ce9cf54541e0a491cd [id=8d2c4997954141129ebbec9e72658d90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:20.358973 10953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a31b999f0fd447eacff13de71aa3fff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:20.368953 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26fae3fbfe374f348a1729c21ec751b9" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:20.369629 10984 raft_consensus.cc:2379] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c43461c4ee854ae1ac52b873ccada318 in current term 1: Already voted for candidate be97ab5d75934a648d4ab76341193ec0 in this term.
I20260925 09:57:20.384068 10659 tablet_service.cc:1467] Processing CreateTablet for tablet 4e3120347c8e4a3eab57d6562ae40e8d (DEFAULT_TABLE table=default.Table_8e186a99ccba425b9b588041d2364a40 [id=2d6bc0e2aaee4b1483b01fc678505bb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:20.384521 10391 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 0a2bbe141e564a96be05de86f2993234 (table default.Table_b5c66e748a8741f78316bd5264f0f186 [id=aed8a2ddc51545ad9cec4d272dcf3daf]). Expected version 3 got 2
I20260925 09:57:20.385071 10659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e3120347c8e4a3eab57d6562ae40e8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:20.385362 10391 catalog_manager.cc:5495] T 42ec275c905343c4be1b72dc2bd0f6e3 P c43461c4ee854ae1ac52b873ccada318 reported cstate change: term changed from 0 to 1, leader changed from <none> to c43461c4ee854ae1ac52b873ccada318 (127.10.9.2). New cstate: current_term: 1 leader_uuid: "c43461c4ee854ae1ac52b873ccada318" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:20.387147 10391 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 28d60531970a4ab58ce5ef99cb3a96bf (table default.Table_ce09d3a8d8e84973894f33b18bbfdb22 [id=6851500c7dba44f1bf353bb79bafd97a]). Expected version 2 got 0
I20260925 09:57:20.388772 10391 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet ef6fefaa76d44fe798201aab2f21b9c7 (table default.Table_46e4c497898b454c85b7c2a299ebf3d9 [id=80b48dd413e3446eb7ac65e5f957b5dc]). Expected version 2 got 0
I20260925 09:57:20.390921 10384 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 4be3f6e81ddf418fabc7b694a2e62f35
I20260925 09:57:20.392246 10384 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:35140 (request call id 181) took 1540 ms. Trace:
W20260925 09:57:20.396575 10918 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 199007us
I20260925 09:57:20.401003 11579 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 144) took 2087 ms. Trace:
I20260925 09:57:20.401321 11579 rpcz_store.cc:276] 0925 09:57:18.313589 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:18.329994 (+ 16405us) service_pool.cc:227] Handling call
0925 09:57:20.194561 (+1864567us) ts_tablet_manager.cc:1145] Deleting tablet 4c4e8790282a49fd849a34ca11bef5bf
0925 09:57:20.194607 (+ 46us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:20.270540 (+ 75933us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:20.315278 (+ 44738us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:20.400942 (+ 85664us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1820150}
I20260925 09:57:20.402668 11579 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 155) took 1365 ms. Trace:
I20260925 09:57:20.392506 10384 rpcz_store.cc:276] 0925 09:57:18.852077 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:18.904173 (+ 52096us) service_pool.cc:227] Handling call
0925 09:57:19.000277 (+ 96104us) spinlock_profiling.cc:243] Waited 95.1 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2adb466 00007fabf061cb34 00007fabf061cac2 00007fabf061c9f1 00007fabf061c96c 00007fabf061bc22 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:19.001422 (+ 1145us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:57:19.002520 (+ 1098us) catalog_manager.cc:2752] Modifying in-memory table state
0925 09:57:19.002675 (+ 155us) catalog_manager.cc:2761] Locking tablets
0925 09:57:19.003785 (+ 1110us) catalog_manager.cc:2775] Removing table and tablets from system table
0925 09:57:19.120924 (+117139us) catalog_manager.cc:2793] Removing table from by-name map
0925 09:57:19.121126 (+ 202us) catalog_manager.cc:2809] Committing in-memory state
0925 09:57:20.390833 (+1269707us) catalog_manager.cc:2813] Aborting table tasks
0925 09:57:20.392176 (+ 1343us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:19.008711 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:19.009073 (+ 362us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:19.009097 (+ 24us) write_op.cc:435] Acquired schema lock
0925 09:57:19.009111 (+ 14us) tablet.cc:660] Decoding operations
0925 09:57:19.009437 (+ 326us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:19.009474 (+ 37us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:19.009497 (+ 23us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:19.009782 (+ 285us) tablet.cc:699] Row locks acquired
0925 09:57:19.009797 (+ 15us) write_op.cc:260] PREPARE: finished
0925 09:57:19.009907 (+ 110us) write_op.cc:270] Start()
0925 09:57:19.009989 (+ 82us) write_op.cc:276] Timestamp: P: 1790330239009836 usec, L: 0
0925 09:57:19.010005 (+ 16us) op_driver.cc:331] REPLICATION: starting
0925 09:57:19.010469 (+ 464us) log.cc:842] Serialized 762 byte log entry
0925 09:57:19.103692 (+ 93223us) op_driver.cc:444] REPLICATION: finished
0925 09:57:19.105074 (+ 1382us) write_op.cc:301] APPLY: starting
0925 09:57:19.105138 (+ 64us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:19.105260 (+ 122us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:19.105370 (+ 110us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:19.105625 (+ 255us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:19.105768 (+ 143us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 09:57:19.105814 (+ 46us) write_op.cc:312] APPLY: finished
0925 09:57:19.105991 (+ 177us) log.cc:842] Serialized 29 byte log entry
0925 09:57:19.106140 (+ 149us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:19.106234 (+ 94us) write_op.cc:454] Released schema lock
0925 09:57:19.106266 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:19.106443 (+ 177us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:19.106487 (+ 44us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":218650496,"child_traces":[["op",{"apply.queue_time_us":849,"apply.run_cpu_time_us":1607,"apply.run_wall_time_us":1607,"num_ops":2,"prepare.queue_time_us":2304,"prepare.run_cpu_time_us":2039,"prepare.run_wall_time_us":3402,"raft.queue_time_us":78,"raft.run_cpu_time_us":16,"raft.run_wall_time_us":16,"replication_time_us":93610}]]}
I20260925 09:57:20.406175 11085 tablet_bootstrap.cc:492] T 359090244c4748e8a2d45e3c77e89498 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:20.406401 11700 raft_consensus.cc:482] T 197f61525b0c4f9da441d98701d231a3 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:20.406813 11700 raft_consensus.cc:504] T 197f61525b0c4f9da441d98701d231a3 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.406543 11085 ts_tablet_manager.cc:1397] T 359090244c4748e8a2d45e3c77e89498 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.284s user 0.015s sys 0.012s
I20260925 09:57:20.409016 11700 leader_election.cc:288] T 197f61525b0c4f9da441d98701d231a3 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:20.411123 11085 raft_consensus.cc:348] T 359090244c4748e8a2d45e3c77e89498 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.411806 11085 raft_consensus.cc:374] T 359090244c4748e8a2d45e3c77e89498 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:20.412029 11085 raft_consensus.cc:729] T 359090244c4748e8a2d45e3c77e89498 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:20.418395 10807 tablet_service.cc:1467] Processing CreateTablet for tablet 4e3120347c8e4a3eab57d6562ae40e8d (DEFAULT_TABLE table=default.Table_8e186a99ccba425b9b588041d2364a40 [id=2d6bc0e2aaee4b1483b01fc678505bb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:20.418296 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "197f61525b0c4f9da441d98701d231a3" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:20.419525 10807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e3120347c8e4a3eab57d6562ae40e8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:20.423854 10810 tablet_service.cc:1514] Processing DeleteTablet for tablet 4be3f6e81ddf418fabc7b694a2e62f35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:18 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:20.402992 11579 rpcz_store.cc:276] 0925 09:57:19.037173 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:19.037301 (+ 128us) service_pool.cc:227] Handling call
0925 09:57:20.402327 (+1365026us) ts_tablet_manager.cc:1145] Deleting tablet 977a59175e5b43eeb6f23b938afd00a3
0925 09:57:20.402372 (+ 45us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:20.402620 (+ 248us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1319962}
I20260925 09:57:20.428916 11579 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 156) took 1390 ms. Trace:
I20260925 09:57:20.433353 11579 rpcz_store.cc:276] 0925 09:57:19.038164 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:19.043521 (+ 5357us) service_pool.cc:227] Handling call
0925 09:57:20.428526 (+1385005us) ts_tablet_manager.cc:1145] Deleting tablet a44d9f2f7e3e48aeb0f31fa29ae1988a
0925 09:57:20.428575 (+ 49us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:20.428858 (+ 283us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1345649}
I20260925 09:57:20.439096 10393 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 04fafb9dcf274806b11b2598e73853e9
I20260925 09:57:20.439854 11579 tablet_replica.cc:330] T c16e096038084b2b9148390b6e773294 P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:20.439661 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 4c4e8790282a49fd849a34ca11bef5bf (table default.Table_beb33698ec744e42b4f6ac2fdd8fe2b9 [id=9c778da25cde488a953f712100b2fbb2]) successfully deleted
I20260925 09:57:20.440511 10393 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:35140 (request call id 184) took 1489 ms. Trace:
I20260925 09:57:20.429617 11085 consensus_queue.cc:260] T 359090244c4748e8a2d45e3c77e89498 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:20.447845 10621 leader_election.cc:341] T 197f61525b0c4f9da441d98701d231a3 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:20.448773 10810 tablet_service.cc:1514] Processing DeleteTablet for tablet 04fafb9dcf274806b11b2598e73853e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:19 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:20.449112 11085 ts_tablet_manager.cc:1428] T 359090244c4748e8a2d45e3c77e89498 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.040s user 0.004s sys 0.000s
I20260925 09:57:20.449944 11579 raft_consensus.cc:2229] T c16e096038084b2b9148390b6e773294 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:20.450528 11579 raft_consensus.cc:2258] T c16e096038084b2b9148390b6e773294 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:20.452509 11579 ts_tablet_manager.cc:1905] T c16e096038084b2b9148390b6e773294 P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:20.440838 10393 rpcz_store.cc:276] 0925 09:57:18.951173 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:19.029191 (+ 78018us) service_pool.cc:227] Handling call
0925 09:57:19.030688 (+ 1497us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:57:19.031323 (+ 635us) catalog_manager.cc:2752] Modifying in-memory table state
0925 09:57:19.082071 (+ 50748us) catalog_manager.cc:2761] Locking tablets
0925 09:57:19.659497 (+577426us) catalog_manager.cc:2775] Removing table and tablets from system table
0925 09:57:19.760761 (+101264us) catalog_manager.cc:2793] Removing table from by-name map
0925 09:57:19.761041 (+ 280us) catalog_manager.cc:2809] Committing in-memory state
0925 09:57:20.438995 (+677954us) catalog_manager.cc:2813] Aborting table tasks
0925 09:57:20.440450 (+ 1455us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:19.661444 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:19.661858 (+ 414us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:19.661891 (+ 33us) write_op.cc:435] Acquired schema lock
0925 09:57:19.661931 (+ 40us) tablet.cc:660] Decoding operations
0925 09:57:19.662344 (+ 413us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:19.662388 (+ 44us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:19.662409 (+ 21us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:19.662652 (+ 243us) tablet.cc:699] Row locks acquired
0925 09:57:19.662666 (+ 14us) write_op.cc:260] PREPARE: finished
0925 09:57:19.662799 (+ 133us) write_op.cc:270] Start()
0925 09:57:19.662878 (+ 79us) write_op.cc:276] Timestamp: P: 1790330239662758 usec, L: 0
0925 09:57:19.662896 (+ 18us) op_driver.cc:331] REPLICATION: starting
0925 09:57:19.663443 (+ 547us) log.cc:842] Serialized 756 byte log entry
0925 09:57:19.757653 (+ 94210us) op_driver.cc:444] REPLICATION: finished
0925 09:57:19.758006 (+ 353us) write_op.cc:301] APPLY: starting
0925 09:57:19.758066 (+ 60us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:19.758177 (+ 111us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:19.758342 (+ 165us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:19.758664 (+ 322us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:19.758874 (+ 210us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 09:57:19.758944 (+ 70us) write_op.cc:312] APPLY: finished
0925 09:57:19.759167 (+ 223us) log.cc:842] Serialized 29 byte log entry
0925 09:57:19.759341 (+ 174us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:19.759448 (+ 107us) write_op.cc:454] Released schema lock
0925 09:57:19.759496 (+ 48us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:19.759729 (+ 233us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:19.759762 (+ 33us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":194,"apply.run_cpu_time_us":1983,"apply.run_wall_time_us":1981,"num_ops":2,"prepare.queue_time_us":162,"prepare.run_cpu_time_us":2339,"prepare.run_wall_time_us":7757,"raft.queue_time_us":3985,"raft.run_cpu_time_us":845,"raft.run_wall_time_us":844,"replication_time_us":94681}]]}
I20260925 09:57:20.467887 11085 tablet_bootstrap.cc:492] T c8522c08bdd94806b5c84f5a41ca5bf4 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:20.468751 10393 master_service.cc:946] ReplaceTablet: received request to replace tablet 6161774f61ae48ab9764a05f2188a80b from {username='slave'} at 127.0.0.1:35140
W20260925 09:57:20.470485 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 977a59175e5b43eeb6f23b938afd00a3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 977a59175e5b43eeb6f23b938afd00a3 already in progress: creating tablet
W20260925 09:57:20.471457 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet a44d9f2f7e3e48aeb0f31fa29ae1988a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a44d9f2f7e3e48aeb0f31fa29ae1988a already in progress: creating tablet
W20260925 09:57:20.466048 10392 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.1:56563 (request call id 29) took 1229 ms (client timeout 1000 ms). Trace:
W20260925 09:57:20.474929 10392 rpcz_store.cc:269] 0925 09:57:19.236348 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:19.260472 (+ 24124us) service_pool.cc:227] Handling call
0925 09:57:20.465978 (+1205506us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":20}
I20260925 09:57:20.476907 10659 tablet_service.cc:1514] Processing DeleteTablet for tablet 04fafb9dcf274806b11b2598e73853e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:19 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:20.477283 10392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_88612386ae4c426fb71eae81bdd32061"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:20.478930 10666 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d55154ceb064f22bd6f37c8ae5dce74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:12 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:20.480556 10666 tablet_service.cc:1514] Processing DeleteTablet for tablet 752105c1514f4c1db2a11b95b413b4f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:14 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:20.482048 11085 tablet_bootstrap.cc:654] T c8522c08bdd94806b5c84f5a41ca5bf4 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:20.483970 10666 tablet_service.cc:1514] Processing DeleteTablet for tablet c07c6205d1284a6fbb369c010a226da7 with delete_type TABLET_DATA_DELETED (replaced by tablet 4e3120347c8e4a3eab57d6562ae40e8d) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:20.485627 10666 tablet_service.cc:1514] Processing DeleteTablet for tablet 41df75fe4b8e4856bdbb5ead3cf4fd24 with delete_type TABLET_DATA_DELETED (replaced by tablet c7cd1d6a603642afa2c8c737079961b5) from {username='slave'} at 127.0.0.1:37876
W20260925 09:57:20.487067 10392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_88612386ae4c426fb71eae81bdd32061 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:20.487274 10666 tablet_service.cc:1514] Processing DeleteTablet for tablet c16e096038084b2b9148390b6e773294 with delete_type TABLET_DATA_DELETED (replaced by tablet d5956be0da6441c79723e72f12dfe54a) from {username='slave'} at 127.0.0.1:37876
W20260925 09:57:20.505174 10388 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.1:56563 (request call id 28) took 2443 ms (client timeout 1000 ms). Trace:
W20260925 09:57:20.505623 10388 rpcz_store.cc:269] 0925 09:57:18.062260 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:18.394513 (+332253us) service_pool.cc:227] Handling call
0925 09:57:20.505108 (+2110595us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:18.945467 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:18.945815 (+ 348us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:18.945837 (+ 22us) write_op.cc:435] Acquired schema lock
0925 09:57:18.945854 (+ 17us) tablet.cc:660] Decoding operations
0925 09:57:18.946253 (+ 399us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:18.946303 (+ 50us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:18.946335 (+ 32us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:18.946675 (+ 340us) tablet.cc:699] Row locks acquired
0925 09:57:18.946692 (+ 17us) write_op.cc:260] PREPARE: finished
0925 09:57:18.946821 (+ 129us) write_op.cc:270] Start()
0925 09:57:18.946921 (+ 100us) write_op.cc:276] Timestamp: P: 1790330238946749 usec, L: 0
0925 09:57:18.946943 (+ 22us) op_driver.cc:331] REPLICATION: starting
0925 09:57:18.995986 (+ 49043us) log.cc:842] Serialized 895 byte log entry
0925 09:57:18.997702 (+ 1716us) spinlock_profiling.cc:243] Waited 48.4 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90471 00007fabf0b07198 00007fabf0f68ade 00007fabf0f682b8 00007fabf0f6cba1 00007fabf0f6cb59 00007fabf0f6cae9 00007fabf0f6cab1 00007fabf0f6bdad 0000000000580784 00000000005805b9
0925 09:57:19.084596 (+ 86894us) op_driver.cc:444] REPLICATION: finished
0925 09:57:19.107967 (+ 23371us) write_op.cc:301] APPLY: starting
0925 09:57:19.108024 (+ 57us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:19.108134 (+ 110us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:19.108158 (+ 24us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:19.112668 (+ 4510us) spinlock_profiling.cc:243] Waited 2.2 ms on lock 0x7b5400011c90. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0935f11 00007fabf094c027 00007fabf0fe1e5f 00007fabf0e8108a 00007fabf0e839db 00007fabf0e835e3 00007fabf0f7db14 00007fabf0f69dfa 00007fabf0f6ddd1 00007fabf0f6dd89 00007fabf0f6dd19
0925 09:57:19.112846 (+ 178us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:19.112988 (+ 142us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 09:57:19.113032 (+ 44us) write_op.cc:312] APPLY: finished
0925 09:57:19.114025 (+ 993us) log.cc:842] Serialized 29 byte log entry
0925 09:57:19.114199 (+ 174us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:19.114261 (+ 62us) write_op.cc:454] Released schema lock
0925 09:57:19.114301 (+ 40us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:19.115435 (+ 1134us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:19.115464 (+ 29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":179664,"reported_tablets":19,"child_traces":[["op",{"apply.queue_time_us":19880,"apply.run_cpu_time_us":3931,"apply.run_wall_time_us":7702,"num_ops":2,"prepare.queue_time_us":83,"prepare.run_cpu_time_us":4584,"prepare.run_wall_time_us":52448,"raft.queue_time_us":1614,"raft.run_cpu_time_us":14,"raft.run_wall_time_us":15,"replication_time_us":137576,"spinlock_wait_cycles":116461184}]]}
I20260925 09:57:20.510913 10388 master_service.cc:946] ReplaceTablet: received request to replace tablet 89a143a5623e4ef38027381a56c95227 from {username='slave'} at 127.0.0.1:35140
I20260925 09:57:20.512291 11709 tablet_replica.cc:330] T 04fafb9dcf274806b11b2598e73853e9 P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:20.513092 11709 raft_consensus.cc:2229] T 04fafb9dcf274806b11b2598e73853e9 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:20.513840 11709 raft_consensus.cc:2258] T 04fafb9dcf274806b11b2598e73853e9 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:20.529632 11709 ts_tablet_manager.cc:1905] T 04fafb9dcf274806b11b2598e73853e9 P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 09:57:20.529923 10389 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.3:41469 (request call id 18) took 3063 ms (client timeout 1000 ms). Trace:
W20260925 09:57:20.530349 10389 rpcz_store.cc:269] 0925 09:57:17.466060 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:17.515079 (+ 49019us) service_pool.cc:227] Handling call
0925 09:57:20.529860 (+3014781us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":39}
I20260925 09:57:20.536104 10659 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c4e8790282a49fd849a34ca11bef5bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:11 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:20.536655 10666 tablet_service.cc:1514] Processing DeleteTablet for tablet 91c09118b43b41eca8f7040323ba76f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:15 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:20.547583 10658 tablet_service.cc:1514] Processing DeleteTablet for tablet 4be3f6e81ddf418fabc7b694a2e62f35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:18 UTC) from {username='slave'} at 127.0.0.1:37876
W20260925 09:57:20.551352 10772 consensus_peers.cc:487] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P c43461c4ee854ae1ac52b873ccada318 -> Peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Couldn't send request to peer be97ab5d75934a648d4ab76341193ec0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 41df75fe4b8e4856bdbb5ead3cf4fd24. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:20.551999 10772 leader_election.cc:302] T 26fae3fbfe374f348a1729c21ec751b9 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:20.553014 11674 raft_consensus.cc:3037] T 26fae3fbfe374f348a1729c21ec751b9 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:20.553535 11082 tablet_bootstrap.cc:492] T c588b0c05a1a4f3ea545066183e76502 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:20.554131 11082 ts_tablet_manager.cc:1397] T c588b0c05a1a4f3ea545066183e76502 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.256s user 0.012s sys 0.025s
I20260925 09:57:20.556588 11082 raft_consensus.cc:348] T c588b0c05a1a4f3ea545066183e76502 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.360504 11655 raft_consensus.cc:482] T 977a59175e5b43eeb6f23b938afd00a3 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:20.563802 11082 raft_consensus.cc:374] T c588b0c05a1a4f3ea545066183e76502 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:20.564047 11082 raft_consensus.cc:729] T c588b0c05a1a4f3ea545066183e76502 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:20.564672 11082 consensus_queue.cc:260] T c588b0c05a1a4f3ea545066183e76502 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.570487 10389 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_c2ce8fea3dc743379c1912445278a487" } new_table_name: "default.Table_c22088d959e84c3cbabadde4d3445178" modify_external_catalogs: true
I20260925 09:57:20.571650 10390 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35140 (request call id 186) took 1338 ms. Trace:
W20260925 09:57:20.543998 10379 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 -> Peer b8be057e0b6d40a3a4b90f2b1f4a0fa2 (127.10.9.60:36985): Couldn't send request to peer b8be057e0b6d40a3a4b90f2b1f4a0fa2. Status: Network error: Client connection negotiation failed: client connection to 127.10.9.60:36985: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260925 09:57:20.576097 11082 ts_tablet_manager.cc:1428] T c588b0c05a1a4f3ea545066183e76502 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.022s user 0.005s sys 0.000s
I20260925 09:57:20.576881 11082 tablet_bootstrap.cc:492] T cfa619b699004882a82976f0ef8204ad P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:20.580237 10387 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35140 (request call id 187) took 1254 ms. Trace:
I20260925 09:57:20.580536 10387 rpcz_store.cc:276] 0925 09:57:19.325588 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:20.336891 (+1011303us) service_pool.cc:227] Handling call
0925 09:57:20.341725 (+ 4834us) catalog_manager.cc:2075] Acquired catalog manager lock
0925 09:57:20.343446 (+ 1721us) catalog_manager.cc:2121] Created new table and tablet info
0925 09:57:20.579620 (+236174us) catalog_manager.cc:2181] Wrote table and tablets to system table
0925 09:57:20.579947 (+ 327us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0925 09:57:20.580184 (+ 237us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:20.494087 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:20.494502 (+ 415us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:20.494534 (+ 32us) write_op.cc:435] Acquired schema lock
0925 09:57:20.494552 (+ 18us) tablet.cc:660] Decoding operations
0925 09:57:20.494879 (+ 327us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:20.494925 (+ 46us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:20.494956 (+ 31us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:20.495175 (+ 219us) tablet.cc:699] Row locks acquired
0925 09:57:20.495187 (+ 12us) write_op.cc:260] PREPARE: finished
0925 09:57:20.495297 (+ 110us) write_op.cc:270] Start()
0925 09:57:20.495495 (+ 198us) write_op.cc:276] Timestamp: P: 1790330240495220 usec, L: 0
0925 09:57:20.495520 (+ 25us) op_driver.cc:331] REPLICATION: starting
0925 09:57:20.496104 (+ 584us) log.cc:842] Serialized 433 byte log entry
0925 09:57:20.576031 (+ 79927us) op_driver.cc:444] REPLICATION: finished
0925 09:57:20.576616 (+ 585us) write_op.cc:301] APPLY: starting
0925 09:57:20.576685 (+ 69us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:20.576797 (+ 112us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:20.576892 (+ 95us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:20.577346 (+ 454us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:20.577598 (+ 252us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 09:57:20.577715 (+ 117us) write_op.cc:312] APPLY: finished
0925 09:57:20.577991 (+ 276us) log.cc:842] Serialized 29 byte log entry
0925 09:57:20.578159 (+ 168us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:20.578219 (+ 60us) write_op.cc:454] Released schema lock
0925 09:57:20.578248 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:20.578402 (+ 154us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:20.578444 (+ 42us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":440,"apply.run_cpu_time_us":1971,"apply.run_wall_time_us":1971,"mutex_wait_us":147312,"num_ops":2,"prepare.queue_time_us":149390,"prepare.run_cpu_time_us":2157,"prepare.run_wall_time_us":3135,"replication_time_us":80433}]]}
I20260925 09:57:20.586987 11084 tablet_bootstrap.cc:654] T 6161774f61ae48ab9764a05f2188a80b P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:20.587457 10387 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_15b07a280b634333985637e0f6746764" } new_table_name: "default.Table_64ba123898174e72a6884dbb9ad07de3" modify_external_catalogs: true
I20260925 09:57:20.590644 10393 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 6161774f61ae48ab9764a05f2188a80b
I20260925 09:57:20.593199 10393 catalog_manager.cc:6565] ReplaceTablet: tablet 6161774f61ae48ab9764a05f2188a80b deleted and replaced by tablet 10657a81058241fc988d37e5ea7ff8cf
I20260925 09:57:20.593901 10393 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:35140 (request call id 188) took 1079 ms. Trace:
I20260925 09:57:20.600342 10658 tablet_service.cc:1514] Processing DeleteTablet for tablet 6161774f61ae48ab9764a05f2188a80b with delete_type TABLET_DATA_DELETED (replaced by tablet 10657a81058241fc988d37e5ea7ff8cf) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:20.572345 10390 rpcz_store.cc:276] 0925 09:57:19.232968 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:20.277383 (+1044415us) service_pool.cc:227] Handling call
0925 09:57:20.280865 (+ 3482us) catalog_manager.cc:2075] Acquired catalog manager lock
0925 09:57:20.321439 (+ 40574us) catalog_manager.cc:2121] Created new table and tablet info
0925 09:57:20.545288 (+223849us) catalog_manager.cc:2181] Wrote table and tablets to system table
0925 09:57:20.571323 (+ 26035us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0925 09:57:20.571598 (+ 275us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:20.489731 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:20.490053 (+ 322us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:20.490080 (+ 27us) write_op.cc:435] Acquired schema lock
0925 09:57:20.490091 (+ 11us) tablet.cc:660] Decoding operations
0925 09:57:20.490351 (+ 260us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:20.490397 (+ 46us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:20.490417 (+ 20us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:20.490625 (+ 208us) tablet.cc:699] Row locks acquired
0925 09:57:20.490643 (+ 18us) write_op.cc:260] PREPARE: finished
0925 09:57:20.490782 (+ 139us) write_op.cc:270] Start()
0925 09:57:20.490880 (+ 98us) write_op.cc:276] Timestamp: P: 1790330240490704 usec, L: 0
0925 09:57:20.490902 (+ 22us) op_driver.cc:331] REPLICATION: starting
0925 09:57:20.491532 (+ 630us) log.cc:842] Serialized 433 byte log entry
0925 09:57:20.532747 (+ 41215us) op_driver.cc:444] REPLICATION: finished
0925 09:57:20.533038 (+ 291us) write_op.cc:301] APPLY: starting
0925 09:57:20.533093 (+ 55us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:20.533196 (+ 103us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:20.533300 (+ 104us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:20.533682 (+ 382us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:20.533812 (+ 130us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 09:57:20.533853 (+ 41us) write_op.cc:312] APPLY: finished
0925 09:57:20.534022 (+ 169us) log.cc:842] Serialized 29 byte log entry
0925 09:57:20.534160 (+ 138us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:20.534218 (+ 58us) write_op.cc:454] Released schema lock
0925 09:57:20.534269 (+ 51us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:20.534433 (+ 164us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:20.534459 (+ 26us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":150,"apply.run_cpu_time_us":1582,"apply.run_wall_time_us":1583,"num_ops":2,"prepare.queue_time_us":167175,"prepare.run_cpu_time_us":2109,"prepare.run_wall_time_us":2109,"raft.queue_time_us":1395,"raft.run_cpu_time_us":581,"raft.run_wall_time_us":581,"replication_time_us":41753}]]}
I20260925 09:57:20.604394 10390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_44ff944ba4b84cb7a677f7dd03f8f7dd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:20.610452 11082 tablet_bootstrap.cc:654] T cfa619b699004882a82976f0ef8204ad P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:20.625543 11709 ts_tablet_manager.cc:1918] T 04fafb9dcf274806b11b2598e73853e9 P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:20.626708 11709 log.cc:1196] T 04fafb9dcf274806b11b2598e73853e9 P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/04fafb9dcf274806b11b2598e73853e9
I20260925 09:57:20.628001 11709 ts_tablet_manager.cc:1939] T 04fafb9dcf274806b11b2598e73853e9 P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:20.583271 11655 raft_consensus.cc:504] T 977a59175e5b43eeb6f23b938afd00a3 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.632872 11082 tablet_bootstrap.cc:492] T cfa619b699004882a82976f0ef8204ad P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:20.633198 11684 raft_consensus.cc:482] T c588b0c05a1a4f3ea545066183e76502 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:20.633654 11684 raft_consensus.cc:504] T c588b0c05a1a4f3ea545066183e76502 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.635083 11082 ts_tablet_manager.cc:1397] T cfa619b699004882a82976f0ef8204ad P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.058s user 0.008s sys 0.001s
I20260925 09:57:20.637353 11684 leader_election.cc:288] T c588b0c05a1a4f3ea545066183e76502 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:20.638178 11565 raft_consensus.cc:482] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:20.639089 11565 raft_consensus.cc:504] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.639253 11082 raft_consensus.cc:348] T cfa619b699004882a82976f0ef8204ad P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.639853 11082 raft_consensus.cc:374] T cfa619b699004882a82976f0ef8204ad P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:20.640108 11082 raft_consensus.cc:729] T cfa619b699004882a82976f0ef8204ad P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:20.641566 11709 tablet_replica.cc:330] T 2d55154ceb064f22bd6f37c8ae5dce74 P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:20.642923 11709 raft_consensus.cc:2229] T 2d55154ceb064f22bd6f37c8ae5dce74 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:20.648167 11709 pending_rounds.cc:62] T 2d55154ceb064f22bd6f37c8ae5dce74 P b8850818e2ee42a082802d0095c08d21: Trying to abort 1 pending ops.
I20260925 09:57:20.649076 11709 pending_rounds.cc:68] T 2d55154ceb064f22bd6f37c8ae5dce74 P b8850818e2ee42a082802d0095c08d21: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192660688863232 op_type: NO_OP noop_request { }
I20260925 09:57:20.650764 11709 raft_consensus.cc:2866] T 2d55154ceb064f22bd6f37c8ae5dce74 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:57:20.651521 11709 raft_consensus.cc:2258] T 2d55154ceb064f22bd6f37c8ae5dce74 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:20.649878 11082 consensus_queue.cc:260] T cfa619b699004882a82976f0ef8204ad P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:20.645838 10983 consensus_meta.cc:287] T cf35f39bcd6a41128acb2a4936fcc316 P be97ab5d75934a648d4ab76341193ec0: Time spent flushing consensus metadata: real 0.534s user 0.007s sys 0.001s
I20260925 09:57:20.656416 10983 raft_consensus.cc:2454] T cf35f39bcd6a41128acb2a4936fcc316 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:20.647596 11655 leader_election.cc:288] T 977a59175e5b43eeb6f23b938afd00a3 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:20.665303 11709 ts_tablet_manager.cc:1905] T 2d55154ceb064f22bd6f37c8ae5dce74 P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:20.666611 11565 leader_election.cc:288] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:20.666050 11082 ts_tablet_manager.cc:1428] T cfa619b699004882a82976f0ef8204ad P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.031s user 0.006s sys 0.000s
I20260925 09:57:20.677065 11082 tablet_bootstrap.cc:492] T c8522c08bdd94806b5c84f5a41ca5bf4 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:20.690256 11082 tablet_bootstrap.cc:654] T c8522c08bdd94806b5c84f5a41ca5bf4 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:20.700012 10688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c588b0c05a1a4f3ea545066183e76502" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:20.700642 10688 raft_consensus.cc:2454] T c588b0c05a1a4f3ea545066183e76502 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be97ab5d75934a648d4ab76341193ec0 in term 0.
I20260925 09:57:20.706530 10805 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c4e8790282a49fd849a34ca11bef5bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:11 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:20.710656 11674 raft_consensus.cc:2726] T 26fae3fbfe374f348a1729c21ec751b9 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:20.713675 10983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "197f61525b0c4f9da441d98701d231a3" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:20.714309 10983 raft_consensus.cc:2454] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:20.715350 11082 tablet_bootstrap.cc:492] T c8522c08bdd94806b5c84f5a41ca5bf4 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:20.715773 11082 ts_tablet_manager.cc:1397] T c8522c08bdd94806b5c84f5a41ca5bf4 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.039s user 0.008s sys 0.002s
I20260925 09:57:20.718046 11082 raft_consensus.cc:348] T c8522c08bdd94806b5c84f5a41ca5bf4 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.718745 11082 raft_consensus.cc:374] T c8522c08bdd94806b5c84f5a41ca5bf4 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:20.718991 11082 raft_consensus.cc:729] T c8522c08bdd94806b5c84f5a41ca5bf4 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:20.719161 11716 raft_consensus.cc:482] T c8522c08bdd94806b5c84f5a41ca5bf4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:20.719230 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c588b0c05a1a4f3ea545066183e76502" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:20.719808 11716 raft_consensus.cc:504] T c8522c08bdd94806b5c84f5a41ca5bf4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.721585 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "977a59175e5b43eeb6f23b938afd00a3" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:20.723315 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7cd1d6a603642afa2c8c737079961b5" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:20.723539 10810 tablet_service.cc:1514] Processing DeleteTablet for tablet 6161774f61ae48ab9764a05f2188a80b with delete_type TABLET_DATA_DELETED (replaced by tablet 10657a81058241fc988d37e5ea7ff8cf) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:20.725076 11084 tablet_bootstrap.cc:492] T 6161774f61ae48ab9764a05f2188a80b P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:20.725906 11084 ts_tablet_manager.cc:1397] T 6161774f61ae48ab9764a05f2188a80b P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.604s user 0.008s sys 0.008s
W20260925 09:57:20.729858 10918 leader_election.cc:341] T 977a59175e5b43eeb6f23b938afd00a3 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:20.730464 10918 leader_election.cc:341] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:20.733584 11082 consensus_queue.cc:260] T c8522c08bdd94806b5c84f5a41ca5bf4 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.737695 11709 ts_tablet_manager.cc:1918] T 2d55154ceb064f22bd6f37c8ae5dce74 P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:20.738509 11579 ts_tablet_manager.cc:1918] T c16e096038084b2b9148390b6e773294 P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:20.738924 11579 log.cc:1196] T c16e096038084b2b9148390b6e773294 P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/c16e096038084b2b9148390b6e773294
I20260925 09:57:20.739734 11579 ts_tablet_manager.cc:1939] T c16e096038084b2b9148390b6e773294 P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:20.741887 11579 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 157) took 1697 ms. Trace:
I20260925 09:57:20.742271 10835 raft_consensus.cc:3037] T 359090244c4748e8a2d45e3c77e89498 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:20.742204 11579 rpcz_store.cc:276] 0925 09:57:19.044237 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:19.044351 (+ 114us) service_pool.cc:227] Handling call
0925 09:57:20.439700 (+1395349us) ts_tablet_manager.cc:1145] Deleting tablet c16e096038084b2b9148390b6e773294
0925 09:57:20.439742 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:20.487187 (+ 47445us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:20.726481 (+239294us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:20.741828 (+ 15347us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1356638}
I20260925 09:57:20.745533 11670 consensus_queue.cc:1035] T 42ec275c905343c4be1b72dc2bd0f6e3 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:20.746075 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8522c08bdd94806b5c84f5a41ca5bf4" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:20.748868 11709 log.cc:1196] T 2d55154ceb064f22bd6f37c8ae5dce74 P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/2d55154ceb064f22bd6f37c8ae5dce74
I20260925 09:57:20.743367 11579 tablet_replica.cc:330] T c07c6205d1284a6fbb369c010a226da7 P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:20.749943 11579 raft_consensus.cc:2229] T c07c6205d1284a6fbb369c010a226da7 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:20.739945 11082 ts_tablet_manager.cc:1428] T c8522c08bdd94806b5c84f5a41ca5bf4 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.024s user 0.005s sys 0.001s
I20260925 09:57:20.750541 11579 raft_consensus.cc:2258] T c07c6205d1284a6fbb369c010a226da7 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:20.750923 11082 tablet_bootstrap.cc:492] T 67ce2e3325214aa6bdc3347e9e8f82b0 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:20.750041 11709 ts_tablet_manager.cc:1939] T 2d55154ceb064f22bd6f37c8ae5dce74 P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:20.758581 11579 ts_tablet_manager.cc:1905] T c07c6205d1284a6fbb369c010a226da7 P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:20.765127 11084 raft_consensus.cc:348] T 6161774f61ae48ab9764a05f2188a80b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.765807 11084 raft_consensus.cc:374] T 6161774f61ae48ab9764a05f2188a80b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:20.766067 11084 raft_consensus.cc:729] T 6161774f61ae48ab9764a05f2188a80b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:20.766597 11084 consensus_queue.cc:260] T 6161774f61ae48ab9764a05f2188a80b P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.767782 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "977a59175e5b43eeb6f23b938afd00a3" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:20.768795 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7cd1d6a603642afa2c8c737079961b5" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:20.769255 11085 tablet_bootstrap.cc:492] T c8522c08bdd94806b5c84f5a41ca5bf4 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:20.769646 11085 ts_tablet_manager.cc:1397] T c8522c08bdd94806b5c84f5a41ca5bf4 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.302s user 0.017s sys 0.030s
I20260925 09:57:20.772290 10961 tablet_service.cc:1514] Processing DeleteTablet for tablet 4be3f6e81ddf418fabc7b694a2e62f35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:18 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:20.776769 11082 tablet_bootstrap.cc:654] T 67ce2e3325214aa6bdc3347e9e8f82b0 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:20.777948 11709 tablet_replica.cc:330] T 752105c1514f4c1db2a11b95b413b4f6 P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:20.778455 10831 raft_consensus.cc:3037] T 89a143a5623e4ef38027381a56c95227 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:20.779807 11709 raft_consensus.cc:2229] T 752105c1514f4c1db2a11b95b413b4f6 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:20.779973 11085 raft_consensus.cc:348] T c8522c08bdd94806b5c84f5a41ca5bf4 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.780614 11085 raft_consensus.cc:374] T c8522c08bdd94806b5c84f5a41ca5bf4 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:20.780858 11085 raft_consensus.cc:729] T c8522c08bdd94806b5c84f5a41ca5bf4 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:20.781054 11716 leader_election.cc:288] T c8522c08bdd94806b5c84f5a41ca5bf4 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:20.781562 11709 raft_consensus.cc:2258] T 752105c1514f4c1db2a11b95b413b4f6 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:20.781451 11085 consensus_queue.cc:260] T c8522c08bdd94806b5c84f5a41ca5bf4 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.785568 11664 tablet.cc:1722] T ef6fefaa76d44fe798201aab2f21b9c7 P c43461c4ee854ae1ac52b873ccada318: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 09:57:20.788213 11674 raft_consensus.cc:482] T c8522c08bdd94806b5c84f5a41ca5bf4 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:20.788630 11674 raft_consensus.cc:504] T c8522c08bdd94806b5c84f5a41ca5bf4 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.789824 10953 tablet_service.cc:1514] Processing DeleteTablet for tablet 04fafb9dcf274806b11b2598e73853e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:19 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:20.790442 11674 leader_election.cc:288] T c8522c08bdd94806b5c84f5a41ca5bf4 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:20.791610 10963 tablet_service.cc:1514] Processing DeleteTablet for tablet 752105c1514f4c1db2a11b95b413b4f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:14 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:20.792181 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8522c08bdd94806b5c84f5a41ca5bf4" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:20.792636 10963 tablet_service.cc:1514] Processing DeleteTablet for tablet 6161774f61ae48ab9764a05f2188a80b with delete_type TABLET_DATA_DELETED (replaced by tablet 10657a81058241fc988d37e5ea7ff8cf) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:20.792761 10687 raft_consensus.cc:2454] T c8522c08bdd94806b5c84f5a41ca5bf4 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
I20260925 09:57:20.795045 10960 tablet_service.cc:1514] Processing DeleteTablet for tablet cf35f39bcd6a41128acb2a4936fcc316 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:16 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:20.795753 10775 leader_election.cc:302] T c8522c08bdd94806b5c84f5a41ca5bf4 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:20.790774 11709 ts_tablet_manager.cc:1905] T 752105c1514f4c1db2a11b95b413b4f6 P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:20.796460 11084 ts_tablet_manager.cc:1428] T 6161774f61ae48ab9764a05f2188a80b P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.070s user 0.007s sys 0.000s
I20260925 09:57:20.796535 11674 raft_consensus.cc:2781] T c8522c08bdd94806b5c84f5a41ca5bf4 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:20.796845 11674 raft_consensus.cc:482] T c8522c08bdd94806b5c84f5a41ca5bf4 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:20.797123 11674 raft_consensus.cc:3037] T c8522c08bdd94806b5c84f5a41ca5bf4 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:20.801745 10619 leader_election.cc:302] T cf35f39bcd6a41128acb2a4936fcc316 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:20.802958 10619 leader_election.cc:302] T 197f61525b0c4f9da441d98701d231a3 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:20.803846 10958 tablet_service.cc:1467] Processing CreateTablet for tablet 4e3120347c8e4a3eab57d6562ae40e8d (DEFAULT_TABLE table=default.Table_8e186a99ccba425b9b588041d2364a40 [id=2d6bc0e2aaee4b1483b01fc678505bb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:20.804426 11700 raft_consensus.cc:2781] T cf35f39bcd6a41128acb2a4936fcc316 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:20.805156 11627 raft_consensus.cc:2781] T 197f61525b0c4f9da441d98701d231a3 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:20.805476 11627 raft_consensus.cc:482] T 197f61525b0c4f9da441d98701d231a3 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:20.805770 11627 raft_consensus.cc:3037] T 197f61525b0c4f9da441d98701d231a3 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:57:20.806306 10772 consensus_peers.cc:487] T 752105c1514f4c1db2a11b95b413b4f6 P c43461c4ee854ae1ac52b873ccada318 -> Peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Couldn't send request to peer be97ab5d75934a648d4ab76341193ec0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 752105c1514f4c1db2a11b95b413b4f6. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:20.806851 11085 ts_tablet_manager.cc:1428] T c8522c08bdd94806b5c84f5a41ca5bf4 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.037s user 0.005s sys 0.001s
I20260925 09:57:20.637360 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 04fafb9dcf274806b11b2598e73853e9 (table default.Table_bc69fdbbaf5b4a4e8cdfe195eefb16ab [id=d76cef751d12440190294e86649e2639]) successfully deleted
W20260925 09:57:20.807889 10380 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 170644us
I20260925 09:57:20.809433 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 2d55154ceb064f22bd6f37c8ae5dce74 (table default.Table_7eaf247fb74f497d80a88b67fa301cab [id=efe6d05ed9084a1c998e95e99672d4fc]) successfully deleted
W20260925 09:57:20.811075 10390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_44ff944ba4b84cb7a677f7dd03f8f7dd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:20.811582 11084 tablet_bootstrap.cc:492] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:20.814507 11085 tablet_bootstrap.cc:492] T 67ce2e3325214aa6bdc3347e9e8f82b0 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
W20260925 09:57:20.816252 10918 leader_election.cc:341] T c588b0c05a1a4f3ea545066183e76502 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:20.816902 10918 leader_election.cc:341] T c8522c08bdd94806b5c84f5a41ca5bf4 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:20.818336 11565 consensus_queue.cc:1035] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:20.819890 10919 leader_election.cc:302] T c588b0c05a1a4f3ea545066183e76502 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:20.821022 11716 raft_consensus.cc:2781] T c588b0c05a1a4f3ea545066183e76502 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:20.821130 11684 raft_consensus.cc:482] T fe91b39f91274a7ca61e9618d8cbb475 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:20.821385 11716 raft_consensus.cc:482] T c588b0c05a1a4f3ea545066183e76502 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:20.821782 11716 raft_consensus.cc:3037] T c588b0c05a1a4f3ea545066183e76502 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:20.821606 11684 raft_consensus.cc:504] T fe91b39f91274a7ca61e9618d8cbb475 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.806208 11674 raft_consensus.cc:504] T c8522c08bdd94806b5c84f5a41ca5bf4 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.823283 11684 leader_election.cc:288] T fe91b39f91274a7ca61e9618d8cbb475 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:20.826251 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8522c08bdd94806b5c84f5a41ca5bf4" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:20.826793 10687 raft_consensus.cc:3037] T c8522c08bdd94806b5c84f5a41ca5bf4 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:20.826962 11674 leader_election.cc:288] T c8522c08bdd94806b5c84f5a41ca5bf4 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:20.829836 11716 raft_consensus.cc:504] T c588b0c05a1a4f3ea545066183e76502 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.828480 11670 raft_consensus.cc:482] T 89a143a5623e4ef38027381a56c95227 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:20.831483 11670 raft_consensus.cc:504] T 89a143a5623e4ef38027381a56c95227 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.833668 11670 leader_election.cc:288] T 89a143a5623e4ef38027381a56c95227 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:20.833854 10687 raft_consensus.cc:2454] T c8522c08bdd94806b5c84f5a41ca5bf4 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:20.836306 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89a143a5623e4ef38027381a56c95227" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:20.840510 10686 raft_consensus.cc:2454] T 89a143a5623e4ef38027381a56c95227 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:20.815222 11627 raft_consensus.cc:504] T 197f61525b0c4f9da441d98701d231a3 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.841439 10775 leader_election.cc:302] T 89a143a5623e4ef38027381a56c95227 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:20.842336 10775 leader_election.cc:302] T c8522c08bdd94806b5c84f5a41ca5bf4 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:20.843097 11670 raft_consensus.cc:2781] T 89a143a5623e4ef38027381a56c95227 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 09:57:20.843472 11670 raft_consensus.cc:482] T 89a143a5623e4ef38027381a56c95227 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:20.843827 11670 raft_consensus.cc:3037] T 89a143a5623e4ef38027381a56c95227 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:57:20.845158 11627 leader_election.cc:288] T 197f61525b0c4f9da441d98701d231a3 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:20.834491 10983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8522c08bdd94806b5c84f5a41ca5bf4" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:20.836122 10980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8522c08bdd94806b5c84f5a41ca5bf4" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:20.836896 10978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89a143a5623e4ef38027381a56c95227" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:20.847543 11674 raft_consensus.cc:2781] T c8522c08bdd94806b5c84f5a41ca5bf4 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:20.848642 11674 raft_consensus.cc:686] T c8522c08bdd94806b5c84f5a41ca5bf4 P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Becoming Leader. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Running, Role: LEADER
I20260925 09:57:20.849347 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "197f61525b0c4f9da441d98701d231a3" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
W20260925 09:57:20.850639 10621 leader_election.cc:341] T 197f61525b0c4f9da441d98701d231a3 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:20.851267 10983 raft_consensus.cc:2454] T c8522c08bdd94806b5c84f5a41ca5bf4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
I20260925 09:57:20.851423 11674 consensus_queue.cc:237] T c8522c08bdd94806b5c84f5a41ca5bf4 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.831092 11085 tablet_bootstrap.cc:654] T 67ce2e3325214aa6bdc3347e9e8f82b0 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:20.858096 11700 raft_consensus.cc:686] T cf35f39bcd6a41128acb2a4936fcc316 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:20.858945 11700 consensus_queue.cc:237] T cf35f39bcd6a41128acb2a4936fcc316 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:20.860543 10619 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 56499.6us
I20260925 09:57:20.861698 10958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e3120347c8e4a3eab57d6562ae40e8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:20.861759 11616 raft_consensus.cc:482] T 359090244c4748e8a2d45e3c77e89498 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:20.862176 11616 raft_consensus.cc:504] T 359090244c4748e8a2d45e3c77e89498 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.864173 11616 leader_election.cc:288] T 359090244c4748e8a2d45e3c77e89498 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:20.866351 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "359090244c4748e8a2d45e3c77e89498" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:20.866956 10687 raft_consensus.cc:2454] T 359090244c4748e8a2d45e3c77e89498 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:20.868091 10775 leader_election.cc:302] T 359090244c4748e8a2d45e3c77e89498 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters:
W20260925 09:57:20.672454 10386 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.2:40761 (request call id 26) took 1599 ms (client timeout 1000 ms). Trace:
I20260925 09:57:20.868861 11082 tablet_bootstrap.cc:492] T 67ce2e3325214aa6bdc3347e9e8f82b0 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:20.594590 10393 rpcz_store.cc:276] 0925 09:57:19.514885 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:20.468638 (+953753us) service_pool.cc:227] Handling call
0925 09:57:20.593663 (+125025us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:20.497382 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:20.497697 (+ 315us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:20.497718 (+ 21us) write_op.cc:435] Acquired schema lock
0925 09:57:20.497730 (+ 12us) tablet.cc:660] Decoding operations
0925 09:57:20.497989 (+ 259us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:20.498021 (+ 32us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:20.498107 (+ 86us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:20.498276 (+ 169us) tablet.cc:699] Row locks acquired
0925 09:57:20.498289 (+ 13us) write_op.cc:260] PREPARE: finished
0925 09:57:20.498420 (+ 131us) write_op.cc:270] Start()
0925 09:57:20.498490 (+ 70us) write_op.cc:276] Timestamp: P: 1790330240498307 usec, L: 0
0925 09:57:20.498504 (+ 14us) op_driver.cc:331] REPLICATION: starting
0925 09:57:20.498901 (+ 397us) log.cc:842] Serialized 619 byte log entry
0925 09:57:20.576240 (+ 77339us) op_driver.cc:444] REPLICATION: finished
0925 09:57:20.578727 (+ 2487us) write_op.cc:301] APPLY: starting
0925 09:57:20.578773 (+ 46us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:20.578864 (+ 91us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:20.578963 (+ 99us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:20.579271 (+ 308us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:20.585145 (+ 5874us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:20.585186 (+ 41us) write_op.cc:312] APPLY: finished
0925 09:57:20.585366 (+ 180us) log.cc:842] Serialized 29 byte log entry
0925 09:57:20.585504 (+ 138us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:20.585689 (+ 185us) write_op.cc:454] Released schema lock
0925 09:57:20.585721 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:20.585878 (+ 157us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:20.585901 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2362,"apply.run_cpu_time_us":1614,"apply.run_wall_time_us":7332,"mutex_wait_us":24537,"num_ops":2,"prepare.queue_time_us":24619,"prepare.run_cpu_time_us":1590,"prepare.run_wall_time_us":2636,"replication_time_us":77673,"thread_start_us":10381,"threads_started":1}]]}
I20260925 09:57:20.868948 11616 raft_consensus.cc:2781] T 359090244c4748e8a2d45e3c77e89498 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 09:57:20.869241 11082 ts_tablet_manager.cc:1397] T 67ce2e3325214aa6bdc3347e9e8f82b0 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.118s user 0.008s sys 0.002s
I20260925 09:57:20.869248 11616 raft_consensus.cc:482] T 359090244c4748e8a2d45e3c77e89498 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:20.869524 11616 raft_consensus.cc:3037] T 359090244c4748e8a2d45e3c77e89498 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:57:20.759677 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet c16e096038084b2b9148390b6e773294 (table default.Table_70f0b47292de423097986b9a1c54f854 [id=8f2eaee8273640fb9ef73ccbe51782f1]) successfully deleted
W20260925 09:57:20.883095 10919 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 62101us
W20260925 09:57:20.883837 10919 leader_election.cc:341] T 977a59175e5b43eeb6f23b938afd00a3 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:20.884239 10919 leader_election.cc:302] T 977a59175e5b43eeb6f23b938afd00a3 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:20.884793 11670 raft_consensus.cc:504] T 89a143a5623e4ef38027381a56c95227 P c43461c4ee854ae1ac52b873ccada318 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.871228 11082 raft_consensus.cc:348] T 67ce2e3325214aa6bdc3347e9e8f82b0 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.886200 11082 raft_consensus.cc:374] T 67ce2e3325214aa6bdc3347e9e8f82b0 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:20.886457 11082 raft_consensus.cc:729] T 67ce2e3325214aa6bdc3347e9e8f82b0 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:20.887260 10688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89a143a5623e4ef38027381a56c95227" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:20.887939 10688 raft_consensus.cc:3037] T 89a143a5623e4ef38027381a56c95227 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:57:20.887051 11082 consensus_queue.cc:260] T 67ce2e3325214aa6bdc3347e9e8f82b0 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.898473 11082 ts_tablet_manager.cc:1428] T 67ce2e3325214aa6bdc3347e9e8f82b0 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.029s user 0.006s sys 0.000s
I20260925 09:57:20.899549 11082 tablet_bootstrap.cc:492] T 3706253e53d841aeaa43889ead13af32 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:20.905750 11082 tablet_bootstrap.cc:654] T 3706253e53d841aeaa43889ead13af32 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:20.868734 10386 rpcz_store.cc:269] 0925 09:57:19.072771 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:19.117149 (+ 44378us) service_pool.cc:227] Handling call
0925 09:57:19.168644 (+ 51495us) spinlock_profiling.cc:243] Waited 42.7 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2ad9c3e 00007fabf0610fb4 00007fabf0610f42 00007fabf0610e71 00007fabf0610dec 00007fabf06100a2 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:20.672383 (+1503739us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":11691,"reported_tablets":26,"spinlock_wait_cycles":98160896}
I20260925 09:57:20.908178 10386 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_077f38b3329348828ef80c0f4e0c972f" } modify_external_catalogs: true
I20260925 09:57:20.914314 11709 ts_tablet_manager.cc:1918] T 752105c1514f4c1db2a11b95b413b4f6 P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:20.914764 11709 log.cc:1196] T 752105c1514f4c1db2a11b95b413b4f6 P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/752105c1514f4c1db2a11b95b413b4f6
W20260925 09:57:20.919227 10919 leader_election.cc:341] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:20.919713 10919 leader_election.cc:302] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:20.919796 11616 raft_consensus.cc:504] T 359090244c4748e8a2d45e3c77e89498 P c43461c4ee854ae1ac52b873ccada318 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:20.886554 10918 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 69080.1us
I20260925 09:57:20.922259 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "359090244c4748e8a2d45e3c77e89498" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:20.922956 10687 raft_consensus.cc:3037] T 359090244c4748e8a2d45e3c77e89498 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:57:20.923229 11616 leader_election.cc:288] T 359090244c4748e8a2d45e3c77e89498 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:20.923170 10386 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_077f38b3329348828ef80c0f4e0c972f" } modify_external_catalogs: true
W20260925 09:57:20.908736 10379 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 149076us
I20260925 09:57:20.925806 11082 tablet_bootstrap.cc:492] T 3706253e53d841aeaa43889ead13af32 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:20.926450 11082 ts_tablet_manager.cc:1397] T 3706253e53d841aeaa43889ead13af32 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.027s user 0.014s sys 0.003s
I20260925 09:57:20.917951 11670 leader_election.cc:288] T 89a143a5623e4ef38027381a56c95227 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:20.927810 11709 ts_tablet_manager.cc:1939] T 752105c1514f4c1db2a11b95b413b4f6 P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:20.928561 11085 tablet_bootstrap.cc:492] T 67ce2e3325214aa6bdc3347e9e8f82b0 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:20.928923 11085 ts_tablet_manager.cc:1397] T 67ce2e3325214aa6bdc3347e9e8f82b0 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.115s user 0.009s sys 0.002s
I20260925 09:57:20.929718 11082 raft_consensus.cc:348] T 3706253e53d841aeaa43889ead13af32 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.930858 11082 raft_consensus.cc:374] T 3706253e53d841aeaa43889ead13af32 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:20.931262 11082 raft_consensus.cc:729] T 3706253e53d841aeaa43889ead13af32 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:20.936008 10392 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35140 (request call id 189) took 1420 ms. Trace:
W20260925 09:57:20.936697 10387 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 10657a81058241fc988d37e5ea7ff8cf has no consensus state
I20260925 09:57:20.936462 10392 rpcz_store.cc:276] 0925 09:57:19.515215 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:20.476830 (+961615us) service_pool.cc:227] Handling call
0925 09:57:20.569152 (+ 92322us) catalog_manager.cc:2075] Acquired catalog manager lock
0925 09:57:20.610769 (+ 41617us) catalog_manager.cc:2121] Created new table and tablet info
0925 09:57:20.935113 (+324344us) catalog_manager.cc:2181] Wrote table and tablets to system table
0925 09:57:20.935566 (+ 453us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0925 09:57:20.935926 (+ 360us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:20.612114 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:20.612476 (+ 362us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:20.612501 (+ 25us) write_op.cc:435] Acquired schema lock
0925 09:57:20.612514 (+ 13us) tablet.cc:660] Decoding operations
0925 09:57:20.612782 (+ 268us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:20.612824 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:20.612845 (+ 21us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:20.613040 (+ 195us) tablet.cc:699] Row locks acquired
0925 09:57:20.613054 (+ 14us) write_op.cc:260] PREPARE: finished
0925 09:57:20.613154 (+ 100us) write_op.cc:270] Start()
0925 09:57:20.613236 (+ 82us) write_op.cc:276] Timestamp: P: 1790330240613073 usec, L: 0
0925 09:57:20.613264 (+ 28us) op_driver.cc:331] REPLICATION: starting
0925 09:57:20.613800 (+ 536us) log.cc:842] Serialized 433 byte log entry
0925 09:57:20.932277 (+318477us) op_driver.cc:444] REPLICATION: finished
0925 09:57:20.932782 (+ 505us) write_op.cc:301] APPLY: starting
0925 09:57:20.932854 (+ 72us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:20.932994 (+ 140us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:20.933135 (+ 141us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:20.933641 (+ 506us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:20.933829 (+ 188us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 09:57:20.933893 (+ 64us) write_op.cc:312] APPLY: finished
0925 09:57:20.934104 (+ 211us) log.cc:842] Serialized 29 byte log entry
0925 09:57:20.934268 (+ 164us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:20.934362 (+ 94us) write_op.cc:454] Released schema lock
0925 09:57:20.934404 (+ 42us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:20.934720 (+ 316us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:20.934755 (+ 35us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":284,"apply.run_cpu_time_us":2189,"apply.run_wall_time_us":2188,"num_ops":2,"prepare.queue_time_us":136,"prepare.run_cpu_time_us":1952,"prepare.run_wall_time_us":1952,"raft.queue_time_us":860,"raft.run_cpu_time_us":12,"raft.run_wall_time_us":12,"replication_time_us":318920}]]}
I20260925 09:57:20.940948 10388 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 89a143a5623e4ef38027381a56c95227
I20260925 09:57:20.942211 10388 catalog_manager.cc:6565] ReplaceTablet: tablet 89a143a5623e4ef38027381a56c95227 deleted and replaced by tablet dba14a44a9de492886d629441606767e
I20260925 09:57:20.942533 10388 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:35140 (request call id 190) took 1335 ms. Trace:
I20260925 09:57:20.932104 11082 consensus_queue.cc:260] T 3706253e53d841aeaa43889ead13af32 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.944458 10961 tablet_service.cc:1514] Processing DeleteTablet for tablet 89a143a5623e4ef38027381a56c95227 with delete_type TABLET_DATA_DELETED (replaced by tablet dba14a44a9de492886d629441606767e) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:20.945734 11085 raft_consensus.cc:348] T 67ce2e3325214aa6bdc3347e9e8f82b0 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.942775 10388 rpcz_store.cc:276] 0925 09:57:19.607188 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:20.510783 (+903595us) service_pool.cc:227] Handling call
0925 09:57:20.942485 (+431702us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:20.623993 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:20.624438 (+ 445us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:20.624460 (+ 22us) write_op.cc:435] Acquired schema lock
0925 09:57:20.624474 (+ 14us) tablet.cc:660] Decoding operations
0925 09:57:20.624774 (+ 300us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:20.624813 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:20.624837 (+ 24us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:20.625049 (+ 212us) tablet.cc:699] Row locks acquired
0925 09:57:20.625064 (+ 15us) write_op.cc:260] PREPARE: finished
0925 09:57:20.625162 (+ 98us) write_op.cc:270] Start()
0925 09:57:20.625238 (+ 76us) write_op.cc:276] Timestamp: P: 1790330240625080 usec, L: 0
0925 09:57:20.625269 (+ 31us) op_driver.cc:331] REPLICATION: starting
0925 09:57:20.625735 (+ 466us) log.cc:842] Serialized 619 byte log entry
0925 09:57:20.932683 (+306948us) op_driver.cc:444] REPLICATION: finished
0925 09:57:20.934743 (+ 2060us) write_op.cc:301] APPLY: starting
0925 09:57:20.934812 (+ 69us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:20.934932 (+ 120us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:20.935047 (+ 115us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:20.939747 (+ 4700us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:20.939954 (+ 207us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:20.939994 (+ 40us) write_op.cc:312] APPLY: finished
0925 09:57:20.940156 (+ 162us) log.cc:842] Serialized 29 byte log entry
0925 09:57:20.940341 (+ 185us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:20.940399 (+ 58us) write_op.cc:454] Released schema lock
0925 09:57:20.940428 (+ 29us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1943,"apply.run_cpu_time_us":1824,"apply.run_wall_time_us":10191,"num_ops":2,"prepare.queue_time_us":6711,"prepare.run_cpu_time_us":1976,"prepare.run_wall_time_us":6397,"raft.queue_time_us":2061,"raft.run_cpu_time_us":15,"raft.run_wall_time_us":15,"replication_time_us":307342,"threads_started":1}]]}
I20260925 09:57:20.948879 10658 tablet_service.cc:1514] Processing DeleteTablet for tablet 89a143a5623e4ef38027381a56c95227 with delete_type TABLET_DATA_DELETED (replaced by tablet dba14a44a9de492886d629441606767e) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:20.946381 11085 raft_consensus.cc:374] T 67ce2e3325214aa6bdc3347e9e8f82b0 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:20.952994 11085 raft_consensus.cc:729] T 67ce2e3325214aa6bdc3347e9e8f82b0 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:20.954033 11082 ts_tablet_manager.cc:1428] T 3706253e53d841aeaa43889ead13af32 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.027s user 0.007s sys 0.000s
I20260925 09:57:20.954864 11082 tablet_bootstrap.cc:492] T 93e1b5837aff48d7907890c40dfcd0d0 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:20.961373 11082 tablet_bootstrap.cc:654] T 93e1b5837aff48d7907890c40dfcd0d0 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:20.956024 11085 consensus_queue.cc:260] T 67ce2e3325214aa6bdc3347e9e8f82b0 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.962925 10807 tablet_service.cc:1514] Processing DeleteTablet for tablet 89a143a5623e4ef38027381a56c95227 with delete_type TABLET_DATA_DELETED (replaced by tablet dba14a44a9de492886d629441606767e) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:20.968830 11085 ts_tablet_manager.cc:1428] T 67ce2e3325214aa6bdc3347e9e8f82b0 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.040s user 0.003s sys 0.002s
I20260925 09:57:20.973301 11085 tablet_bootstrap.cc:492] T cfa619b699004882a82976f0ef8204ad P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:20.978314 11085 tablet_bootstrap.cc:654] T cfa619b699004882a82976f0ef8204ad P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:20.978220 11700 raft_consensus.cc:482] T 3706253e53d841aeaa43889ead13af32 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:20.978719 11700 raft_consensus.cc:504] T 3706253e53d841aeaa43889ead13af32 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.910535 10393 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_cd9d59504d7a408ba2f9b149cbe1c5fa" } modify_external_catalogs: true
I20260925 09:57:20.980201 10393 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_cd9d59504d7a408ba2f9b149cbe1c5fa" } modify_external_catalogs: true
I20260925 09:57:20.982990 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3706253e53d841aeaa43889ead13af32" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:20.984357 10621 leader_election.cc:341] T 3706253e53d841aeaa43889ead13af32 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:20.984794 11700 leader_election.cc:288] T 3706253e53d841aeaa43889ead13af32 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:20.985800 11082 tablet_bootstrap.cc:492] T 93e1b5837aff48d7907890c40dfcd0d0 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:20.986320 11082 ts_tablet_manager.cc:1397] T 93e1b5837aff48d7907890c40dfcd0d0 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.032s user 0.012s sys 0.001s
I20260925 09:57:20.987361 11709 tablet_replica.cc:330] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:20.988549 10687 raft_consensus.cc:2454] T 359090244c4748e8a2d45e3c77e89498 P b8850818e2ee42a082802d0095c08d21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 2.
I20260925 09:57:20.989017 11709 raft_consensus.cc:2229] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:20.989820 11709 raft_consensus.cc:2258] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:20.989850 10775 leader_election.cc:302] T 359090244c4748e8a2d45e3c77e89498 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:20.991357 11674 raft_consensus.cc:2781] T 359090244c4748e8a2d45e3c77e89498 P c43461c4ee854ae1ac52b873ccada318 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 09:57:20.991824 11674 raft_consensus.cc:686] T 359090244c4748e8a2d45e3c77e89498 P c43461c4ee854ae1ac52b873ccada318 [term 2 LEADER]: Becoming Leader. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Running, Role: LEADER
I20260925 09:57:20.989794 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 752105c1514f4c1db2a11b95b413b4f6 (table default.Table_8c5b820f0db34e3bbea2d404afb1909a [id=fc979ab5541c41ef82a95e11f5a455ab]) successfully deleted
W20260925 09:57:20.992467 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet c07c6205d1284a6fbb369c010a226da7 because the tablet was not found. No further retry: Not found: Tablet not found: c07c6205d1284a6fbb369c010a226da7
I20260925 09:57:20.992658 11674 consensus_queue.cc:237] T 359090244c4748e8a2d45e3c77e89498 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.996312 11709 ts_tablet_manager.cc:1905] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:20.988641 11082 raft_consensus.cc:348] T 93e1b5837aff48d7907890c40dfcd0d0 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:20.999713 11082 raft_consensus.cc:374] T 93e1b5837aff48d7907890c40dfcd0d0 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:21.000058 11082 raft_consensus.cc:729] T 93e1b5837aff48d7907890c40dfcd0d0 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:21.000648 11579 ts_tablet_manager.cc:1918] T c07c6205d1284a6fbb369c010a226da7 P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:21.001012 11579 log.cc:1196] T c07c6205d1284a6fbb369c010a226da7 P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/c07c6205d1284a6fbb369c010a226da7
I20260925 09:57:21.000723 11082 consensus_queue.cc:260] T 93e1b5837aff48d7907890c40dfcd0d0 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:21.001748 11579 ts_tablet_manager.cc:1939] T c07c6205d1284a6fbb369c010a226da7 P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:21.014228 11579 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 158) took 1969 ms. Trace:
I20260925 09:57:21.014114 10392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_0f7925265a1741aa97617300030475a1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:21.014498 11579 rpcz_store.cc:276] 0925 09:57:19.045018 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:19.045318 (+ 300us) service_pool.cc:227] Handling call
0925 09:57:20.743213 (+1697895us) ts_tablet_manager.cc:1145] Deleting tablet c07c6205d1284a6fbb369c010a226da7
0925 09:57:20.743260 (+ 47us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:20.941245 (+197985us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:21.000153 (+ 58908us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:21.014175 (+ 14022us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1659962}
I20260925 09:57:21.017865 10387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_eb46cc92849a4a6db19755c3ea1268a3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:21.019236 10387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eb46cc92849a4a6db19755c3ea1268a3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:21.019620 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet c07c6205d1284a6fbb369c010a226da7 (table default.Table_8e186a99ccba425b9b588041d2364a40 [id=2d6bc0e2aaee4b1483b01fc678505bb0]) successfully deleted
W20260925 09:57:21.020203 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 1969ac471b6f471ebe8e048a96c22837 because the tablet was not found. No further retry: Not found: Tablet not found: 1969ac471b6f471ebe8e048a96c22837
W20260925 09:57:21.020730 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 833fac37485141f6a18104eaea907b0e because the tablet was not found. No further retry: Not found: Tablet not found: 833fac37485141f6a18104eaea907b0e
W20260925 09:57:21.022352 10392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0f7925265a1741aa97617300030475a1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:21.028479 11579 tablet_replica.cc:330] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:21.029238 11579 raft_consensus.cc:2229] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:21.033253 11082 ts_tablet_manager.cc:1428] T 93e1b5837aff48d7907890c40dfcd0d0 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.047s user 0.001s sys 0.004s
I20260925 09:57:21.034215 11082 tablet_bootstrap.cc:492] T ec1b78e61e8d46cb8b87f60786059658 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
W20260925 09:57:21.038332 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 91c09118b43b41eca8f7040323ba76f7 because the tablet was not found. No further retry: Not found: Tablet not found: 91c09118b43b41eca8f7040323ba76f7
W20260925 09:57:21.039276 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 2d55154ceb064f22bd6f37c8ae5dce74 because the tablet was not found. No further retry: Not found: Tablet not found: 2d55154ceb064f22bd6f37c8ae5dce74
I20260925 09:57:21.040026 11579 raft_consensus.cc:2258] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:21.042249 11579 ts_tablet_manager.cc:1905] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:21.053700 11084 tablet_bootstrap.cc:654] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:21.061432 11700 raft_consensus.cc:482] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:21.056885 11709 ts_tablet_manager.cc:1918] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:21.061900 11700 raft_consensus.cc:504] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:21.063850 11700 leader_election.cc:288] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:21.065536 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "229aaa1fc7b84fb887a8c0d8a53bf0ae" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:21.067503 10621 leader_election.cc:341] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:21.068367 11709 log.cc:1196] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/41df75fe4b8e4856bdbb5ead3cf4fd24
I20260925 09:57:21.068511 10688 raft_consensus.cc:2454] T 89a143a5623e4ef38027381a56c95227 P b8850818e2ee42a082802d0095c08d21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 2.
I20260925 09:57:21.070619 11709 ts_tablet_manager.cc:1939] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:20.855307 10980 raft_consensus.cc:3037] T c8522c08bdd94806b5c84f5a41ca5bf4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:21.072638 10775 leader_election.cc:302] T 89a143a5623e4ef38027381a56c95227 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:21.043679 11655 raft_consensus.cc:2726] T 977a59175e5b43eeb6f23b938afd00a3 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:21.073824 11674 raft_consensus.cc:2781] T 89a143a5623e4ef38027381a56c95227 P c43461c4ee854ae1ac52b873ccada318 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 09:57:21.074239 11674 raft_consensus.cc:686] T 89a143a5623e4ef38027381a56c95227 P c43461c4ee854ae1ac52b873ccada318 [term 2 LEADER]: Becoming Leader. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Running, Role: LEADER
I20260925 09:57:21.056191 11605 raft_consensus.cc:2726] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:21.075011 11674 consensus_queue.cc:237] T 89a143a5623e4ef38027381a56c95227 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:21.074798 10982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "359090244c4748e8a2d45e3c77e89498" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:21.078698 11725 tablet.cc:1722] T ef6fefaa76d44fe798201aab2f21b9c7 P be97ab5d75934a648d4ab76341193ec0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 09:57:21.072557 11085 tablet_bootstrap.cc:492] T cfa619b699004882a82976f0ef8204ad P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:21.085093 11085 ts_tablet_manager.cc:1397] T cfa619b699004882a82976f0ef8204ad P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.112s user 0.002s sys 0.008s
I20260925 09:57:21.085134 11727 tablet_replica.cc:330] T 4be3f6e81ddf418fabc7b694a2e62f35 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:21.087324 11670 consensus_queue.cc:1035] T 42ec275c905343c4be1b72dc2bd0f6e3 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:21.091933 11700 raft_consensus.cc:482] T 93e1b5837aff48d7907890c40dfcd0d0 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:21.092386 11700 raft_consensus.cc:504] T 93e1b5837aff48d7907890c40dfcd0d0 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:21.094480 11634 raft_consensus.cc:482] T cfa619b699004882a82976f0ef8204ad P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:21.094954 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93e1b5837aff48d7907890c40dfcd0d0" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:21.094892 11634 raft_consensus.cc:504] T cfa619b699004882a82976f0ef8204ad P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:21.096618 11634 leader_election.cc:288] T cfa619b699004882a82976f0ef8204ad P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
W20260925 09:57:21.097513 10621 leader_election.cc:341] T 93e1b5837aff48d7907890c40dfcd0d0 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:21.097919 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa619b699004882a82976f0ef8204ad" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:21.099099 10621 leader_election.cc:341] T cfa619b699004882a82976f0ef8204ad P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:21.096626 11700 leader_election.cc:288] T 93e1b5837aff48d7907890c40dfcd0d0 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:21.100872 10974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93e1b5837aff48d7907890c40dfcd0d0" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:21.101508 10974 raft_consensus.cc:2454] T 93e1b5837aff48d7907890c40dfcd0d0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
W20260925 09:57:21.102679 10379 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.62:34455, user_credentials={real_user=slave}} blocked reactor thread for 62853.3us
I20260925 09:57:21.103915 10619 leader_election.cc:302] T 93e1b5837aff48d7907890c40dfcd0d0 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:21.104910 11700 raft_consensus.cc:2781] T 93e1b5837aff48d7907890c40dfcd0d0 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:21.104573 11085 raft_consensus.cc:348] T cfa619b699004882a82976f0ef8204ad P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:21.105355 11085 raft_consensus.cc:374] T cfa619b699004882a82976f0ef8204ad P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:21.105253 11700 raft_consensus.cc:482] T 93e1b5837aff48d7907890c40dfcd0d0 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:21.105604 11085 raft_consensus.cc:729] T cfa619b699004882a82976f0ef8204ad P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:21.105795 11700 raft_consensus.cc:3037] T 93e1b5837aff48d7907890c40dfcd0d0 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:21.107257 10973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa619b699004882a82976f0ef8204ad" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:21.111850 10973 raft_consensus.cc:2454] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:21.105435 11565 consensus_queue.cc:1035] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260925 09:57:21.116582 11700 raft_consensus.cc:504] T 93e1b5837aff48d7907890c40dfcd0d0 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:21.123584 11716 leader_election.cc:288] T c588b0c05a1a4f3ea545066183e76502 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:21.128685 11085 consensus_queue.cc:260] T cfa619b699004882a82976f0ef8204ad P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:21.119402 10619 leader_election.cc:302] T cfa619b699004882a82976f0ef8204ad P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:21.133747 11579 ts_tablet_manager.cc:1918] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:21.134109 11579 log.cc:1196] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/41df75fe4b8e4856bdbb5ead3cf4fd24
I20260925 09:57:21.134274 11634 raft_consensus.cc:2781] T cfa619b699004882a82976f0ef8204ad P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:21.134608 11634 raft_consensus.cc:482] T cfa619b699004882a82976f0ef8204ad P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:21.134800 11579 ts_tablet_manager.cc:1939] T 41df75fe4b8e4856bdbb5ead3cf4fd24 P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:21.134898 11634 raft_consensus.cc:3037] T cfa619b699004882a82976f0ef8204ad P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:21.141789 10390 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_3a00773e32a34a41bead0b4116e4c6df" } new_table_name: "default.Table_6eff7b87b3b74ca4bab719140667a87b" modify_external_catalogs: true
I20260925 09:57:21.143410 10980 raft_consensus.cc:2454] T c8522c08bdd94806b5c84f5a41ca5bf4 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:21.147874 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 41df75fe4b8e4856bdbb5ead3cf4fd24 (table default.Table_0507f47042b9445abaf8671c979f8994 [id=55860c5dba9f4c818cfa6d523041ec5f]) successfully deleted
W20260925 09:57:21.149798 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet c16e096038084b2b9148390b6e773294 because the tablet was not found. No further retry: Not found: Tablet not found: c16e096038084b2b9148390b6e773294
I20260925 09:57:21.169255 11085 ts_tablet_manager.cc:1428] T cfa619b699004882a82976f0ef8204ad P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.084s user 0.004s sys 0.001s
W20260925 09:57:21.175608 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 4c4e8790282a49fd849a34ca11bef5bf because the tablet was not found. No further retry: Not found: Tablet not found: 4c4e8790282a49fd849a34ca11bef5bf
I20260925 09:57:21.182739 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93e1b5837aff48d7907890c40dfcd0d0" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
W20260925 09:57:21.183893 10621 leader_election.cc:341] T 93e1b5837aff48d7907890c40dfcd0d0 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:21.186316 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "197f61525b0c4f9da441d98701d231a3" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:21.186558 10959 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53560 (request call id 165) took 1052 ms. Trace:
I20260925 09:57:21.186842 10984 raft_consensus.cc:3037] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:21.186842 10959 rpcz_store.cc:276] 0925 09:57:20.133694 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:20.133807 (+ 113us) service_pool.cc:227] Handling call
0925 09:57:20.134912 (+ 1105us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0925 09:57:20.134988 (+ 76us) ts_tablet_manager.cc:676] Creating new metadata...
0925 09:57:20.560880 (+425892us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:20.917421 (+356541us) tablet_replica.cc:205] Creating consensus instance
0925 09:57:21.186510 (+269089us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260925 09:57:21.192337 10975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "229aaa1fc7b84fb887a8c0d8a53bf0ae" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:21.192638 11725 tablet.cc:1722] T 0a2bbe141e564a96be05de86f2993234 P be97ab5d75934a648d4ab76341193ec0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260925 09:57:21.193033 10975 raft_consensus.cc:2454] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:21.043311 11082 tablet_bootstrap.cc:654] T ec1b78e61e8d46cb8b87f60786059658 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:21.216039 10984 raft_consensus.cc:2454] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:21.217027 10619 leader_election.cc:302] T 197f61525b0c4f9da441d98701d231a3 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
W20260925 09:57:21.218560 10379 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.62:34455, user_credentials={real_user=slave}} blocked reactor thread for 46996.5us
W20260925 09:57:21.220542 10918 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 299411us
I20260925 09:57:21.221413 11605 consensus_queue.cc:1035] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:21.221414 11713 raft_consensus.cc:482] T ec1b78e61e8d46cb8b87f60786059658 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:21.222906 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 41df75fe4b8e4856bdbb5ead3cf4fd24 (table default.Table_0507f47042b9445abaf8671c979f8994 [id=55860c5dba9f4c818cfa6d523041ec5f]) successfully deleted
I20260925 09:57:21.225494 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe91b39f91274a7ca61e9618d8cbb475" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:21.224644 10919 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 303783us
I20260925 09:57:21.174273 11579 tablet_replica.cc:330] T 752105c1514f4c1db2a11b95b413b4f6 P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:21.233412 11700 leader_election.cc:288] T 93e1b5837aff48d7907890c40dfcd0d0 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:21.233963 11700 raft_consensus.cc:2781] T 197f61525b0c4f9da441d98701d231a3 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:21.234031 10619 leader_election.cc:302] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
W20260925 09:57:21.234344 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 91c09118b43b41eca8f7040323ba76f7 because the tablet was not found. No further retry: Not found: Tablet not found: 91c09118b43b41eca8f7040323ba76f7
I20260925 09:57:21.234401 11700 raft_consensus.cc:686] T 197f61525b0c4f9da441d98701d231a3 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:21.235242 11729 raft_consensus.cc:2781] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:21.236066 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93e1b5837aff48d7907890c40dfcd0d0" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:21.236570 10984 raft_consensus.cc:3037] T 93e1b5837aff48d7907890c40dfcd0d0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:21.220615 11716 raft_consensus.cc:482] T 3706253e53d841aeaa43889ead13af32 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:21.242882 11085 tablet_bootstrap.cc:492] T c588b0c05a1a4f3ea545066183e76502 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:21.222659 11713 raft_consensus.cc:504] T ec1b78e61e8d46cb8b87f60786059658 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:21.245316 10984 raft_consensus.cc:2454] T 93e1b5837aff48d7907890c40dfcd0d0 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:21.246093 11713 leader_election.cc:288] T ec1b78e61e8d46cb8b87f60786059658 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:21.246558 11713 raft_consensus.cc:482] T 3d652a76d96f4dbeafde15f3c16534a6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:21.246946 11713 raft_consensus.cc:504] T 3d652a76d96f4dbeafde15f3c16534a6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:21.248190 10619 leader_election.cc:302] T 93e1b5837aff48d7907890c40dfcd0d0 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:21.249017 11713 leader_election.cc:288] T 3d652a76d96f4dbeafde15f3c16534a6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:21.250842 10835 raft_consensus.cc:1226] T 359090244c4748e8a2d45e3c77e89498 P c43461c4ee854ae1ac52b873ccada318 [term 2 LEADER]: Rejecting Update request from peer be97ab5d75934a648d4ab76341193ec0 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
W20260925 09:57:21.252916 10772 consensus_peers.cc:487] T 4be3f6e81ddf418fabc7b694a2e62f35 P c43461c4ee854ae1ac52b873ccada318 -> Peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Couldn't send request to peer be97ab5d75934a648d4ab76341193ec0. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:21.253722 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c588b0c05a1a4f3ea545066183e76502" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:21.256220 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec1b78e61e8d46cb8b87f60786059658" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:21.259670 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d652a76d96f4dbeafde15f3c16534a6" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:21.249536 11740 tablet.cc:1722] T 28d60531970a4ab58ce5ef99cb3a96bf P be97ab5d75934a648d4ab76341193ec0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 09:57:21.263106 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe91b39f91274a7ca61e9618d8cbb475" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:21.264117 10688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8522c08bdd94806b5c84f5a41ca5bf4" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:21.264832 10688 raft_consensus.cc:2379] T c8522c08bdd94806b5c84f5a41ca5bf4 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be97ab5d75934a648d4ab76341193ec0 in current term 1: Already voted for candidate c43461c4ee854ae1ac52b873ccada318 in this term.
I20260925 09:57:21.250804 11084 tablet_bootstrap.cc:492] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:21.265548 10682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c588b0c05a1a4f3ea545066183e76502" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:21.265830 11084 ts_tablet_manager.cc:1397] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.469s user 0.014s sys 0.015s
I20260925 09:57:21.251350 10978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3706253e53d841aeaa43889ead13af32" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:21.240120 11716 raft_consensus.cc:504] T 3706253e53d841aeaa43889ead13af32 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:21.266090 10682 raft_consensus.cc:3037] T c588b0c05a1a4f3ea545066183e76502 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:21.266268 11579 raft_consensus.cc:2229] T 752105c1514f4c1db2a11b95b413b4f6 P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:21.260941 11605 raft_consensus.cc:482] T 6161774f61ae48ab9764a05f2188a80b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:21.268891 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "359090244c4748e8a2d45e3c77e89498" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:21.270829 11627 raft_consensus.cc:2781] T 93e1b5837aff48d7907890c40dfcd0d0 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:21.235632 11729 raft_consensus.cc:482] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:21.272956 11729 raft_consensus.cc:3037] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:21.273591 11716 leader_election.cc:288] T 3706253e53d841aeaa43889ead13af32 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:21.273998 10978 raft_consensus.cc:2454] T 3706253e53d841aeaa43889ead13af32 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:21.276304 10619 leader_election.cc:302] T 3706253e53d841aeaa43889ead13af32 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:21.277052 11730 raft_consensus.cc:2781] T 3706253e53d841aeaa43889ead13af32 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:21.277361 11730 raft_consensus.cc:482] T 3706253e53d841aeaa43889ead13af32 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:21.270236 10977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89a143a5623e4ef38027381a56c95227" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:21.277673 11730 raft_consensus.cc:3037] T 3706253e53d841aeaa43889ead13af32 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:57:21.283725 10377 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 10657a81058241fc988d37e5ea7ff8cf (table default.Table_64ba123898174e72a6884dbb9ad07de3 [id=7ee526720b2442119e0fc487b7c4fcd4], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 10657a81058241fc988d37e5ea7ff8cf has no consensus state
I20260925 09:57:21.273420 11579 raft_consensus.cc:2258] T 752105c1514f4c1db2a11b95b413b4f6 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:21.269644 11605 raft_consensus.cc:504] T 6161774f61ae48ab9764a05f2188a80b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:21.289012 11605 leader_election.cc:288] T 6161774f61ae48ab9764a05f2188a80b P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:21.289469 11742 consensus_queue.cc:1035] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.046s
I20260925 09:57:21.186170 11634 raft_consensus.cc:504] T cfa619b699004882a82976f0ef8204ad P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:21.305991 11634 leader_election.cc:288] T cfa619b699004882a82976f0ef8204ad P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:21.307603 10977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa619b699004882a82976f0ef8204ad" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:21.308194 10977 raft_consensus.cc:3037] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:21.235184 11700 consensus_queue.cc:237] T 197f61525b0c4f9da441d98701d231a3 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:21.325604 10977 raft_consensus.cc:2454] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:21.329273 10619 leader_election.cc:302] T cfa619b699004882a82976f0ef8204ad P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:21.319116 10682 raft_consensus.cc:2454] T c588b0c05a1a4f3ea545066183e76502 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be97ab5d75934a648d4ab76341193ec0 in term 1.
I20260925 09:57:21.330895 11634 raft_consensus.cc:2781] T cfa619b699004882a82976f0ef8204ad P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:21.332172 11634 raft_consensus.cc:686] T cfa619b699004882a82976f0ef8204ad P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:21.333062 11634 consensus_queue.cc:237] T cfa619b699004882a82976f0ef8204ad P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:21.337141 11727 raft_consensus.cc:2229] T 4be3f6e81ddf418fabc7b694a2e62f35 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:21.350998 10393 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 5b4352e6aa324934bd95836d396cc9ed
I20260925 09:57:21.353953 10393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_836d63b3fbed4250b0757ef2d432a704"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:21.356211 10393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_836d63b3fbed4250b0757ef2d432a704 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:21.356528 11730 raft_consensus.cc:504] T 3706253e53d841aeaa43889ead13af32 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:21.359208 10658 tablet_service.cc:1467] Processing CreateTablet for tablet 2f07dc9548364870923165f4ea524310 (DEFAULT_TABLE table=default.Table_e613b02df28b46f2b371344af4971233 [id=a988264fe43c4eb88a84f8b6359e77a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:21.359668 10807 tablet_service.cc:1467] Processing CreateTablet for tablet 2f07dc9548364870923165f4ea524310 (DEFAULT_TABLE table=default.Table_e613b02df28b46f2b371344af4971233 [id=a988264fe43c4eb88a84f8b6359e77a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:21.360177 10658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f07dc9548364870923165f4ea524310. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:21.360656 10807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f07dc9548364870923165f4ea524310. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:21.376336 11729 raft_consensus.cc:504] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:21.377489 10958 tablet_service.cc:1467] Processing CreateTablet for tablet 2f07dc9548364870923165f4ea524310 (DEFAULT_TABLE table=default.Table_e613b02df28b46f2b371344af4971233 [id=a988264fe43c4eb88a84f8b6359e77a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:21.378520 10958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f07dc9548364870923165f4ea524310. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:21.380117 11579 ts_tablet_manager.cc:1905] T 752105c1514f4c1db2a11b95b413b4f6 P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:21.380380 10978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "229aaa1fc7b84fb887a8c0d8a53bf0ae" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:21.380913 10978 raft_consensus.cc:3037] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:21.391575 10978 raft_consensus.cc:2454] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:21.392534 10619 leader_election.cc:302] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:21.374550 10392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_ae6c60e2023f4a48b6690dcd81dd81b5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:21.397644 10392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ae6c60e2023f4a48b6690dcd81dd81b5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:21.396440 11760 raft_consensus.cc:2781] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:21.398248 11760 raft_consensus.cc:686] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:21.399037 11760 consensus_queue.cc:237] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:21.409268 11709 tablet_replica.cc:330] T 4be3f6e81ddf418fabc7b694a2e62f35 P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:21.410395 11709 raft_consensus.cc:2229] T 4be3f6e81ddf418fabc7b694a2e62f35 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:21.410979 11709 raft_consensus.cc:2258] T 4be3f6e81ddf418fabc7b694a2e62f35 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:21.412976 11709 ts_tablet_manager.cc:1905] T 4be3f6e81ddf418fabc7b694a2e62f35 P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:21.415073 11655 consensus_queue.cc:1035] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 09:57:21.416767 10918 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 155224us
I20260925 09:57:21.416973 11084 raft_consensus.cc:348] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:21.417457 10918 leader_election.cc:341] T fe91b39f91274a7ca61e9618d8cbb475 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:21.417672 11084 raft_consensus.cc:374] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:21.417944 11084 raft_consensus.cc:729] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:21.418473 11084 consensus_queue.cc:260] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:21.420375 11084 ts_tablet_manager.cc:1428] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.154s user 0.005s sys 0.000s
I20260925 09:57:21.421150 11084 tablet_bootstrap.cc:492] T 417058493c6547c69e72347cef373292 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:21.423725 11727 raft_consensus.cc:2258] T 4be3f6e81ddf418fabc7b694a2e62f35 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:21.426096 11727 ts_tablet_manager.cc:1905] T 4be3f6e81ddf418fabc7b694a2e62f35 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:21.439615 11655 consensus_queue.cc:1046] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:21.440454 11655 consensus_queue.cc:1225] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [LEADER]: Peer c43461c4ee854ae1ac52b873ccada318 log is divergent from this leader: its last log entry 2.1 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20260925 09:57:21.441720 11751 raft_consensus.cc:3032] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [term 1 LEADER]: Stepping down as leader of term 1
I20260925 09:57:21.442166 11751 raft_consensus.cc:729] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Running, Role: LEADER
I20260925 09:57:21.443091 11751 consensus_queue.cc:260] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:21.480894 11085 tablet_bootstrap.cc:654] T c588b0c05a1a4f3ea545066183e76502 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:21.499315 11742 consensus_queue.cc:1035] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 09:57:21.500845 10919 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 52320.3us
W20260925 09:57:21.501684 10919 leader_election.cc:341] T fe91b39f91274a7ca61e9618d8cbb475 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:21.502121 10919 leader_election.cc:302] T fe91b39f91274a7ca61e9618d8cbb475 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:21.502766 11742 raft_consensus.cc:2726] T fe91b39f91274a7ca61e9618d8cbb475 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:21.503716 10919 leader_election.cc:302] T c8522c08bdd94806b5c84f5a41ca5bf4 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:21.504365 11742 raft_consensus.cc:2726] T c8522c08bdd94806b5c84f5a41ca5bf4 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:21.505294 10919 leader_election.cc:302] T c588b0c05a1a4f3ea545066183e76502 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:21.449677 10688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d652a76d96f4dbeafde15f3c16534a6" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:21.451308 10687 raft_consensus.cc:1226] T 359090244c4748e8a2d45e3c77e89498 P b8850818e2ee42a082802d0095c08d21 [term 2 FOLLOWER]: Rejecting Update request from peer be97ab5d75934a648d4ab76341193ec0 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260925 09:57:21.452590 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3706253e53d841aeaa43889ead13af32" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:21.523001 10686 raft_consensus.cc:2379] T 3706253e53d841aeaa43889ead13af32 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be97ab5d75934a648d4ab76341193ec0 in current term 1: Already voted for candidate b8850818e2ee42a082802d0095c08d21 in this term.
I20260925 09:57:21.454301 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6161774f61ae48ab9764a05f2188a80b" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:21.456243 10679 raft_consensus.cc:1226] T 89a143a5623e4ef38027381a56c95227 P b8850818e2ee42a082802d0095c08d21 [term 2 FOLLOWER]: Rejecting Update request from peer be97ab5d75934a648d4ab76341193ec0 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260925 09:57:21.457613 10682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec1b78e61e8d46cb8b87f60786059658" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:21.476615 11729 leader_election.cc:288] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:21.479882 11754 tablet.cc:1722] T ef6fefaa76d44fe798201aab2f21b9c7 P b8850818e2ee42a082802d0095c08d21: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 09:57:21.498548 11627 raft_consensus.cc:686] T 93e1b5837aff48d7907890c40dfcd0d0 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:21.531319 11627 consensus_queue.cc:237] T 93e1b5837aff48d7907890c40dfcd0d0 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:21.506239 11730 leader_election.cc:288] T 3706253e53d841aeaa43889ead13af32 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:21.511994 11082 tablet_bootstrap.cc:492] T ec1b78e61e8d46cb8b87f60786059658 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:21.533807 11082 ts_tablet_manager.cc:1397] T ec1b78e61e8d46cb8b87f60786059658 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.500s user 0.009s sys 0.004s
I20260925 09:57:21.506783 10983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3706253e53d841aeaa43889ead13af32" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:21.544143 10983 raft_consensus.cc:3037] T 3706253e53d841aeaa43889ead13af32 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:21.544313 11655 raft_consensus.cc:2781] T c588b0c05a1a4f3ea545066183e76502 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:21.544948 11655 raft_consensus.cc:686] T c588b0c05a1a4f3ea545066183e76502 P be97ab5d75934a648d4ab76341193ec0 [term 1 LEADER]: Becoming Leader. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Running, Role: LEADER
I20260925 09:57:21.545759 11655 consensus_queue.cc:237] T c588b0c05a1a4f3ea545066183e76502 P be97ab5d75934a648d4ab76341193ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:21.509514 11751 raft_consensus.cc:3037] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:57:21.553711 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa619b699004882a82976f0ef8204ad" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:21.554497 10835 raft_consensus.cc:3037] T cfa619b699004882a82976f0ef8204ad P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:21.555850 10983 raft_consensus.cc:2454] T 3706253e53d841aeaa43889ead13af32 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:21.556977 10619 leader_election.cc:302] T 3706253e53d841aeaa43889ead13af32 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:21.561933 10392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_f7e141887342416f9c4468ca211d74e5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:21.563800 10392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f7e141887342416f9c4468ca211d74e5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:21.563630 10835 raft_consensus.cc:2454] T cfa619b699004882a82976f0ef8204ad P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:21.564838 11730 raft_consensus.cc:2781] T 3706253e53d841aeaa43889ead13af32 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:21.565318 11730 raft_consensus.cc:686] T 3706253e53d841aeaa43889ead13af32 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:21.566206 11730 consensus_queue.cc:237] T 3706253e53d841aeaa43889ead13af32 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:21.509923 10918 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 74150.5us
W20260925 09:57:21.569330 10918 leader_election.cc:341] T c588b0c05a1a4f3ea545066183e76502 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:21.569957 10918 leader_election.cc:341] T ec1b78e61e8d46cb8b87f60786059658 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:21.570502 10918 leader_election.cc:341] T 3d652a76d96f4dbeafde15f3c16534a6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:21.512370 11084 tablet_bootstrap.cc:654] T 417058493c6547c69e72347cef373292 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:21.577734 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3706253e53d841aeaa43889ead13af32" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:21.581872 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "229aaa1fc7b84fb887a8c0d8a53bf0ae" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:21.583676 11085 tablet_bootstrap.cc:492] T c588b0c05a1a4f3ea545066183e76502 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:21.584069 11085 ts_tablet_manager.cc:1397] T c588b0c05a1a4f3ea545066183e76502 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.342s user 0.015s sys 0.003s
I20260925 09:57:21.587090 10831 raft_consensus.cc:1226] T 89a143a5623e4ef38027381a56c95227 P c43461c4ee854ae1ac52b873ccada318 [term 2 LEADER]: Rejecting Update request from peer be97ab5d75934a648d4ab76341193ec0 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260925 09:57:21.591552 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3706253e53d841aeaa43889ead13af32" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:21.596585 10919 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 91336.1us
I20260925 09:57:21.604832 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6161774f61ae48ab9764a05f2188a80b" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:21.610954 10380 catalog_manager.cc:4914] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): alter failed for tablet c07c6205d1284a6fbb369c010a226da7 (table default.Table_8e186a99ccba425b9b588041d2364a40 [id=2d6bc0e2aaee4b1483b01fc678505bb0]),no further retry: Not found: Tablet not found: c07c6205d1284a6fbb369c010a226da7
I20260925 09:57:21.565210 10384 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 26fae3fbfe374f348a1729c21ec751b9 (table default.Table_0011387b45014de4b24b466f4cb907f7 [id=e3f7aa5e988c4b6c98be5cd3d465245d]). Expected version 1 got 0
I20260925 09:57:21.612665 10384 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 0a2bbe141e564a96be05de86f2993234 (table default.Table_b5c66e748a8741f78316bd5264f0f186 [id=aed8a2ddc51545ad9cec4d272dcf3daf]). Expected version 3 got 2
I20260925 09:57:21.614187 10384 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 28d60531970a4ab58ce5ef99cb3a96bf (table default.Table_ce09d3a8d8e84973894f33b18bbfdb22 [id=6851500c7dba44f1bf353bb79bafd97a]). Expected version 2 got 0
I20260925 09:57:21.616432 10384 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet ef6fefaa76d44fe798201aab2f21b9c7 (table default.Table_46e4c497898b454c85b7c2a299ebf3d9 [id=80b48dd413e3446eb7ac65e5f957b5dc]). Expected version 2 got 0
I20260925 09:57:21.622201 11579 ts_tablet_manager.cc:1918] T 752105c1514f4c1db2a11b95b413b4f6 P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:21.626713 11751 consensus_queue.cc:1035] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 09:57:21.628432 10919 leader_election.cc:341] T 3d652a76d96f4dbeafde15f3c16534a6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:21.628840 10919 leader_election.cc:302] T 3d652a76d96f4dbeafde15f3c16534a6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:21.629501 11742 raft_consensus.cc:2726] T 3d652a76d96f4dbeafde15f3c16534a6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:21.536314 11082 raft_consensus.cc:348] T ec1b78e61e8d46cb8b87f60786059658 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:21.631223 11082 raft_consensus.cc:374] T ec1b78e61e8d46cb8b87f60786059658 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:21.631567 11082 raft_consensus.cc:729] T ec1b78e61e8d46cb8b87f60786059658 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:21.633890 11579 log.cc:1196] T 752105c1514f4c1db2a11b95b413b4f6 P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/752105c1514f4c1db2a11b95b413b4f6
I20260925 09:57:21.634622 11579 ts_tablet_manager.cc:1939] T 752105c1514f4c1db2a11b95b413b4f6 P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
W20260925 09:57:21.634879 10775 consensus_peers.cc:487] T 4be3f6e81ddf418fabc7b694a2e62f35 P c43461c4ee854ae1ac52b873ccada318 -> Peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Couldn't send request to peer b8850818e2ee42a082802d0095c08d21. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:21.604415 10393 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_09fe8f366734488c93a79eeeec2c96f9" } modify_external_catalogs: true
I20260925 09:57:21.640941 10393 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_09fe8f366734488c93a79eeeec2c96f9" } modify_external_catalogs: true
I20260925 09:57:21.641500 10384 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet b555eeba505740988c704df5c734e7c0 (table default.Table_48f368d811494119be0dc534e2fdcfbd [id=3e32026d175b4b92a3ee036f17e54b9d]). Expected version 1 got 0
I20260925 09:57:21.644758 11084 tablet_bootstrap.cc:492] T 417058493c6547c69e72347cef373292 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:21.645164 11084 ts_tablet_manager.cc:1397] T 417058493c6547c69e72347cef373292 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.224s user 0.011s sys 0.000s
I20260925 09:57:21.649968 11082 consensus_queue.cc:260] T ec1b78e61e8d46cb8b87f60786059658 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:21.663650 10918 consensus_peers.cc:487] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:21.647704 11085 raft_consensus.cc:348] T c588b0c05a1a4f3ea545066183e76502 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:21.670049 11085 raft_consensus.cc:374] T c588b0c05a1a4f3ea545066183e76502 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:21.670318 11085 raft_consensus.cc:729] T c588b0c05a1a4f3ea545066183e76502 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
W20260925 09:57:21.670228 10918 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 47656.8us
W20260925 09:57:21.671850 10621 leader_election.cc:341] T 3706253e53d841aeaa43889ead13af32 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:21.669610 11727 ts_tablet_manager.cc:1918] T 4be3f6e81ddf418fabc7b694a2e62f35 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:57:21.680157 11727 log.cc:1196] T 4be3f6e81ddf418fabc7b694a2e62f35 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/4be3f6e81ddf418fabc7b694a2e62f35
I20260925 09:57:21.681305 11727 ts_tablet_manager.cc:1939] T 4be3f6e81ddf418fabc7b694a2e62f35 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:21.677942 11084 raft_consensus.cc:348] T 417058493c6547c69e72347cef373292 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:21.681838 11084 raft_consensus.cc:374] T 417058493c6547c69e72347cef373292 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:21.682226 11084 raft_consensus.cc:729] T 417058493c6547c69e72347cef373292 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:21.683046 11084 consensus_queue.cc:260] T 417058493c6547c69e72347cef373292 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:21.686213 11084 ts_tablet_manager.cc:1428] T 417058493c6547c69e72347cef373292 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.041s user 0.006s sys 0.000s
I20260925 09:57:21.688265 11084 tablet_bootstrap.cc:492] T 5246648245e04e26be2abf1dbf706688 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:21.690732 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 4be3f6e81ddf418fabc7b694a2e62f35 (table default.Table_1e7379199917430eb9e027fc7bb2a4f9 [id=c2d0336fd84c4dbe95472f458b813fe6]) successfully deleted
W20260925 09:57:21.693341 10918 leader_election.cc:341] T 3706253e53d841aeaa43889ead13af32 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:21.693209 11655 consensus_queue.cc:1046] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:21.693667 10918 leader_election.cc:302] T 3706253e53d841aeaa43889ead13af32 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:21.693840 11655 consensus_queue.cc:1225] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0 [LEADER]: Peer c43461c4ee854ae1ac52b873ccada318 log is divergent from this leader: its last log entry 2.1 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20260925 09:57:21.694226 11742 raft_consensus.cc:3032] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0 [term 1 LEADER]: Stepping down as leader of term 1
I20260925 09:57:21.692909 11727 tablet_replica.cc:330] T 04fafb9dcf274806b11b2598e73853e9 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:21.694631 11742 raft_consensus.cc:729] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Running, Role: LEADER
W20260925 09:57:21.664882 10919 leader_election.cc:341] T 6161774f61ae48ab9764a05f2188a80b P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:21.697027 10919 leader_election.cc:341] T ec1b78e61e8d46cb8b87f60786059658 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:21.696161 11742 consensus_queue.cc:260] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:21.697893 11655 consensus_queue.cc:1046] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260925 09:57:21.697477 10919 leader_election.cc:302] T ec1b78e61e8d46cb8b87f60786059658 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:21.706192 11605 raft_consensus.cc:2726] T ec1b78e61e8d46cb8b87f60786059658 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:21.708577 10392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_5062a73a7614459a8690df32165076dc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:21.709720 11605 raft_consensus.cc:482] T 7360a869f3be45a884a0379f4ef78e85 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260925 09:57:21.710041 10392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5062a73a7614459a8690df32165076dc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:21.710107 11605 raft_consensus.cc:504] T 7360a869f3be45a884a0379f4ef78e85 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:21.711860 11605 leader_election.cc:288] T 7360a869f3be45a884a0379f4ef78e85 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:21.712669 11742 raft_consensus.cc:3037] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:57:21.702023 11751 raft_consensus.cc:2726] T 3706253e53d841aeaa43889ead13af32 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:21.699080 11727 raft_consensus.cc:2229] T 04fafb9dcf274806b11b2598e73853e9 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260925 09:57:21.726392 10379 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.9.60:36985: connect: Connection refused (error 111) [suppressed 8 similar messages]
I20260925 09:57:21.731699 11084 tablet_bootstrap.cc:654] T 5246648245e04e26be2abf1dbf706688 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:21.702533 10918 leader_election.cc:341] T 6161774f61ae48ab9764a05f2188a80b P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:21.736807 10621 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 64249.6us
I20260925 09:57:21.733816 10918 leader_election.cc:302] T 6161774f61ae48ab9764a05f2188a80b P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:21.742617 11082 ts_tablet_manager.cc:1428] T ec1b78e61e8d46cb8b87f60786059658 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.208s user 0.005s sys 0.000s
I20260925 09:57:21.741173 11759 raft_consensus.cc:482] T 67ce2e3325214aa6bdc3347e9e8f82b0 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260925 09:57:21.737363 10621 leader_election.cc:341] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:21.747596 11605 raft_consensus.cc:482] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:21.748348 11605 raft_consensus.cc:504] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:21.750028 11727 raft_consensus.cc:2258] T 04fafb9dcf274806b11b2598e73853e9 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:21.752404 11727 ts_tablet_manager.cc:1905] T 04fafb9dcf274806b11b2598e73853e9 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:21.758328 11082 tablet_bootstrap.cc:492] T 3d652a76d96f4dbeafde15f3c16534a6 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:21.750829 11605 leader_election.cc:288] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:21.768196 11605 raft_consensus.cc:2726] T 6161774f61ae48ab9764a05f2188a80b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:21.768352 10918 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 65842.7us
W20260925 09:57:21.770961 10621 consensus_peers.cc:487] T cf35f39bcd6a41128acb2a4936fcc316 P b8850818e2ee42a082802d0095c08d21 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:21.772008 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5608c5f9b65b4da7b2fbb56417c92d88" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:21.773221 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7360a869f3be45a884a0379f4ef78e85" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:21.780689 10379 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.60:36985, user_credentials={real_user=slave}} blocked reactor thread for 54307.5us
I20260925 09:57:21.761245 11759 raft_consensus.cc:504] T 67ce2e3325214aa6bdc3347e9e8f82b0 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:21.793031 11759 leader_election.cc:288] T 67ce2e3325214aa6bdc3347e9e8f82b0 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:21.793660 10977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67ce2e3325214aa6bdc3347e9e8f82b0" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:21.794241 10977 raft_consensus.cc:2454] T 67ce2e3325214aa6bdc3347e9e8f82b0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:21.795322 10619 leader_election.cc:302] T 67ce2e3325214aa6bdc3347e9e8f82b0 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:21.798396 11605 raft_consensus.cc:482] T 67ce2e3325214aa6bdc3347e9e8f82b0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:21.799033 11605 raft_consensus.cc:504] T 67ce2e3325214aa6bdc3347e9e8f82b0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:21.801014 11605 leader_election.cc:288] T 67ce2e3325214aa6bdc3347e9e8f82b0 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:21.798890 11759 raft_consensus.cc:2781] T 67ce2e3325214aa6bdc3347e9e8f82b0 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:21.812300 11082 tablet_bootstrap.cc:654] T 3d652a76d96f4dbeafde15f3c16534a6 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:21.811256 11759 raft_consensus.cc:482] T 67ce2e3325214aa6bdc3347e9e8f82b0 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:21.821856 11759 raft_consensus.cc:3037] T 67ce2e3325214aa6bdc3347e9e8f82b0 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:57:21.827903 10919 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 47339.3us
W20260925 09:57:21.828617 10919 leader_election.cc:341] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:21.829185 10919 leader_election.cc:341] T 7360a869f3be45a884a0379f4ef78e85 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:21.858541 11085 consensus_queue.cc:260] T c588b0c05a1a4f3ea545066183e76502 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:21.843112 10388 catalog_manager.cc:5495] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 reported cstate change: term changed from 0 to 1, leader changed from <none> to be97ab5d75934a648d4ab76341193ec0 (127.10.9.3). New cstate: current_term: 1 leader_uuid: "be97ab5d75934a648d4ab76341193ec0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: HEALTHY } } }
I20260925 09:57:21.864785 10388 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 28d60531970a4ab58ce5ef99cb3a96bf (table default.Table_ce09d3a8d8e84973894f33b18bbfdb22 [id=6851500c7dba44f1bf353bb79bafd97a]). Expected version 2 got 0
I20260925 09:57:21.865921 10388 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 3d652a76d96f4dbeafde15f3c16534a6 (table default.Table_be0f658aa41f4188bc16db37e96f4ad1 [id=73083c1560cb4db5a12964387cb49b73]). Expected version 1 got 0
I20260925 09:57:21.866585 10682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67ce2e3325214aa6bdc3347e9e8f82b0" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:21.874066 11674 raft_consensus.cc:482] T 67ce2e3325214aa6bdc3347e9e8f82b0 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:21.874471 11674 raft_consensus.cc:504] T 67ce2e3325214aa6bdc3347e9e8f82b0 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:21.876636 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7360a869f3be45a884a0379f4ef78e85" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:21.876951 11579 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 167) took 1630 ms. Trace:
W20260925 09:57:21.877488 11769 log.cc:924] Time spent T 3706253e53d841aeaa43889ead13af32 P b8850818e2ee42a082802d0095c08d21: Append to log took a long time: real 0.243s user 0.001s sys 0.000s
I20260925 09:57:21.880553 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67ce2e3325214aa6bdc3347e9e8f82b0" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:21.881090 10836 raft_consensus.cc:2454] T 67ce2e3325214aa6bdc3347e9e8f82b0 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:21.886603 11727 ts_tablet_manager.cc:1918] T 04fafb9dcf274806b11b2598e73853e9 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:21.887004 11727 log.cc:1196] T 04fafb9dcf274806b11b2598e73853e9 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/04fafb9dcf274806b11b2598e73853e9
I20260925 09:57:21.877313 11579 rpcz_store.cc:276] 0925 09:57:20.246103 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:20.284403 (+ 38300us) service_pool.cc:227] Handling call
0925 09:57:21.174119 (+889716us) ts_tablet_manager.cc:1145] Deleting tablet 752105c1514f4c1db2a11b95b413b4f6
0925 09:57:21.174162 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:21.472271 (+298109us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:21.621982 (+149711us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:21.876884 (+254902us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":145637,"tablet-delete.queue_time_us":889160}
I20260925 09:57:21.899551 11579 tablet_replica.cc:330] T 4be3f6e81ddf418fabc7b694a2e62f35 P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:21.900573 11579 raft_consensus.cc:2229] T 4be3f6e81ddf418fabc7b694a2e62f35 P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:21.901597 11579 raft_consensus.cc:2258] T 4be3f6e81ddf418fabc7b694a2e62f35 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:21.902551 11674 leader_election.cc:288] T 67ce2e3325214aa6bdc3347e9e8f82b0 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:21.907708 11085 ts_tablet_manager.cc:1428] T c588b0c05a1a4f3ea545066183e76502 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.323s user 0.012s sys 0.007s
I20260925 09:57:21.908723 11085 tablet_bootstrap.cc:492] T cf35f39bcd6a41128acb2a4936fcc316 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
W20260925 09:57:21.909169 10379 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.60:36985, user_credentials={real_user=slave}} blocked reactor thread for 72028.3us
I20260925 09:57:21.917095 11084 tablet_bootstrap.cc:492] T 5246648245e04e26be2abf1dbf706688 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:21.917632 11084 ts_tablet_manager.cc:1397] T 5246648245e04e26be2abf1dbf706688 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.230s user 0.010s sys 0.012s
I20260925 09:57:21.866693 10388 catalog_manager.cc:5495] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0 reported cstate change: term changed from 0 to 1, leader changed from <none> to be97ab5d75934a648d4ab76341193ec0 (127.10.9.3). New cstate: current_term: 1 leader_uuid: "be97ab5d75934a648d4ab76341193ec0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: HEALTHY } } }
I20260925 09:57:21.924719 10388 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet b555eeba505740988c704df5c734e7c0 (table default.Table_48f368d811494119be0dc534e2fdcfbd [id=3e32026d175b4b92a3ee036f17e54b9d]). Expected version 1 got 0
I20260925 09:57:21.933466 11727 ts_tablet_manager.cc:1939] T 04fafb9dcf274806b11b2598e73853e9 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:21.939589 11084 raft_consensus.cc:348] T 5246648245e04e26be2abf1dbf706688 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:21.940250 11084 raft_consensus.cc:374] T 5246648245e04e26be2abf1dbf706688 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:21.940480 11084 raft_consensus.cc:729] T 5246648245e04e26be2abf1dbf706688 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:21.941371 11579 ts_tablet_manager.cc:1905] T 4be3f6e81ddf418fabc7b694a2e62f35 P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:21.944774 10388 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet ef6fefaa76d44fe798201aab2f21b9c7 (table default.Table_46e4c497898b454c85b7c2a299ebf3d9 [id=80b48dd413e3446eb7ac65e5f957b5dc]). Expected version 2 got 0
I20260925 09:57:21.948091 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 171) took 1163 ms. Trace:
I20260925 09:57:21.949306 11082 tablet_bootstrap.cc:492] T 3d652a76d96f4dbeafde15f3c16534a6 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:21.949682 11082 ts_tablet_manager.cc:1397] T 3d652a76d96f4dbeafde15f3c16534a6 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.192s user 0.008s sys 0.002s
I20260925 09:57:21.951833 11085 tablet_bootstrap.cc:654] T cf35f39bcd6a41128acb2a4936fcc316 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:21.945501 10388 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 26fae3fbfe374f348a1729c21ec751b9 (table default.Table_0011387b45014de4b24b466f4cb907f7 [id=e3f7aa5e988c4b6c98be5cd3d465245d]). Expected version 1 got 0
I20260925 09:57:21.954704 10388 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 0a2bbe141e564a96be05de86f2993234 (table default.Table_b5c66e748a8741f78316bd5264f0f186 [id=aed8a2ddc51545ad9cec4d272dcf3daf]). Expected version 3 got 2
I20260925 09:57:21.970611 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 752105c1514f4c1db2a11b95b413b4f6 (table default.Table_8c5b820f0db34e3bbea2d404afb1909a [id=fc979ab5541c41ef82a95e11f5a455ab]) successfully deleted
I20260925 09:57:21.976016 10392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_755c3887714845d2bd8bb30bd444adef"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:21.977398 10392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_755c3887714845d2bd8bb30bd444adef in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:21.981410 10393 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet ec1b78e61e8d46cb8b87f60786059658
I20260925 09:57:21.983223 10393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_86a9e15122f04c9c9a111c49547520dc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:21.984571 10393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_86a9e15122f04c9c9a111c49547520dc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:21.987717 10658 tablet_service.cc:1514] Processing DeleteTablet for tablet ec1b78e61e8d46cb8b87f60786059658 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:21 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:21.988509 11759 raft_consensus.cc:504] T 67ce2e3325214aa6bdc3347e9e8f82b0 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:21.990265 10682 raft_consensus.cc:2379] T 67ce2e3325214aa6bdc3347e9e8f82b0 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be97ab5d75934a648d4ab76341193ec0 in current term 1: Already voted for candidate b8850818e2ee42a082802d0095c08d21 in this term.
I20260925 09:57:21.948439 11727 rpcz_store.cc:276] 0925 09:57:20.786399 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:20.789674 (+ 3275us) service_pool.cc:227] Handling call
0925 09:57:21.692635 (+902961us) ts_tablet_manager.cc:1145] Deleting tablet 04fafb9dcf274806b11b2598e73853e9
0925 09:57:21.692700 (+ 65us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:21.816567 (+123867us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:21.886387 (+ 69820us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:21.948024 (+ 61637us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":25157,"tablet-delete.queue_time_us":901983}
I20260925 09:57:21.989173 10958 tablet_service.cc:1514] Processing DeleteTablet for tablet ec1b78e61e8d46cb8b87f60786059658 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:21 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:21.996101 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 04fafb9dcf274806b11b2598e73853e9 (table default.Table_bc69fdbbaf5b4a4e8cdfe195eefb16ab [id=d76cef751d12440190294e86649e2639]) successfully deleted
I20260925 09:57:21.997135 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 173) took 1210 ms. Trace:
I20260925 09:57:21.997486 11727 rpcz_store.cc:276] 0925 09:57:20.787212 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:20.791497 (+ 4285us) service_pool.cc:227] Handling call
0925 09:57:21.996854 (+1205357us) ts_tablet_manager.cc:1145] Deleting tablet 752105c1514f4c1db2a11b95b413b4f6
0925 09:57:21.996904 (+ 50us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:21.997081 (+ 177us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1204819}
W20260925 09:57:21.998757 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 752105c1514f4c1db2a11b95b413b4f6 because the tablet was not found. No further retry: Not found: Tablet not found: 752105c1514f4c1db2a11b95b413b4f6
I20260925 09:57:21.953595 11084 consensus_queue.cc:260] T 5246648245e04e26be2abf1dbf706688 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:22.001967 11084 ts_tablet_manager.cc:1428] T 5246648245e04e26be2abf1dbf706688 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.084s user 0.004s sys 0.003s
I20260925 09:57:22.007141 11084 tablet_bootstrap.cc:492] T e684798155d445bfb325a1b7c5c48753 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:22.005362 11727 tablet_replica.cc:330] T 6161774f61ae48ab9764a05f2188a80b P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:22.011969 11727 raft_consensus.cc:2229] T 6161774f61ae48ab9764a05f2188a80b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:22.016237 11727 raft_consensus.cc:2258] T 6161774f61ae48ab9764a05f2188a80b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260925 09:57:22.017485 10379 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.62:34455, user_credentials={real_user=slave}} blocked reactor thread for 46118.9us
I20260925 09:57:22.019960 11727 ts_tablet_manager.cc:1905] T 6161774f61ae48ab9764a05f2188a80b P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:22.028596 11709 ts_tablet_manager.cc:1918] T 4be3f6e81ddf418fabc7b694a2e62f35 P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:57:22.028990 11709 log.cc:1196] T 4be3f6e81ddf418fabc7b694a2e62f35 P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/4be3f6e81ddf418fabc7b694a2e62f35
I20260925 09:57:22.029769 11709 ts_tablet_manager.cc:1939] T 4be3f6e81ddf418fabc7b694a2e62f35 P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:22.032505 11759 leader_election.cc:288] T 67ce2e3325214aa6bdc3347e9e8f82b0 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:21.951938 11082 raft_consensus.cc:348] T 3d652a76d96f4dbeafde15f3c16534a6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:22.033507 11082 raft_consensus.cc:374] T 3d652a76d96f4dbeafde15f3c16534a6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:22.032529 11627 consensus_queue.cc:1035] T cf35f39bcd6a41128acb2a4936fcc316 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260925 09:57:22.043612 11709 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 165) took 1641 ms. Trace:
I20260925 09:57:22.033749 11082 raft_consensus.cc:729] T 3d652a76d96f4dbeafde15f3c16534a6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:22.046722 11082 consensus_queue.cc:260] T 3d652a76d96f4dbeafde15f3c16534a6 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:22.043962 11709 rpcz_store.cc:276] 0925 09:57:20.402309 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:20.547451 (+145142us) service_pool.cc:227] Handling call
0925 09:57:21.408982 (+861531us) ts_tablet_manager.cc:1145] Deleting tablet 4be3f6e81ddf418fabc7b694a2e62f35
0925 09:57:21.409028 (+ 46us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:21.830734 (+421706us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:22.028362 (+197628us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:22.043538 (+ 15176us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":860927}
I20260925 09:57:22.061023 10393 master_service.cc:946] ReplaceTablet: received request to replace tablet 60fdc299865446c89d3e5374d573029a from {username='slave'} at 127.0.0.1:35140
I20260925 09:57:22.061259 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 4be3f6e81ddf418fabc7b694a2e62f35 (table default.Table_1e7379199917430eb9e027fc7bb2a4f9 [id=c2d0336fd84c4dbe95472f458b813fe6]) successfully deleted
I20260925 09:57:22.062438 11709 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 175) took 1462 ms. Trace:
I20260925 09:57:22.091988 11709 rpcz_store.cc:276] 0925 09:57:20.599971 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:20.600188 (+ 217us) service_pool.cc:227] Handling call
0925 09:57:22.062054 (+1461866us) ts_tablet_manager.cc:1145] Deleting tablet 6161774f61ae48ab9764a05f2188a80b
0925 09:57:22.062102 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:22.062383 (+ 281us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1461241}
W20260925 09:57:22.093868 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 6161774f61ae48ab9764a05f2188a80b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6161774f61ae48ab9764a05f2188a80b already in progress: creating tablet
I20260925 09:57:22.095098 11709 tablet_replica.cc:330] T 89a143a5623e4ef38027381a56c95227 P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:22.039132 10977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67ce2e3325214aa6bdc3347e9e8f82b0" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:22.095670 10977 raft_consensus.cc:3037] T 67ce2e3325214aa6bdc3347e9e8f82b0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:22.097502 10807 tablet_service.cc:1514] Processing DeleteTablet for tablet ec1b78e61e8d46cb8b87f60786059658 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:21 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:22.099046 10978 raft_consensus.cc:1261] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [term 2 FOLLOWER]: Refusing update from remote peer c43461c4ee854ae1ac52b873ccada318: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260925 09:57:22.099373 10978 pending_rounds.cc:76] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0: Aborting all ops after (but not including) 0
I20260925 09:57:22.089480 11751 raft_consensus.cc:482] T 417058493c6547c69e72347cef373292 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:22.099607 10978 pending_rounds.cc:97] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260925 09:57:22.099797 11751 raft_consensus.cc:504] T 417058493c6547c69e72347cef373292 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:22.101969 11751 leader_election.cc:288] T 417058493c6547c69e72347cef373292 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:22.103142 10682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "417058493c6547c69e72347cef373292" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:22.104785 10919 leader_election.cc:341] T 417058493c6547c69e72347cef373292 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:22.105664 10918 leader_election.cc:341] T 7360a869f3be45a884a0379f4ef78e85 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:22.106014 10918 leader_election.cc:302] T 7360a869f3be45a884a0379f4ef78e85 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:22.107743 11605 raft_consensus.cc:2726] T 7360a869f3be45a884a0379f4ef78e85 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:22.109598 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67ce2e3325214aa6bdc3347e9e8f82b0" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:22.110507 10977 raft_consensus.cc:2454] T 67ce2e3325214aa6bdc3347e9e8f82b0 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:22.112022 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67ce2e3325214aa6bdc3347e9e8f82b0" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:22.112754 10684 raft_consensus.cc:2379] T 67ce2e3325214aa6bdc3347e9e8f82b0 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c43461c4ee854ae1ac52b873ccada318 in current term 1: Already voted for candidate b8850818e2ee42a082802d0095c08d21 in this term.
I20260925 09:57:22.116115 10619 leader_election.cc:302] T 67ce2e3325214aa6bdc3347e9e8f82b0 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:22.116943 11627 raft_consensus.cc:2781] T 67ce2e3325214aa6bdc3347e9e8f82b0 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:22.117388 11627 raft_consensus.cc:686] T 67ce2e3325214aa6bdc3347e9e8f82b0 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:22.099917 10978 raft_consensus.cc:2866] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260925 09:57:22.118243 11627 consensus_queue.cc:237] T 67ce2e3325214aa6bdc3347e9e8f82b0 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:22.125897 10977 raft_consensus.cc:1261] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0 [term 2 FOLLOWER]: Refusing update from remote peer c43461c4ee854ae1ac52b873ccada318: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260925 09:57:22.126480 10977 pending_rounds.cc:76] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0: Aborting all ops after (but not including) 0
I20260925 09:57:22.135170 10977 pending_rounds.cc:97] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260925 09:57:22.135558 10977 raft_consensus.cc:2866] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260925 09:57:22.140314 10387 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 26fae3fbfe374f348a1729c21ec751b9 (table default.Table_0011387b45014de4b24b466f4cb907f7 [id=e3f7aa5e988c4b6c98be5cd3d465245d]). Expected version 1 got 0
I20260925 09:57:22.072232 11082 ts_tablet_manager.cc:1428] T 3d652a76d96f4dbeafde15f3c16534a6 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.122s user 0.006s sys 0.000s
I20260925 09:57:22.141302 10387 catalog_manager.cc:5495] T c8522c08bdd94806b5c84f5a41ca5bf4 P c43461c4ee854ae1ac52b873ccada318 reported cstate change: term changed from 0 to 1, leader changed from <none> to c43461c4ee854ae1ac52b873ccada318 (127.10.9.2). New cstate: current_term: 1 leader_uuid: "c43461c4ee854ae1ac52b873ccada318" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:22.142460 11082 tablet_bootstrap.cc:492] T a21909e2e5794f57971f4e42795235c7 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:22.146270 10393 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 60fdc299865446c89d3e5374d573029a
I20260925 09:57:22.148136 10393 catalog_manager.cc:6565] ReplaceTablet: tablet 60fdc299865446c89d3e5374d573029a deleted and replaced by tablet 61aa6380d30a4b21b70ee2d321247bd8
I20260925 09:57:22.148896 10393 master_service.cc:946] ReplaceTablet: received request to replace tablet 545ac025f2cb469da847979011a0b3ce from {username='slave'} at 127.0.0.1:35140
I20260925 09:57:22.148846 10958 tablet_service.cc:1514] Processing DeleteTablet for tablet 60fdc299865446c89d3e5374d573029a with delete_type TABLET_DATA_DELETED (replaced by tablet 61aa6380d30a4b21b70ee2d321247bd8) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:22.153368 10387 catalog_manager.cc:5495] T 359090244c4748e8a2d45e3c77e89498 P c43461c4ee854ae1ac52b873ccada318 reported cstate change: term changed from 1 to 2, leader changed from be97ab5d75934a648d4ab76341193ec0 (127.10.9.3) to c43461c4ee854ae1ac52b873ccada318 (127.10.9.2). New cstate: current_term: 2 leader_uuid: "c43461c4ee854ae1ac52b873ccada318" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:22.162611 11751 raft_consensus.cc:482] T d9e454f82a4e4c429b58f32926319b7c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:22.163045 11751 raft_consensus.cc:504] T d9e454f82a4e4c429b58f32926319b7c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:22.163941 10393 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 545ac025f2cb469da847979011a0b3ce
I20260925 09:57:22.165179 11751 leader_election.cc:288] T d9e454f82a4e4c429b58f32926319b7c P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:22.165839 10807 tablet_service.cc:1514] Processing DeleteTablet for tablet 60fdc299865446c89d3e5374d573029a with delete_type TABLET_DATA_DELETED (replaced by tablet 61aa6380d30a4b21b70ee2d321247bd8) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:22.166676 10393 catalog_manager.cc:6565] ReplaceTablet: tablet 545ac025f2cb469da847979011a0b3ce deleted and replaced by tablet ab6bdba5ca5d41ae921a90c1f77006c5
I20260925 09:57:22.167748 10393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_a749aff6c09e4425afc67db6a98196bd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:22.168213 10807 tablet_service.cc:1514] Processing DeleteTablet for tablet 545ac025f2cb469da847979011a0b3ce with delete_type TABLET_DATA_DELETED (replaced by tablet ab6bdba5ca5d41ae921a90c1f77006c5) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:22.168252 10958 tablet_service.cc:1514] Processing DeleteTablet for tablet 545ac025f2cb469da847979011a0b3ce with delete_type TABLET_DATA_DELETED (replaced by tablet ab6bdba5ca5d41ae921a90c1f77006c5) from {username='slave'} at 127.0.0.1:53560
W20260925 09:57:22.169286 10393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a749aff6c09e4425afc67db6a98196bd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:22.160840 11791 consensus_queue.cc:1035] T c8522c08bdd94806b5c84f5a41ca5bf4 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260925 09:57:22.177325 11084 tablet_bootstrap.cc:654] T e684798155d445bfb325a1b7c5c48753 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:22.182423 10772 leader_election.cc:302] T 67ce2e3325214aa6bdc3347e9e8f82b0 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
W20260925 09:57:22.192111 10775 consensus_peers.cc:487] T 89a143a5623e4ef38027381a56c95227 P c43461c4ee854ae1ac52b873ccada318 -> Peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Couldn't send request to peer b8850818e2ee42a082802d0095c08d21. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:22.198525 11792 consensus_queue.cc:1035] T 89a143a5623e4ef38027381a56c95227 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:22.196455 11794 raft_consensus.cc:3037] T 67ce2e3325214aa6bdc3347e9e8f82b0 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:22.213624 11794 raft_consensus.cc:2726] T 67ce2e3325214aa6bdc3347e9e8f82b0 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:22.147969 10658 tablet_service.cc:1514] Processing DeleteTablet for tablet 60fdc299865446c89d3e5374d573029a with delete_type TABLET_DATA_DELETED (replaced by tablet 61aa6380d30a4b21b70ee2d321247bd8) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:22.166039 10666 tablet_service.cc:1514] Processing DeleteTablet for tablet 545ac025f2cb469da847979011a0b3ce with delete_type TABLET_DATA_DELETED (replaced by tablet ab6bdba5ca5d41ae921a90c1f77006c5) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:22.166949 10682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9e454f82a4e4c429b58f32926319b7c" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:22.179128 10684 raft_consensus.cc:1261] T 359090244c4748e8a2d45e3c77e89498 P b8850818e2ee42a082802d0095c08d21 [term 2 FOLLOWER]: Refusing update from remote peer c43461c4ee854ae1ac52b873ccada318: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
W20260925 09:57:22.223585 10919 leader_election.cc:341] T d9e454f82a4e4c429b58f32926319b7c P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:22.197578 11789 consensus_queue.cc:1035] T 359090244c4748e8a2d45e3c77e89498 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.026s
I20260925 09:57:22.227926 11794 consensus_queue.cc:1035] T c8522c08bdd94806b5c84f5a41ca5bf4 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:22.230072 11794 consensus_queue.cc:1035] T 359090244c4748e8a2d45e3c77e89498 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:22.234006 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5608c5f9b65b4da7b2fbb56417c92d88" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:22.238248 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67ce2e3325214aa6bdc3347e9e8f82b0" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:22.238401 10393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_29e5f5a8cdbd472a9595a84716d0bfa1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:22.238973 10835 raft_consensus.cc:2454] T 67ce2e3325214aa6bdc3347e9e8f82b0 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be97ab5d75934a648d4ab76341193ec0 in term 1.
I20260925 09:57:22.240253 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67ce2e3325214aa6bdc3347e9e8f82b0" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:22.255906 11742 raft_consensus.cc:482] T 5246648245e04e26be2abf1dbf706688 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:22.256407 11742 raft_consensus.cc:504] T 5246648245e04e26be2abf1dbf706688 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:22.258368 11742 leader_election.cc:288] T 5246648245e04e26be2abf1dbf706688 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:22.259217 10682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5246648245e04e26be2abf1dbf706688" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:22.260825 10919 leader_election.cc:341] T 5246648245e04e26be2abf1dbf706688 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:22.262928 11579 ts_tablet_manager.cc:1918] T 4be3f6e81ddf418fabc7b694a2e62f35 P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:57:22.266867 10836 raft_consensus.cc:2454] T 67ce2e3325214aa6bdc3347e9e8f82b0 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:22.217227 11709 raft_consensus.cc:2229] T 89a143a5623e4ef38027381a56c95227 P b8850818e2ee42a082802d0095c08d21 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260925 09:57:22.269550 10393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_29e5f5a8cdbd472a9595a84716d0bfa1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:22.271937 11752 consensus_queue.cc:1035] T cfa619b699004882a82976f0ef8204ad P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:22.220732 11627 raft_consensus.cc:482] T 3d652a76d96f4dbeafde15f3c16534a6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:22.273959 11627 raft_consensus.cc:504] T 3d652a76d96f4dbeafde15f3c16534a6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:22.267796 11709 raft_consensus.cc:2258] T 89a143a5623e4ef38027381a56c95227 P b8850818e2ee42a082802d0095c08d21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:22.290113 11709 ts_tablet_manager.cc:1905] T 89a143a5623e4ef38027381a56c95227 P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:22.291488 10393 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_95a4c3e1f6014e63811447ba4a0e34cf" } modify_external_catalogs: true
I20260925 09:57:22.292196 10393 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_95a4c3e1f6014e63811447ba4a0e34cf" } modify_external_catalogs: true
I20260925 09:57:22.293174 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "417058493c6547c69e72347cef373292" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:22.297814 11085 tablet_bootstrap.cc:492] T cf35f39bcd6a41128acb2a4936fcc316 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:22.298596 11085 ts_tablet_manager.cc:1397] T cf35f39bcd6a41128acb2a4936fcc316 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.390s user 0.009s sys 0.002s
I20260925 09:57:22.308893 10977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d652a76d96f4dbeafde15f3c16534a6" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:22.309687 10977 raft_consensus.cc:2454] T 3d652a76d96f4dbeafde15f3c16534a6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:22.310781 10619 leader_election.cc:302] T 3d652a76d96f4dbeafde15f3c16534a6 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:22.310842 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9e454f82a4e4c429b58f32926319b7c" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:22.311902 11752 raft_consensus.cc:2781] T 3d652a76d96f4dbeafde15f3c16534a6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:22.312220 11752 raft_consensus.cc:482] T 3d652a76d96f4dbeafde15f3c16534a6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:22.312500 11752 raft_consensus.cc:3037] T 3d652a76d96f4dbeafde15f3c16534a6 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:22.319362 11752 raft_consensus.cc:504] T 3d652a76d96f4dbeafde15f3c16534a6 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:22.321111 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5246648245e04e26be2abf1dbf706688" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:22.322886 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d652a76d96f4dbeafde15f3c16534a6" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:22.323748 10976 raft_consensus.cc:3037] T 3d652a76d96f4dbeafde15f3c16534a6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:22.331455 10976 raft_consensus.cc:2454] T 3d652a76d96f4dbeafde15f3c16534a6 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:22.334246 11752 leader_election.cc:288] T 3d652a76d96f4dbeafde15f3c16534a6 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:22.334151 10619 leader_election.cc:302] T 3d652a76d96f4dbeafde15f3c16534a6 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:22.335317 11634 raft_consensus.cc:2781] T 3d652a76d96f4dbeafde15f3c16534a6 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:22.341981 11752 consensus_queue.cc:1035] T 197f61525b0c4f9da441d98701d231a3 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:22.349674 11634 raft_consensus.cc:686] T 3d652a76d96f4dbeafde15f3c16534a6 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:22.352346 11634 consensus_queue.cc:237] T 3d652a76d96f4dbeafde15f3c16534a6 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:22.353399 10807 tablet_service.cc:1467] Processing CreateTablet for tablet 6540aba5854a44419addc5d454e473b7 (DEFAULT_TABLE table=default.Table_0f7925265a1741aa97617300030475a1 [id=416a4b989570457e919fb7e7b16a017d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:22.354836 10807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6540aba5854a44419addc5d454e473b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:22.359905 11627 leader_election.cc:288] T 3d652a76d96f4dbeafde15f3c16534a6 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:22.360149 10393 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet d9e454f82a4e4c429b58f32926319b7c
I20260925 09:57:22.362803 10958 tablet_service.cc:1514] Processing DeleteTablet for tablet d9e454f82a4e4c429b58f32926319b7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:22 UTC) from {username='slave'} at 127.0.0.1:53560
W20260925 09:57:22.364346 10918 leader_election.cc:341] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:22.364679 10666 tablet_service.cc:1514] Processing DeleteTablet for tablet d9e454f82a4e4c429b58f32926319b7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:22 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:22.365022 10918 leader_election.cc:302] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:22.365537 10393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_14fecfa780f146fba2e90a2751353aca"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:22.365860 11751 raft_consensus.cc:2726] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:22.367074 10393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_14fecfa780f146fba2e90a2751353aca in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:22.369645 10666 tablet_service.cc:1467] Processing CreateTablet for tablet 6540aba5854a44419addc5d454e473b7 (DEFAULT_TABLE table=default.Table_0f7925265a1741aa97617300030475a1 [id=416a4b989570457e919fb7e7b16a017d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:22.370553 10666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6540aba5854a44419addc5d454e473b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:22.380288 10918 leader_election.cc:302] T 67ce2e3325214aa6bdc3347e9e8f82b0 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21
I20260925 09:57:22.381428 10958 tablet_service.cc:1467] Processing CreateTablet for tablet 6540aba5854a44419addc5d454e473b7 (DEFAULT_TABLE table=default.Table_0f7925265a1741aa97617300030475a1 [id=416a4b989570457e919fb7e7b16a017d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:22.381793 11742 raft_consensus.cc:2741] T 67ce2e3325214aa6bdc3347e9e8f82b0 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
W20260925 09:57:22.383249 10918 leader_election.cc:341] T 417058493c6547c69e72347cef373292 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:22.383617 10658 tablet_service.cc:1467] Processing CreateTablet for tablet fb9c51f7e7fd4c029be2a710cb9462eb (DEFAULT_TABLE table=default.Table_44ff944ba4b84cb7a677f7dd03f8f7dd [id=80ab4d354310414eb9374b8ac6a3a946]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:22.383723 10918 leader_election.cc:302] T 417058493c6547c69e72347cef373292 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:22.384634 10658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb9c51f7e7fd4c029be2a710cb9462eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:22.384450 11751 raft_consensus.cc:2726] T 417058493c6547c69e72347cef373292 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:22.395869 10380 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 4e3120347c8e4a3eab57d6562ae40e8d (table default.Table_8e186a99ccba425b9b588041d2364a40 [id=2d6bc0e2aaee4b1483b01fc678505bb0], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 4e3120347c8e4a3eab57d6562ae40e8d consensus state has no leader
I20260925 09:57:22.404418 11627 consensus_queue.cc:1035] T 93e1b5837aff48d7907890c40dfcd0d0 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:22.406771 10392 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 26fae3fbfe374f348a1729c21ec751b9 (table default.Table_0011387b45014de4b24b466f4cb907f7 [id=e3f7aa5e988c4b6c98be5cd3d465245d]). Expected version 1 got 0
W20260925 09:57:22.408123 10918 leader_election.cc:341] T d9e454f82a4e4c429b58f32926319b7c P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:22.408571 10918 leader_election.cc:302] T d9e454f82a4e4c429b58f32926319b7c P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:22.409417 11751 raft_consensus.cc:2726] T d9e454f82a4e4c429b58f32926319b7c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:22.382460 11579 log.cc:1196] T 4be3f6e81ddf418fabc7b694a2e62f35 P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/4be3f6e81ddf418fabc7b694a2e62f35
I20260925 09:57:22.302560 11085 raft_consensus.cc:348] T cf35f39bcd6a41128acb2a4936fcc316 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:22.415611 11085 raft_consensus.cc:374] T cf35f39bcd6a41128acb2a4936fcc316 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:22.415895 11085 raft_consensus.cc:729] T cf35f39bcd6a41128acb2a4936fcc316 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:22.416495 11085 consensus_queue.cc:260] T cf35f39bcd6a41128acb2a4936fcc316 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:22.417620 11579 ts_tablet_manager.cc:1939] T 4be3f6e81ddf418fabc7b694a2e62f35 P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:22.419296 10959 tablet_service.cc:1467] Processing CreateTablet for tablet fb9c51f7e7fd4c029be2a710cb9462eb (DEFAULT_TABLE table=default.Table_44ff944ba4b84cb7a677f7dd03f8f7dd [id=80ab4d354310414eb9374b8ac6a3a946]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:22.421312 10659 tablet_service.cc:1467] Processing CreateTablet for tablet a5d872c3620646e5ac33ceb95de2cbd4 (DEFAULT_TABLE table=default.Table_88612386ae4c426fb71eae81bdd32061 [id=6dbd6b8fade644a19756352bba63764a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:22.422358 10659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5d872c3620646e5ac33ceb95de2cbd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:22.423556 11751 raft_consensus.cc:482] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:22.424222 11751 raft_consensus.cc:504] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:22.427137 11751 leader_election.cc:288] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:22.428166 10810 tablet_service.cc:1514] Processing DeleteTablet for tablet d9e454f82a4e4c429b58f32926319b7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:22 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:22.428126 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d087d3e35814c90a03eccd90eeea86c" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:22.429359 10919 leader_election.cc:341] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:22.433140 10810 tablet_service.cc:1467] Processing CreateTablet for tablet fb9c51f7e7fd4c029be2a710cb9462eb (DEFAULT_TABLE table=default.Table_44ff944ba4b84cb7a677f7dd03f8f7dd [id=80ab4d354310414eb9374b8ac6a3a946]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:22.434877 10805 tablet_service.cc:1467] Processing CreateTablet for tablet a5d872c3620646e5ac33ceb95de2cbd4 (DEFAULT_TABLE table=default.Table_88612386ae4c426fb71eae81bdd32061 [id=6dbd6b8fade644a19756352bba63764a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:22.445286 11627 raft_consensus.cc:482] T ec1b78e61e8d46cb8b87f60786059658 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:22.445791 11627 raft_consensus.cc:504] T ec1b78e61e8d46cb8b87f60786059658 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:22.449098 10973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec1b78e61e8d46cb8b87f60786059658" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:22.449815 10973 raft_consensus.cc:2454] T ec1b78e61e8d46cb8b87f60786059658 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:22.451112 10619 leader_election.cc:302] T ec1b78e61e8d46cb8b87f60786059658 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:22.452093 11801 raft_consensus.cc:2781] T ec1b78e61e8d46cb8b87f60786059658 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:22.452510 11801 raft_consensus.cc:482] T ec1b78e61e8d46cb8b87f60786059658 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:22.452973 11801 raft_consensus.cc:3037] T ec1b78e61e8d46cb8b87f60786059658 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:22.452756 10393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_c720bf4c23b44a94b73ef67ad45560ce"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:22.454562 10393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c720bf4c23b44a94b73ef67ad45560ce in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:22.461465 10958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6540aba5854a44419addc5d454e473b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:22.462128 10959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb9c51f7e7fd4c029be2a710cb9462eb. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:22.462174 10918 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 54086.8us
W20260925 09:57:22.463270 10918 leader_election.cc:341] T 5246648245e04e26be2abf1dbf706688 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:22.463701 10918 leader_election.cc:302] T 5246648245e04e26be2abf1dbf706688 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:22.465082 11751 raft_consensus.cc:2726] T 5246648245e04e26be2abf1dbf706688 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:22.472332 11627 leader_election.cc:288] T ec1b78e61e8d46cb8b87f60786059658 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
W20260925 09:57:22.474733 10775 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 54709.3us
I20260925 09:57:22.477005 11082 tablet_bootstrap.cc:654] T a21909e2e5794f57971f4e42795235c7 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:22.477182 11670 raft_consensus.cc:482] T c588b0c05a1a4f3ea545066183e76502 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:22.477751 11670 raft_consensus.cc:504] T c588b0c05a1a4f3ea545066183e76502 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:22.478899 11085 ts_tablet_manager.cc:1428] T cf35f39bcd6a41128acb2a4936fcc316 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.180s user 0.004s sys 0.004s
I20260925 09:57:22.480783 11742 raft_consensus.cc:482] T df006163f72743b594baf1a3efd1c771 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:22.481299 11742 raft_consensus.cc:504] T df006163f72743b594baf1a3efd1c771 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:22.484563 11670 leader_election.cc:288] T c588b0c05a1a4f3ea545066183e76502 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:22.485139 10959 tablet_service.cc:1467] Processing CreateTablet for tablet a5d872c3620646e5ac33ceb95de2cbd4 (DEFAULT_TABLE table=default.Table_88612386ae4c426fb71eae81bdd32061 [id=6dbd6b8fade644a19756352bba63764a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:22.485868 11742 leader_election.cc:288] T df006163f72743b594baf1a3efd1c771 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:22.486050 10959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5d872c3620646e5ac33ceb95de2cbd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:22.486481 10682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df006163f72743b594baf1a3efd1c771" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:22.488031 10961 tablet_service.cc:1467] Processing CreateTablet for tablet 37bab60afe7a4c758a08ef7c9c24aba0 (DEFAULT_TABLE table=default.Table_8b45818055ec40e386d43e2641ec6ed0 [id=38b800f8e1e948909d40cc976272fdd6]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 09:57:22.488814 10919 leader_election.cc:341] T df006163f72743b594baf1a3efd1c771 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:22.488929 10961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37bab60afe7a4c758a08ef7c9c24aba0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:22.489845 10664 tablet_service.cc:1467] Processing CreateTablet for tablet 37bab60afe7a4c758a08ef7c9c24aba0 (DEFAULT_TABLE table=default.Table_8b45818055ec40e386d43e2641ec6ed0 [id=38b800f8e1e948909d40cc976272fdd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:22.490705 10664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37bab60afe7a4c758a08ef7c9c24aba0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:22.489457 10392 catalog_manager.cc:5495] T cfa619b699004882a82976f0ef8204ad P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:22.492530 10665 tablet_service.cc:1467] Processing CreateTablet for tablet 10657a81058241fc988d37e5ea7ff8cf (DEFAULT_TABLE table=default.Table_64ba123898174e72a6884dbb9ad07de3 [id=7ee526720b2442119e0fc487b7c4fcd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:22.493494 10665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10657a81058241fc988d37e5ea7ff8cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:22.503762 10983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c588b0c05a1a4f3ea545066183e76502" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:22.514715 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d652a76d96f4dbeafde15f3c16534a6" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:22.500625 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d652a76d96f4dbeafde15f3c16534a6" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:22.521271 11084 tablet_bootstrap.cc:492] T e684798155d445bfb325a1b7c5c48753 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:22.521642 11084 ts_tablet_manager.cc:1397] T e684798155d445bfb325a1b7c5c48753 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.515s user 0.007s sys 0.006s
I20260925 09:57:22.536438 10682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c588b0c05a1a4f3ea545066183e76502" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:22.537338 10682 raft_consensus.cc:2379] T c588b0c05a1a4f3ea545066183e76502 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c43461c4ee854ae1ac52b873ccada318 in current term 1: Already voted for candidate be97ab5d75934a648d4ab76341193ec0 in this term.
I20260925 09:57:22.538682 10775 leader_election.cc:302] T c588b0c05a1a4f3ea545066183e76502 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:22.540088 11779 raft_consensus.cc:3037] T c588b0c05a1a4f3ea545066183e76502 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:22.540900 11579 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 173) took 2123 ms. Trace:
I20260925 09:57:22.541206 11579 rpcz_store.cc:276] 0925 09:57:20.417336 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:20.423696 (+ 6360us) service_pool.cc:227] Handling call
0925 09:57:21.899345 (+1475649us) ts_tablet_manager.cc:1145] Deleting tablet 4be3f6e81ddf418fabc7b694a2e62f35
0925 09:57:21.899415 (+ 70us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:21.976931 (+ 77516us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:22.262493 (+285562us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:22.540833 (+278340us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1475020}
I20260925 09:57:22.542639 11579 tablet_replica.cc:330] T 04fafb9dcf274806b11b2598e73853e9 P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:22.549045 11779 raft_consensus.cc:2726] T c588b0c05a1a4f3ea545066183e76502 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:22.555121 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d087d3e35814c90a03eccd90eeea86c" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:22.561611 11709 ts_tablet_manager.cc:1918] T 89a143a5623e4ef38027381a56c95227 P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:22.564939 11084 raft_consensus.cc:348] T e684798155d445bfb325a1b7c5c48753 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:22.565575 11084 raft_consensus.cc:374] T e684798155d445bfb325a1b7c5c48753 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:22.565812 11084 raft_consensus.cc:729] T e684798155d445bfb325a1b7c5c48753 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:22.573243 10959 tablet_service.cc:1467] Processing CreateTablet for tablet 10657a81058241fc988d37e5ea7ff8cf (DEFAULT_TABLE table=default.Table_64ba123898174e72a6884dbb9ad07de3 [id=7ee526720b2442119e0fc487b7c4fcd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:22.574641 10960 tablet_service.cc:1467] Processing CreateTablet for tablet dba14a44a9de492886d629441606767e (DEFAULT_TABLE table=default.Table_bb742c618d96453aaabe897d54fc657b [id=68683893b91c4f0190a0050ecdc6d396]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:22.575677 10960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dba14a44a9de492886d629441606767e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:22.577419 11742 raft_consensus.cc:482] T c2110d4ada2d45c2b07259374465a8df P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:22.578344 10810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb9c51f7e7fd4c029be2a710cb9462eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:22.577874 11742 raft_consensus.cc:504] T c2110d4ada2d45c2b07259374465a8df P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:22.494685 10392 catalog_manager.cc:5495] T 93e1b5837aff48d7907890c40dfcd0d0 P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:22.583999 10805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5d872c3620646e5ac33ceb95de2cbd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:22.583911 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df006163f72743b594baf1a3efd1c771" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:22.584441 10963 tablet_service.cc:1467] Processing CreateTablet for tablet 8aff53cfd4994c629a408ee8250280c5 (DEFAULT_TABLE table=default.Table_eb46cc92849a4a6db19755c3ea1268a3 [id=b84108d99dc44f1897c7bf236c56dac7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:22.585289 10963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8aff53cfd4994c629a408ee8250280c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:22.587227 10392 catalog_manager.cc:5495] T 197f61525b0c4f9da441d98701d231a3 P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:22.588851 11085 tablet_bootstrap.cc:492] T 197f61525b0c4f9da441d98701d231a3 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:22.589653 10392 catalog_manager.cc:5495] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:22.582935 11742 leader_election.cc:288] T c2110d4ada2d45c2b07259374465a8df P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
W20260925 09:57:22.592092 10918 leader_election.cc:341] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:22.592458 10918 leader_election.cc:302] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:22.600899 11812 tablet.cc:1722] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:22.604503 11579 raft_consensus.cc:2229] T 04fafb9dcf274806b11b2598e73853e9 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:22.605084 11579 raft_consensus.cc:2258] T 04fafb9dcf274806b11b2598e73853e9 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:22.608093 10959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10657a81058241fc988d37e5ea7ff8cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:22.608507 11751 raft_consensus.cc:2726] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:22.610049 10918 leader_election.cc:341] T df006163f72743b594baf1a3efd1c771 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:22.610450 10918 leader_election.cc:302] T df006163f72743b594baf1a3efd1c771 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:22.612289 11727 ts_tablet_manager.cc:1918] T 6161774f61ae48ab9764a05f2188a80b P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:22.612746 11727 log.cc:1196] T 6161774f61ae48ab9764a05f2188a80b P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/6161774f61ae48ab9764a05f2188a80b
I20260925 09:57:22.612774 11742 consensus_queue.cc:1035] T c588b0c05a1a4f3ea545066183e76502 P be97ab5d75934a648d4ab76341193ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:22.613546 11727 ts_tablet_manager.cc:1939] T 6161774f61ae48ab9764a05f2188a80b P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:22.614152 10834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2110d4ada2d45c2b07259374465a8df" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:22.614506 11742 raft_consensus.cc:2726] T df006163f72743b594baf1a3efd1c771 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:22.616350 10833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec1b78e61e8d46cb8b87f60786059658" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:22.563139 10621 consensus_peers.cc:487] T 197f61525b0c4f9da441d98701d231a3 P b8850818e2ee42a082802d0095c08d21 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260925 09:57:22.617063 10621 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 54016.7us
I20260925 09:57:22.618546 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 4be3f6e81ddf418fabc7b694a2e62f35 (table default.Table_1e7379199917430eb9e027fc7bb2a4f9 [id=c2d0336fd84c4dbe95472f458b813fe6]) successfully deleted
I20260925 09:57:22.618847 10392 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet b555eeba505740988c704df5c734e7c0 (table default.Table_48f368d811494119be0dc534e2fdcfbd [id=3e32026d175b4b92a3ee036f17e54b9d]). Expected version 1 got 0
I20260925 09:57:22.620119 10392 catalog_manager.cc:5495] T 3706253e53d841aeaa43889ead13af32 P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:22.628816 10819 tablet_service.cc:1467] Processing CreateTablet for tablet 37bab60afe7a4c758a08ef7c9c24aba0 (DEFAULT_TABLE table=default.Table_8b45818055ec40e386d43e2641ec6ed0 [id=38b800f8e1e948909d40cc976272fdd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:22.630501 10386 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 689ab2b6cf1640129f6db8b113cf2fc6
I20260925 09:57:22.630571 10819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37bab60afe7a4c758a08ef7c9c24aba0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:22.631846 10386 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:35140 (request call id 194) took 2014 ms. Trace:
I20260925 09:57:22.632117 10386 rpcz_store.cc:276] 0925 09:57:20.617614 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:20.907799 (+290185us) service_pool.cc:227] Handling call
0925 09:57:20.923613 (+ 15814us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:57:20.924250 (+ 637us) catalog_manager.cc:2752] Modifying in-memory table state
0925 09:57:20.924413 (+ 163us) catalog_manager.cc:2761] Locking tablets
0925 09:57:21.199754 (+275341us) catalog_manager.cc:2775] Removing table and tablets from system table
0925 09:57:21.415268 (+215514us) catalog_manager.cc:2793] Removing table from by-name map
0925 09:57:21.415543 (+ 275us) catalog_manager.cc:2809] Committing in-memory state
0925 09:57:22.630411 (+1214868us) catalog_manager.cc:2813] Aborting table tasks
0925 09:57:22.631787 (+ 1376us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:21.202063 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:21.202545 (+ 482us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:21.202579 (+ 34us) write_op.cc:435] Acquired schema lock
0925 09:57:21.202599 (+ 20us) tablet.cc:660] Decoding operations
0925 09:57:21.202970 (+ 371us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:21.203015 (+ 45us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:21.203045 (+ 30us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:21.203356 (+ 311us) tablet.cc:699] Row locks acquired
0925 09:57:21.203375 (+ 19us) write_op.cc:260] PREPARE: finished
0925 09:57:21.203519 (+ 144us) write_op.cc:270] Start()
0925 09:57:21.203617 (+ 98us) write_op.cc:276] Timestamp: P: 1790330241203475 usec, L: 0
0925 09:57:21.203638 (+ 21us) op_driver.cc:331] REPLICATION: starting
0925 09:57:21.211553 (+ 7915us) log.cc:842] Serialized 722 byte log entry
0925 09:57:21.212131 (+ 578us) spinlock_profiling.cc:243] Waited 7.37 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90471 00007fabf0b07198 00007fabf0f68ade 00007fabf0f682b8 00007fabf0f6cba1 00007fabf0f6cb59 00007fabf0f6cae9 00007fabf0f6cab1 00007fabf0f6bdad 0000000000580784 00000000005805b9
0925 09:57:21.410791 (+198660us) op_driver.cc:444] REPLICATION: finished
0925 09:57:21.412595 (+ 1804us) write_op.cc:301] APPLY: starting
0925 09:57:21.412637 (+ 42us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:21.412726 (+ 89us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:21.412745 (+ 19us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:21.412980 (+ 235us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:21.413114 (+ 134us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 09:57:21.413154 (+ 40us) write_op.cc:312] APPLY: finished
0925 09:57:21.413347 (+ 193us) log.cc:842] Serialized 29 byte log entry
0925 09:57:21.413489 (+ 142us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:21.413567 (+ 78us) write_op.cc:454] Released schema lock
0925 09:57:21.413603 (+ 36us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:21.413790 (+ 187us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:21.413823 (+ 33us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1690,"apply.run_cpu_time_us":1427,"apply.run_wall_time_us":1426,"num_ops":2,"prepare.queue_time_us":163,"prepare.run_cpu_time_us":2934,"prepare.run_wall_time_us":10197,"replication_time_us":207088,"spinlock_wait_cycles":16954240}]]}
I20260925 09:57:22.636137 10961 tablet_service.cc:1514] Processing DeleteTablet for tablet 689ab2b6cf1640129f6db8b113cf2fc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:20 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:22.637368 10386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_4f4ab1f0170a42e2826f1e3143803a6b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:22.638901 10386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4f4ab1f0170a42e2826f1e3143803a6b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:22.626464 11084 consensus_queue.cc:260] T e684798155d445bfb325a1b7c5c48753 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:22.640410 10815 tablet_service.cc:1467] Processing CreateTablet for tablet dba14a44a9de492886d629441606767e (DEFAULT_TABLE table=default.Table_bb742c618d96453aaabe897d54fc657b [id=68683893b91c4f0190a0050ecdc6d396]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:22.641533 10815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dba14a44a9de492886d629441606767e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:22.645699 10918 leader_election.cc:341] T c2110d4ada2d45c2b07259374465a8df P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:22.607193 11579 ts_tablet_manager.cc:1905] T 04fafb9dcf274806b11b2598e73853e9 P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:22.656747 11084 ts_tablet_manager.cc:1428] T e684798155d445bfb325a1b7c5c48753 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.135s user 0.007s sys 0.001s
I20260925 09:57:22.657743 11084 tablet_bootstrap.cc:492] T 545ac025f2cb469da847979011a0b3ce P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:22.660390 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 174) took 1868 ms. Trace:
I20260925 09:57:22.660910 11727 rpcz_store.cc:276] 0925 09:57:20.792418 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:20.792524 (+ 106us) service_pool.cc:227] Handling call
0925 09:57:22.005184 (+1212660us) ts_tablet_manager.cc:1145] Deleting tablet 6161774f61ae48ab9764a05f2188a80b
0925 09:57:22.005232 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:22.461331 (+456099us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:22.612049 (+150718us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:22.660323 (+ 48274us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1212208}
I20260925 09:57:22.662309 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 6161774f61ae48ab9764a05f2188a80b (table default.Table_64ba123898174e72a6884dbb9ad07de3 [id=7ee526720b2442119e0fc487b7c4fcd4]) successfully deleted
I20260925 09:57:22.663086 11727 tablet_replica.cc:330] T cf35f39bcd6a41128acb2a4936fcc316 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:22.664052 11084 tablet_bootstrap.cc:654] T 545ac025f2cb469da847979011a0b3ce P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:22.670009 10387 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: default.Table_b5c66e748a8741f78316bd5264f0f186 [id=aed8a2ddc51545ad9cec4d272dcf3daf] alter complete (version 3)
I20260925 09:57:22.692689 10806 tablet_service.cc:1467] Processing CreateTablet for tablet 8aff53cfd4994c629a408ee8250280c5 (DEFAULT_TABLE table=default.Table_eb46cc92849a4a6db19755c3ea1268a3 [id=b84108d99dc44f1897c7bf236c56dac7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:22.695801 10816 tablet_service.cc:1514] Processing DeleteTablet for tablet 689ab2b6cf1640129f6db8b113cf2fc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:20 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:22.701548 10834 raft_consensus.cc:3037] T cf35f39bcd6a41128acb2a4936fcc316 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:22.707020 10960 tablet_service.cc:1514] Processing DeleteTablet for tablet 4be3f6e81ddf418fabc7b694a2e62f35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:18 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:22.712868 10960 tablet_service.cc:1514] Processing DeleteTablet for tablet 752105c1514f4c1db2a11b95b413b4f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:14 UTC) from {username='slave'} at 127.0.0.1:53560
W20260925 09:57:22.717437 10385 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.3:41469 (request call id 19) took 3936 ms (client timeout 1000 ms). Trace:
I20260925 09:57:22.719682 10960 tablet_service.cc:1514] Processing DeleteTablet for tablet c07c6205d1284a6fbb369c010a226da7 with delete_type TABLET_DATA_DELETED (replaced by tablet 4e3120347c8e4a3eab57d6562ae40e8d) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:22.721230 10963 tablet_service.cc:1514] Processing DeleteTablet for tablet c16e096038084b2b9148390b6e773294 with delete_type TABLET_DATA_DELETED (replaced by tablet d5956be0da6441c79723e72f12dfe54a) from {username='slave'} at 127.0.0.1:53560
W20260925 09:57:22.723105 10385 rpcz_store.cc:269] 0925 09:57:18.780617 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:18.780741 (+ 124us) service_pool.cc:227] Handling call
0925 09:57:18.816766 (+ 36025us) spinlock_profiling.cc:243] Waited 29.5 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2ad9c3e 00007fabf0610fb4 00007fabf0610f42 00007fabf0610e71 00007fabf0610dec 00007fabf06100a2 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:19.645910 (+829144us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:19.645935 (+ 25us) write_op.cc:454] Released schema lock
0925 09:57:22.717370 (+3071435us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:19.623579 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:19.623932 (+ 353us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:19.623955 (+ 23us) write_op.cc:435] Acquired schema lock
0925 09:57:19.623967 (+ 12us) tablet.cc:660] Decoding operations
0925 09:57:19.624215 (+ 248us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:19.624255 (+ 40us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:19.624277 (+ 22us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:19.624410 (+ 133us) tablet.cc:699] Row locks acquired
0925 09:57:19.624422 (+ 12us) write_op.cc:260] PREPARE: finished
0925 09:57:19.624527 (+ 105us) write_op.cc:270] Start()
0925 09:57:19.624605 (+ 78us) write_op.cc:276] Timestamp: P: 1790330239624452 usec, L: 0
0925 09:57:19.624619 (+ 14us) op_driver.cc:331] REPLICATION: starting
0925 09:57:19.625189 (+ 570us) log.cc:842] Serialized 524 byte log entry
0925 09:57:19.633407 (+ 8218us) op_driver.cc:444] REPLICATION: finished
0925 09:57:19.633760 (+ 353us) write_op.cc:301] APPLY: starting
0925 09:57:19.633808 (+ 48us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:19.633897 (+ 89us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:19.633985 (+ 88us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:19.634160 (+ 175us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:19.634325 (+ 165us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:19.634391 (+ 66us) write_op.cc:312] APPLY: finished
0925 09:57:19.634570 (+ 179us) log.cc:842] Serialized 23 byte log entry
0925 09:57:19.634691 (+ 121us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:19.634762 (+ 71us) write_op.cc:454] Released schema lock
0925 09:57:19.634793 (+ 31us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":43110,"reported_tablets":45,"spinlock_wait_cycles":67912064,"child_traces":[["op",{"apply.queue_time_us":204,"apply.run_cpu_time_us":1202,"apply.run_wall_time_us":1202,"num_ops":1,"prepare.queue_time_us":6442,"prepare.run_cpu_time_us":1872,"prepare.run_wall_time_us":1872,"raft.queue_time_us":1143,"raft.run_cpu_time_us":613,"raft.run_wall_time_us":612,"replication_time_us":8697}]]}
I20260925 09:57:22.727833 11813 tablet.cc:1722] T b555eeba505740988c704df5c734e7c0 P b8850818e2ee42a082802d0095c08d21: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:22.744956 10960 tablet_service.cc:1514] Processing DeleteTablet for tablet 04fafb9dcf274806b11b2598e73853e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:19 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:22.752665 10963 tablet_service.cc:1514] Processing DeleteTablet for tablet a44d9f2f7e3e48aeb0f31fa29ae1988a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:18 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:22.753485 10958 tablet_service.cc:1514] Processing DeleteTablet for tablet 977a59175e5b43eeb6f23b938afd00a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:18 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:22.753827 10806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8aff53cfd4994c629a408ee8250280c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:22.761083 11084 tablet_bootstrap.cc:492] T 545ac025f2cb469da847979011a0b3ce P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:22.729323 11709 log.cc:1196] T 89a143a5623e4ef38027381a56c95227 P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/89a143a5623e4ef38027381a56c95227
W20260925 09:57:22.786970 10621 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 169292us
W20260925 09:57:22.787894 10621 leader_election.cc:341] T 3d652a76d96f4dbeafde15f3c16534a6 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:22.788735 10621 leader_election.cc:341] T 3d652a76d96f4dbeafde15f3c16534a6 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:22.792147 10621 leader_election.cc:341] T ec1b78e61e8d46cb8b87f60786059658 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:22.806272 11084 ts_tablet_manager.cc:1397] T 545ac025f2cb469da847979011a0b3ce P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.149s user 0.010s sys 0.002s
I20260925 09:57:22.793282 11605 raft_consensus.cc:482] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:22.764705 11801 raft_consensus.cc:504] T ec1b78e61e8d46cb8b87f60786059658 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:22.796271 11709 ts_tablet_manager.cc:1939] T 89a143a5623e4ef38027381a56c95227 P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:22.807641 11605 raft_consensus.cc:504] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:22.808830 11709 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 177) took 1860 ms. Trace:
I20260925 09:57:22.808962 11801 leader_election.cc:288] T ec1b78e61e8d46cb8b87f60786059658 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:22.809706 11605 leader_election.cc:288] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:22.809188 11709 rpcz_store.cc:276] 0925 09:57:20.948526 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:20.948701 (+ 175us) service_pool.cc:227] Handling call
0925 09:57:22.094909 (+1146208us) ts_tablet_manager.cc:1145] Deleting tablet 89a143a5623e4ef38027381a56c95227
0925 09:57:22.094972 (+ 63us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:22.463473 (+368501us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:22.561387 (+ 97914us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:22.808761 (+247374us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":36713,"tablet-delete.queue_time_us":1145520}
I20260925 09:57:22.810509 11709 tablet_replica.cc:330] T ec1b78e61e8d46cb8b87f60786059658 P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:22.813256 11084 raft_consensus.cc:348] T 545ac025f2cb469da847979011a0b3ce P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:22.814061 11084 raft_consensus.cc:374] T 545ac025f2cb469da847979011a0b3ce P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:22.814401 11084 raft_consensus.cc:729] T 545ac025f2cb469da847979011a0b3ce P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:22.816093 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a21909e2e5794f57971f4e42795235c7" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:22.820891 10682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2110d4ada2d45c2b07259374465a8df" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:22.822535 10918 leader_election.cc:341] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:22.824769 10807 tablet_service.cc:1467] Processing CreateTablet for tablet 10657a81058241fc988d37e5ea7ff8cf (DEFAULT_TABLE table=default.Table_64ba123898174e72a6884dbb9ad07de3 [id=7ee526720b2442119e0fc487b7c4fcd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:22.825997 10807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10657a81058241fc988d37e5ea7ff8cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:22.830117 11727 raft_consensus.cc:2229] T cf35f39bcd6a41128acb2a4936fcc316 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:22.830901 11727 raft_consensus.cc:2258] T cf35f39bcd6a41128acb2a4936fcc316 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:22.844533 11727 ts_tablet_manager.cc:1905] T cf35f39bcd6a41128acb2a4936fcc316 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:22.811553 11709 raft_consensus.cc:2229] T ec1b78e61e8d46cb8b87f60786059658 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:22.852058 10664 tablet_service.cc:1467] Processing CreateTablet for tablet 8aff53cfd4994c629a408ee8250280c5 (DEFAULT_TABLE table=default.Table_eb46cc92849a4a6db19755c3ea1268a3 [id=b84108d99dc44f1897c7bf236c56dac7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:22.853204 10664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8aff53cfd4994c629a408ee8250280c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:22.853698 10658 tablet_service.cc:1514] Processing DeleteTablet for tablet 689ab2b6cf1640129f6db8b113cf2fc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:20 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:22.851099 10665 tablet_service.cc:1467] Processing CreateTablet for tablet dba14a44a9de492886d629441606767e (DEFAULT_TABLE table=default.Table_bb742c618d96453aaabe897d54fc657b [id=68683893b91c4f0190a0050ecdc6d396]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:22.856071 10665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dba14a44a9de492886d629441606767e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:22.862946 11709 raft_consensus.cc:2258] T ec1b78e61e8d46cb8b87f60786059658 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 09:57:22.863354 10619 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 301991us
I20260925 09:57:22.865444 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec1b78e61e8d46cb8b87f60786059658" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:22.865984 10984 raft_consensus.cc:3037] T ec1b78e61e8d46cb8b87f60786059658 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:22.873448 10984 raft_consensus.cc:2454] T ec1b78e61e8d46cb8b87f60786059658 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:22.874974 10666 tablet_service.cc:1514] Processing DeleteTablet for tablet 04fafb9dcf274806b11b2598e73853e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:19 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:22.878626 10666 tablet_service.cc:1514] Processing DeleteTablet for tablet cf35f39bcd6a41128acb2a4936fcc316 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:16 UTC) from {username='slave'} at 127.0.0.1:37876
W20260925 09:57:22.882355 10919 leader_election.cc:341] T c2110d4ada2d45c2b07259374465a8df P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:22.882704 10919 leader_election.cc:302] T c2110d4ada2d45c2b07259374465a8df P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:22.883301 11605 raft_consensus.cc:2726] T c2110d4ada2d45c2b07259374465a8df P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:22.727268 10387 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: default.Table_ce09d3a8d8e84973894f33b18bbfdb22 [id=6851500c7dba44f1bf353bb79bafd97a] alter complete (version 2)
W20260925 09:57:22.748909 10388 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.3:41469 (request call id 21) took 1638 ms (client timeout 1000 ms). Trace:
I20260925 09:57:22.884516 11742 consensus_queue.cc:1035] T c588b0c05a1a4f3ea545066183e76502 P be97ab5d75934a648d4ab76341193ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:22.816541 10392 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: default.Table_46e4c497898b454c85b7c2a299ebf3d9 [id=80b48dd413e3446eb7ac65e5f957b5dc] alter complete (version 2)
I20260925 09:57:22.818198 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 89a143a5623e4ef38027381a56c95227 (table default.Table_bb742c618d96453aaabe897d54fc657b [id=68683893b91c4f0190a0050ecdc6d396]) successfully deleted
W20260925 09:57:22.885558 10380 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 67372.3us
W20260925 09:57:22.884850 10388 rpcz_store.cc:269] 0925 09:57:21.110876 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:21.130269 (+ 19393us) service_pool.cc:227] Handling call
0925 09:57:22.748845 (+1618576us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:22.012521 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:22.012883 (+ 362us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:22.012905 (+ 22us) write_op.cc:435] Acquired schema lock
0925 09:57:22.012918 (+ 13us) tablet.cc:660] Decoding operations
0925 09:57:22.013208 (+ 290us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:22.013243 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:22.013264 (+ 21us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:22.013521 (+ 257us) tablet.cc:699] Row locks acquired
0925 09:57:22.013536 (+ 15us) write_op.cc:260] PREPARE: finished
0925 09:57:22.013639 (+ 103us) write_op.cc:270] Start()
0925 09:57:22.013718 (+ 79us) write_op.cc:276] Timestamp: P: 1790330242013579 usec, L: 0
0925 09:57:22.013733 (+ 15us) op_driver.cc:331] REPLICATION: starting
0925 09:57:22.014229 (+ 496us) log.cc:842] Serialized 895 byte log entry
0925 09:57:22.087628 (+ 73399us) op_driver.cc:444] REPLICATION: finished
0925 09:57:22.107117 (+ 19489us) write_op.cc:301] APPLY: starting
0925 09:57:22.107172 (+ 55us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:22.107285 (+ 113us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:22.107417 (+ 132us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:22.107668 (+ 251us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:22.107800 (+ 132us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 09:57:22.107844 (+ 44us) write_op.cc:312] APPLY: finished
0925 09:57:22.108022 (+ 178us) log.cc:842] Serialized 29 byte log entry
0925 09:57:22.115343 (+ 7321us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:22.115440 (+ 97us) write_op.cc:454] Released schema lock
0925 09:57:22.115473 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:22.115661 (+ 188us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:22.118542 (+ 2881us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":92,"reported_tablets":53,"child_traces":[["op",{"apply.queue_time_us":19313,"apply.run_cpu_time_us":1695,"apply.run_wall_time_us":11665,"num_ops":2,"prepare.queue_time_us":2911,"prepare.run_cpu_time_us":1896,"prepare.run_wall_time_us":1896,"replication_time_us":73814}]]}
I20260925 09:57:22.888055 10388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_4066ae2ab4534ff58ac51f74bef05db9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:22.889634 10388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4066ae2ab4534ff58ac51f74bef05db9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 09:57:22.896855 10387 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.2:40761 (request call id 28) took 1733 ms (client timeout 1000 ms). Trace:
I20260925 09:57:22.900055 10668 tablet_service.cc:1514] Processing DeleteTablet for tablet 89a143a5623e4ef38027381a56c95227 with delete_type TABLET_DATA_DELETED (replaced by tablet dba14a44a9de492886d629441606767e) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:22.865547 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a21909e2e5794f57971f4e42795235c7" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:22.906613 10661 tablet_service.cc:1514] Processing DeleteTablet for tablet 752105c1514f4c1db2a11b95b413b4f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:14 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:22.841370 11085 tablet_bootstrap.cc:700] T 197f61525b0c4f9da441d98701d231a3 P c43461c4ee854ae1ac52b873ccada318: Time spent opening tablet: real 0.214s user 0.002s sys 0.000s
I20260925 09:57:22.908025 11085 tablet_bootstrap.cc:654] T 197f61525b0c4f9da441d98701d231a3 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:22.908027 11812 tablet.cc:1722] T 42ec275c905343c4be1b72dc2bd0f6e3 P be97ab5d75934a648d4ab76341193ec0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260925 09:57:22.910951 11825 log.cc:924] Time spent T c588b0c05a1a4f3ea545066183e76502 P c43461c4ee854ae1ac52b873ccada318: Append to log took a long time: real 0.069s user 0.001s sys 0.000s
I20260925 09:57:22.913970 10659 tablet_service.cc:1514] Processing DeleteTablet for tablet 41df75fe4b8e4856bdbb5ead3cf4fd24 with delete_type TABLET_DATA_DELETED (replaced by tablet c7cd1d6a603642afa2c8c737079961b5) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:22.926141 11727 ts_tablet_manager.cc:1918] T cf35f39bcd6a41128acb2a4936fcc316 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:22.926566 11727 log.cc:1196] T cf35f39bcd6a41128acb2a4936fcc316 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/cf35f39bcd6a41128acb2a4936fcc316
I20260925 09:57:22.928675 11709 ts_tablet_manager.cc:1905] T ec1b78e61e8d46cb8b87f60786059658 P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:22.932170 10666 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d55154ceb064f22bd6f37c8ae5dce74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:12 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:22.937170 11727 ts_tablet_manager.cc:1939] T cf35f39bcd6a41128acb2a4936fcc316 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:22.939467 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 172) took 2152 ms. Trace:
I20260925 09:57:22.939970 11727 rpcz_store.cc:276] 0925 09:57:20.786848 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:20.794941 (+ 8093us) service_pool.cc:227] Handling call
0925 09:57:22.662935 (+1867994us) ts_tablet_manager.cc:1145] Deleting tablet cf35f39bcd6a41128acb2a4936fcc316
0925 09:57:22.662982 (+ 47us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:22.897446 (+234464us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:22.925806 (+ 28360us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:22.939366 (+ 13560us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":70422,"tablet-delete.queue_time_us":1866759}
W20260925 09:57:22.941161 11810 log.cc:924] Time spent T 197f61525b0c4f9da441d98701d231a3 P b8850818e2ee42a082802d0095c08d21: Append to log took a long time: real 0.078s user 0.002s sys 0.001s
I20260925 09:57:22.874599 10390 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:35140 (request call id 198) took 1733 ms. Trace:
I20260925 09:57:22.879323 10389 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:35140 (request call id 191) took 2454 ms. Trace:
I20260925 09:57:22.949414 10389 rpcz_store.cc:276] 0925 09:57:20.475889 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:20.569443 (+ 93554us) service_pool.cc:227] Handling call
0925 09:57:20.571147 (+ 1704us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:57:20.627717 (+ 56570us) catalog_manager.cc:3616] Apply alter schema
0925 09:57:20.628689 (+ 972us) catalog_manager.cc:3639] Acquired catalog manager lock
0925 09:57:20.628941 (+ 252us) catalog_manager.cc:3786] Updating metadata on disk
0925 09:57:21.011637 (+382696us) catalog_manager.cc:3817] Committing alterations to in-memory state
0925 09:57:22.879251 (+1867614us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:20.630470 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:20.630825 (+ 355us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:20.630851 (+ 26us) write_op.cc:435] Acquired schema lock
0925 09:57:20.630863 (+ 12us) tablet.cc:660] Decoding operations
0925 09:57:20.631151 (+ 288us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:20.631200 (+ 49us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:20.631235 (+ 35us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:20.631440 (+ 205us) tablet.cc:699] Row locks acquired
0925 09:57:20.631460 (+ 20us) write_op.cc:260] PREPARE: finished
0925 09:57:20.639992 (+ 8532us) write_op.cc:270] Start()
0925 09:57:20.640069 (+ 77us) write_op.cc:276] Timestamp: P: 1790330240639897 usec, L: 0
0925 09:57:20.640087 (+ 18us) op_driver.cc:331] REPLICATION: starting
0925 09:57:20.640537 (+ 450us) log.cc:842] Serialized 405 byte log entry
0925 09:57:21.009832 (+369295us) op_driver.cc:444] REPLICATION: finished
0925 09:57:21.010066 (+ 234us) write_op.cc:301] APPLY: starting
0925 09:57:21.010115 (+ 49us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:21.010210 (+ 95us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:21.010305 (+ 95us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:21.010451 (+ 146us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:21.010573 (+ 122us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:21.010614 (+ 41us) write_op.cc:312] APPLY: finished
0925 09:57:21.010780 (+ 166us) log.cc:842] Serialized 23 byte log entry
0925 09:57:21.011355 (+ 575us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:21.011442 (+ 87us) write_op.cc:454] Released schema lock
0925 09:57:21.011473 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:21.011634 (+ 161us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:21.011657 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":452,"child_traces":[["op",{"apply.queue_time_us":96,"apply.run_cpu_time_us":1306,"apply.run_wall_time_us":1729,"num_ops":1,"prepare.queue_time_us":342,"prepare.run_cpu_time_us":1970,"prepare.run_wall_time_us":12878,"raft.queue_time_us":230,"raft.run_cpu_time_us":11,"raft.run_wall_time_us":10,"replication_time_us":369665}]]}
I20260925 09:57:22.947571 10390 rpcz_store.cc:276] 0925 09:57:21.141321 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:21.141458 (+ 137us) service_pool.cc:227] Handling call
0925 09:57:21.142417 (+ 959us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:57:21.143060 (+ 643us) catalog_manager.cc:3616] Apply alter schema
0925 09:57:21.143693 (+ 633us) catalog_manager.cc:3639] Acquired catalog manager lock
0925 09:57:21.143897 (+ 204us) catalog_manager.cc:3786] Updating metadata on disk
0925 09:57:21.415034 (+271137us) catalog_manager.cc:3817] Committing alterations to in-memory state
0925 09:57:22.874520 (+1459486us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:21.147166 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:21.147589 (+ 423us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:21.147625 (+ 36us) write_op.cc:435] Acquired schema lock
0925 09:57:21.147645 (+ 20us) tablet.cc:660] Decoding operations
0925 09:57:21.147984 (+ 339us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:21.148035 (+ 51us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:21.148067 (+ 32us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:21.148250 (+ 183us) tablet.cc:699] Row locks acquired
0925 09:57:21.148280 (+ 30us) write_op.cc:260] PREPARE: finished
0925 09:57:21.148418 (+ 138us) write_op.cc:270] Start()
0925 09:57:21.148521 (+ 103us) write_op.cc:276] Timestamp: P: 1790330241148331 usec, L: 0
0925 09:57:21.148543 (+ 22us) op_driver.cc:331] REPLICATION: starting
0925 09:57:21.148986 (+ 443us) log.cc:842] Serialized 405 byte log entry
0925 09:57:21.410582 (+261596us) op_driver.cc:444] REPLICATION: finished
0925 09:57:21.411113 (+ 531us) write_op.cc:301] APPLY: starting
0925 09:57:21.411168 (+ 55us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:21.411263 (+ 95us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:21.411370 (+ 107us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:21.411550 (+ 180us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:21.411675 (+ 125us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:21.411716 (+ 41us) write_op.cc:312] APPLY: finished
0925 09:57:21.411898 (+ 182us) log.cc:842] Serialized 23 byte log entry
0925 09:57:21.412038 (+ 140us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:21.412098 (+ 60us) write_op.cc:454] Released schema lock
0925 09:57:21.412129 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:21.412294 (+ 165us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:21.412328 (+ 34us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":381,"apply.run_cpu_time_us":1373,"apply.run_wall_time_us":1371,"num_ops":1,"prepare.queue_time_us":2068,"prepare.run_cpu_time_us":1996,"prepare.run_wall_time_us":1998,"replication_time_us":261955}]]}
I20260925 09:57:22.958225 10390 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_bbc3c50c06634d119ae96dcf649f44e7" } modify_external_catalogs: true
W20260925 09:57:22.902621 10392 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.1:56563 (request call id 31) took 895 ms (client timeout 1000 ms). Trace:
I20260925 09:57:22.941226 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet cf35f39bcd6a41128acb2a4936fcc316 (table default.Table_fea25102a7f84d58927645a0feae51e3 [id=61a4c50b2c6e43c1a471f63b59d218cb]) successfully deleted
W20260925 09:57:22.958882 10919 leader_election.cc:341] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:22.959254 10919 leader_election.cc:302] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:22.960042 11605 raft_consensus.cc:2726] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:22.871102 10384 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.1:56563 (request call id 30) took 2499 ms (client timeout 1000 ms). Trace:
W20260925 09:57:22.973632 10384 rpcz_store.cc:269] 0925 09:57:20.371195 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:20.408028 (+ 36833us) service_pool.cc:227] Handling call
0925 09:57:22.871033 (+2463005us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":134076,"reported_tablets":25,"spinlock_wait_cycles":43264}
I20260925 09:57:22.975706 10384 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_fad50b13f9024bf8b4df345241f8c32b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:22.977130 10384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fad50b13f9024bf8b4df345241f8c32b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 09:57:22.978940 10391 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.2:40761 (request call id 27) took 2881 ms (client timeout 1000 ms). Trace:
W20260925 09:57:22.979252 10391 rpcz_store.cc:269] 0925 09:57:20.097339 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:20.220751 (+123412us) service_pool.cc:227] Handling call
0925 09:57:22.978880 (+2758129us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:20.520823 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:20.521257 (+ 434us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:20.521286 (+ 29us) write_op.cc:435] Acquired schema lock
0925 09:57:20.521304 (+ 18us) tablet.cc:660] Decoding operations
0925 09:57:20.521574 (+ 270us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:20.521616 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:20.521647 (+ 31us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:20.521817 (+ 170us) tablet.cc:699] Row locks acquired
0925 09:57:20.521834 (+ 17us) write_op.cc:260] PREPARE: finished
0925 09:57:20.521970 (+ 136us) write_op.cc:270] Start()
0925 09:57:20.522068 (+ 98us) write_op.cc:276] Timestamp: P: 1790330240521894 usec, L: 0
0925 09:57:20.522088 (+ 20us) op_driver.cc:331] REPLICATION: starting
0925 09:57:20.522613 (+ 525us) log.cc:842] Serialized 524 byte log entry
0925 09:57:20.586894 (+ 64281us) op_driver.cc:444] REPLICATION: finished
0925 09:57:20.588266 (+ 1372us) write_op.cc:301] APPLY: starting
0925 09:57:20.588319 (+ 53us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:20.588401 (+ 82us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:20.588480 (+ 79us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:20.588641 (+ 161us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:20.588785 (+ 144us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:20.588832 (+ 47us) write_op.cc:312] APPLY: finished
0925 09:57:20.588988 (+ 156us) log.cc:842] Serialized 23 byte log entry
0925 09:57:20.589114 (+ 126us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:20.589169 (+ 55us) write_op.cc:454] Released schema lock
0925 09:57:20.589195 (+ 26us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:20.589345 (+ 150us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:20.589368 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":9376,"reported_tablets":26,"child_traces":[["op",{"apply.queue_time_us":1228,"apply.run_cpu_time_us":1243,"apply.run_wall_time_us":1243,"num_ops":1,"prepare.queue_time_us":102,"prepare.run_cpu_time_us":1973,"prepare.run_wall_time_us":3181,"replication_time_us":64736}]]}
I20260925 09:57:22.969067 11727 tablet_replica.cc:330] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:22.986878 11727 raft_consensus.cc:2229] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:22.987538 11727 raft_consensus.cc:2258] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:22.999820 10389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_908fa06a45334bf7a6700063843028b1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:23.002167 10389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_908fa06a45334bf7a6700063843028b1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:23.011441 10833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec1b78e61e8d46cb8b87f60786059658" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:23.013478 10806 tablet_service.cc:1514] Processing DeleteTablet for tablet 4be3f6e81ddf418fabc7b694a2e62f35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:18 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:23.014478 10806 tablet_service.cc:1514] Processing DeleteTablet for tablet 41df75fe4b8e4856bdbb5ead3cf4fd24 with delete_type TABLET_DATA_DELETED (replaced by tablet c7cd1d6a603642afa2c8c737079961b5) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:23.015405 10806 tablet_service.cc:1514] Processing DeleteTablet for tablet c07c6205d1284a6fbb369c010a226da7 with delete_type TABLET_DATA_DELETED (replaced by tablet 4e3120347c8e4a3eab57d6562ae40e8d) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:23.018265 10805 tablet_service.cc:1514] Processing DeleteTablet for tablet c16e096038084b2b9148390b6e773294 with delete_type TABLET_DATA_DELETED (replaced by tablet d5956be0da6441c79723e72f12dfe54a) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:23.018883 10810 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d55154ceb064f22bd6f37c8ae5dce74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:12 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:23.019996 10385 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_489e2c74ed6249ce9cf54541e0a491cd" } modify_external_catalogs: true
I20260925 09:57:22.984293 11831 tablet.cc:1722] T 42ec275c905343c4be1b72dc2bd0f6e3 P c43461c4ee854ae1ac52b873ccada318: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:23.026790 10815 tablet_service.cc:1514] Processing DeleteTablet for tablet 752105c1514f4c1db2a11b95b413b4f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:14 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:23.036336 11085 tablet_bootstrap.cc:492] T 197f61525b0c4f9da441d98701d231a3 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:23.036707 11085 ts_tablet_manager.cc:1397] T 197f61525b0c4f9da441d98701d231a3 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.448s user 0.010s sys 0.001s
I20260925 09:57:23.038677 11085 raft_consensus.cc:348] T 197f61525b0c4f9da441d98701d231a3 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:23.043505 11727 ts_tablet_manager.cc:1905] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 09:57:22.943459 10387 rpcz_store.cc:269] 0925 09:57:21.163174 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:21.347445 (+184271us) service_pool.cc:227] Handling call
0925 09:57:22.896793 (+1549348us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:22.178232 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:22.178588 (+ 356us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:22.178620 (+ 32us) write_op.cc:435] Acquired schema lock
0925 09:57:22.178639 (+ 19us) tablet.cc:660] Decoding operations
0925 09:57:22.179107 (+ 468us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:22.179144 (+ 37us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:22.179164 (+ 20us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:22.179450 (+ 286us) tablet.cc:699] Row locks acquired
0925 09:57:22.179464 (+ 14us) write_op.cc:260] PREPARE: finished
0925 09:57:22.179562 (+ 98us) write_op.cc:270] Start()
0925 09:57:22.179641 (+ 79us) write_op.cc:276] Timestamp: P: 1790330242179489 usec, L: 0
0925 09:57:22.179663 (+ 22us) op_driver.cc:331] REPLICATION: starting
0925 09:57:22.190615 (+ 10952us) log.cc:842] Serialized 895 byte log entry
0925 09:57:22.201257 (+ 10642us) op_driver.cc:444] REPLICATION: finished
0925 09:57:22.202006 (+ 749us) write_op.cc:301] APPLY: starting
0925 09:57:22.202059 (+ 53us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:22.202166 (+ 107us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:22.202261 (+ 95us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:22.202492 (+ 231us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:22.202627 (+ 135us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 09:57:22.202671 (+ 44us) write_op.cc:312] APPLY: finished
0925 09:57:22.202851 (+ 180us) log.cc:842] Serialized 29 byte log entry
0925 09:57:22.203528 (+ 677us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:22.203597 (+ 69us) write_op.cc:454] Released schema lock
0925 09:57:22.203639 (+ 42us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:22.203835 (+ 196us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:22.203872 (+ 37us) write_op.cc:454] Released schema lock
Related trace 'op':
0925 09:57:22.245234 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:22.245715 (+ 481us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:22.245745 (+ 30us) write_op.cc:435] Acquired schema lock
0925 09:57:22.245759 (+ 14us) tablet.cc:660] Decoding operations
0925 09:57:22.246034 (+ 275us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:22.246082 (+ 48us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:22.246107 (+ 25us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:22.246260 (+ 153us) tablet.cc:699] Row locks acquired
0925 09:57:22.246277 (+ 17us) write_op.cc:260] PREPARE: finished
0925 09:57:22.246386 (+ 109us) write_op.cc:270] Start()
0925 09:57:22.246484 (+ 98us) write_op.cc:276] Timestamp: P: 1790330242246338 usec, L: 0
0925 09:57:22.246501 (+ 17us) op_driver.cc:331] REPLICATION: starting
0925 09:57:22.247674 (+ 1173us) log.cc:842] Serialized 356 byte log entry
0925 09:57:22.256983 (+ 9309us) op_driver.cc:444] REPLICATION: finished
0925 09:57:22.257903 (+ 920us) write_op.cc:301] APPLY: starting
0925 09:57:22.257962 (+ 59us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:22.258066 (+ 104us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:22.258160 (+ 94us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:22.258346 (+ 186us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:22.258491 (+ 145us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:22.258536 (+ 45us) write_op.cc:312] APPLY: finished
0925 09:57:22.258749 (+ 213us) log.cc:842] Serialized 23 byte log entry
0925 09:57:22.259475 (+ 726us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:22.260157 (+ 682us) write_op.cc:454] Released schema lock
0925 09:57:22.260200 (+ 43us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:22.260398 (+ 198us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:22.260430 (+ 32us) write_op.cc:454] Released schema lock
Related trace 'op':
0925 09:57:22.680314 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:22.680700 (+ 386us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:22.680731 (+ 31us) write_op.cc:435] Acquired schema lock
0925 09:57:22.680744 (+ 13us) tablet.cc:660] Decoding operations
0925 09:57:22.680962 (+ 218us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:22.680995 (+ 33us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:22.681016 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:22.681136 (+ 120us) tablet.cc:699] Row locks acquired
0925 09:57:22.681147 (+ 11us) write_op.cc:260] PREPARE: finished
0925 09:57:22.681249 (+ 102us) write_op.cc:270] Start()
0925 09:57:22.681326 (+ 77us) write_op.cc:276] Timestamp: P: 1790330242681169 usec, L: 0
0925 09:57:22.681343 (+ 17us) op_driver.cc:331] REPLICATION: starting
0925 09:57:22.681779 (+ 436us) log.cc:842] Serialized 356 byte log entry
0925 09:57:22.722116 (+ 40337us) op_driver.cc:444] REPLICATION: finished
0925 09:57:22.724387 (+ 2271us) write_op.cc:301] APPLY: starting
0925 09:57:22.724437 (+ 50us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:22.724539 (+ 102us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:22.724626 (+ 87us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:22.724765 (+ 139us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:22.724903 (+ 138us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:22.724940 (+ 37us) write_op.cc:312] APPLY: finished
0925 09:57:22.725079 (+ 139us) log.cc:842] Serialized 23 byte log entry
0925 09:57:22.725182 (+ 103us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:22.725231 (+ 49us) write_op.cc:454] Released schema lock
0925 09:57:22.725264 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:22.725406 (+ 142us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:22.725432 (+ 26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":12401,"reported_tablets":26,"child_traces":[["op",{"apply.queue_time_us":593,"apply.run_cpu_time_us":1533,"apply.run_wall_time_us":2065,"num_ops":2,"prepare.queue_time_us":1160,"prepare.run_cpu_time_us":2131,"prepare.run_wall_time_us":15114,"raft.queue_time_us":135,"raft.run_cpu_time_us":658,"raft.run_wall_time_us":1025,"replication_time_us":21514}],["op",{"apply.queue_time_us":745,"apply.run_cpu_time_us":1606,"apply.run_wall_time_us":2713,"num_ops":1,"prepare.queue_time_us":250,"prepare.run_cpu_time_us":2213,"prepare.run_wall_time_us":6400,"raft.queue_time_us":161,"raft.run_cpu_time_us":631,"raft.run_wall_time_us":1026,"replication_time_us":10406}],["op",{"apply.queue_time_us":2168,"apply.run_cpu_time_us":1165,"apply.run_wall_time_us":1164,"num_ops":1,"prepare.queue_time_us":165,"prepare.run_cpu_time_us":1716,"prepare.run_wall_time_us":4964,"replication_time_us":40714,"thread_start_us":30048,"threads_started":1}]]}
W20260925 09:57:22.959249 10392 rpcz_store.cc:269] 0925 09:57:22.006946 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:22.051466 (+ 44520us) service_pool.cc:227] Handling call
0925 09:57:22.097295 (+ 45829us) spinlock_profiling.cc:243] Waited 34.1 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2ad9c3e 00007fabf0610fb4 00007fabf0610f42 00007fabf0610e71 00007fabf0610dec 00007fabf06100a2 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:22.902566 (+805271us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:22.674775 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:22.675099 (+ 324us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:22.675119 (+ 20us) write_op.cc:435] Acquired schema lock
0925 09:57:22.675130 (+ 11us) tablet.cc:660] Decoding operations
0925 09:57:22.675610 (+ 480us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:22.675643 (+ 33us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:22.675666 (+ 23us) tablet.cc:683] Acquiring locks for 5 operations
0925 09:57:22.676064 (+ 398us) tablet.cc:699] Row locks acquired
0925 09:57:22.676078 (+ 14us) write_op.cc:260] PREPARE: finished
0925 09:57:22.676182 (+ 104us) write_op.cc:270] Start()
0925 09:57:22.676258 (+ 76us) write_op.cc:276] Timestamp: P: 1790330242676100 usec, L: 0
0925 09:57:22.676276 (+ 18us) op_driver.cc:331] REPLICATION: starting
0925 09:57:22.677211 (+ 935us) log.cc:842] Serialized 2009 byte log entry
0925 09:57:22.721908 (+ 44697us) op_driver.cc:444] REPLICATION: finished
0925 09:57:22.722473 (+ 565us) write_op.cc:301] APPLY: starting
0925 09:57:22.722524 (+ 51us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:22.722661 (+ 137us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:22.722773 (+ 112us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:22.723231 (+ 458us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:22.723371 (+ 140us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0925 09:57:22.723441 (+ 70us) write_op.cc:312] APPLY: finished
0925 09:57:22.723636 (+ 195us) log.cc:842] Serialized 47 byte log entry
0925 09:57:22.723768 (+ 132us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:22.723851 (+ 83us) write_op.cc:454] Released schema lock
0925 09:57:22.723889 (+ 38us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:22.724042 (+ 153us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:22.724066 (+ 24us) write_op.cc:454] Released schema lock
Related trace 'op':
0925 09:57:22.795890 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:22.796291 (+ 401us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:22.796316 (+ 25us) write_op.cc:435] Acquired schema lock
0925 09:57:22.796329 (+ 13us) tablet.cc:660] Decoding operations
0925 09:57:22.796596 (+ 267us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:22.796642 (+ 46us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:22.796665 (+ 23us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:22.796808 (+ 143us) tablet.cc:699] Row locks acquired
0925 09:57:22.796822 (+ 14us) write_op.cc:260] PREPARE: finished
0925 09:57:22.796932 (+ 110us) write_op.cc:270] Start()
0925 09:57:22.797015 (+ 83us) write_op.cc:276] Timestamp: P: 1790330242796886 usec, L: 0
0925 09:57:22.797031 (+ 16us) op_driver.cc:331] REPLICATION: starting
0925 09:57:22.797561 (+ 530us) log.cc:842] Serialized 356 byte log entry
0925 09:57:22.803723 (+ 6162us) op_driver.cc:444] REPLICATION: finished
0925 09:57:22.804231 (+ 508us) write_op.cc:301] APPLY: starting
0925 09:57:22.804296 (+ 65us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:22.804410 (+ 114us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:22.804535 (+ 125us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:22.804722 (+ 187us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:22.804900 (+ 178us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:22.804960 (+ 60us) write_op.cc:312] APPLY: finished
0925 09:57:22.805161 (+ 201us) log.cc:842] Serialized 23 byte log entry
0925 09:57:22.805313 (+ 152us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:22.805391 (+ 78us) write_op.cc:454] Released schema lock
0925 09:57:22.805429 (+ 38us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:22.805654 (+ 225us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:22.805691 (+ 37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4109,"reported_tablets":29,"spinlock_wait_cycles":78402432,"child_traces":[["op",{"apply.queue_time_us":410,"apply.run_cpu_time_us":1810,"apply.run_wall_time_us":1810,"num_ops":5,"prepare.queue_time_us":115,"prepare.run_cpu_time_us":2642,"prepare.run_wall_time_us":2643,"replication_time_us":45565}],["op",{"apply.queue_time_us":318,"apply.run_cpu_time_us":1664,"apply.run_wall_time_us":1669,"num_ops":1,"prepare.queue_time_us":107,"prepare.run_cpu_time_us":1973,"prepare.run_wall_time_us":3207,"raft.queue_time_us":143,"raft.run_cpu_time_us":608,"raft.run_wall_time_us":609,"replication_time_us":6604}]]}
I20260925 09:57:22.815049 11084 consensus_queue.cc:260] T 545ac025f2cb469da847979011a0b3ce P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:23.065177 11084 ts_tablet_manager.cc:1428] T 545ac025f2cb469da847979011a0b3ce P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.259s user 0.005s sys 0.000s
I20260925 09:57:23.066059 11084 tablet_bootstrap.cc:492] T d5956be0da6441c79723e72f12dfe54a P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:23.066380 10390 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_bbc3c50c06634d119ae96dcf649f44e7" } modify_external_catalogs: true
I20260925 09:57:23.070942 11082 tablet_bootstrap.cc:492] T a21909e2e5794f57971f4e42795235c7 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:23.071360 11082 ts_tablet_manager.cc:1397] T a21909e2e5794f57971f4e42795235c7 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.929s user 0.018s sys 0.010s
W20260925 09:57:23.071738 10621 consensus_peers.cc:487] T 3706253e53d841aeaa43889ead13af32 P b8850818e2ee42a082802d0095c08d21 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260925 09:57:23.077441 10621 consensus_peers.cc:487] T 93e1b5837aff48d7907890c40dfcd0d0 P b8850818e2ee42a082802d0095c08d21 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260925 09:57:23.077977 10621 consensus_peers.cc:487] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P b8850818e2ee42a082802d0095c08d21 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:23.086090 10385 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_489e2c74ed6249ce9cf54541e0a491cd" } modify_external_catalogs: true
I20260925 09:57:23.096444 11082 raft_consensus.cc:348] T a21909e2e5794f57971f4e42795235c7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:23.097129 11082 raft_consensus.cc:374] T a21909e2e5794f57971f4e42795235c7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:23.099428 11082 raft_consensus.cc:729] T a21909e2e5794f57971f4e42795235c7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:23.100718 11082 consensus_queue.cc:260] T a21909e2e5794f57971f4e42795235c7 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:23.112725 11082 ts_tablet_manager.cc:1428] T a21909e2e5794f57971f4e42795235c7 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.038s user 0.003s sys 0.003s
I20260925 09:57:23.039278 11085 raft_consensus.cc:374] T 197f61525b0c4f9da441d98701d231a3 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:23.114532 11085 raft_consensus.cc:729] T 197f61525b0c4f9da441d98701d231a3 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:23.115224 11085 consensus_queue.cc:260] T 197f61525b0c4f9da441d98701d231a3 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:23.113667 11082 tablet_bootstrap.cc:492] T 7360a869f3be45a884a0379f4ef78e85 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
W20260925 09:57:23.142988 10619 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 270231us
I20260925 09:57:23.144649 10619 leader_election.cc:302] T ec1b78e61e8d46cb8b87f60786059658 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
W20260925 09:57:23.145972 10621 leader_election.cc:341] T ec1b78e61e8d46cb8b87f60786059658 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:23.103603 10388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_876195e38a7349cc94043563f026f7f6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:23.155509 10391 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_5adf3f6c195e490fa4c978d7e1b4c187" } new_table_name: "default.Table_f8b067e4e4cf498bbf44d7bf24c29363" modify_external_catalogs: true
W20260925 09:57:23.156584 10388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_876195e38a7349cc94043563f026f7f6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:23.167557 11709 ts_tablet_manager.cc:1918] T ec1b78e61e8d46cb8b87f60786059658 P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:23.167953 11709 log.cc:1196] T ec1b78e61e8d46cb8b87f60786059658 P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/ec1b78e61e8d46cb8b87f60786059658
I20260925 09:57:23.168586 11709 ts_tablet_manager.cc:1939] T ec1b78e61e8d46cb8b87f60786059658 P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:23.169401 11709 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 180) took 1186 ms. Trace:
I20260925 09:57:23.169691 11709 rpcz_store.cc:276] 0925 09:57:21.982928 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:21.987585 (+ 4657us) service_pool.cc:227] Handling call
0925 09:57:22.810349 (+822764us) ts_tablet_manager.cc:1145] Deleting tablet ec1b78e61e8d46cb8b87f60786059658
0925 09:57:22.810391 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:23.069961 (+259570us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:23.167309 (+ 97348us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:23.169351 (+ 2042us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":17517,"tablet-delete.queue_time_us":822195}
I20260925 09:57:23.170919 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet ec1b78e61e8d46cb8b87f60786059658 (table default.Table_09fe8f366734488c93a79eeeec2c96f9 [id=a7023f3c32884ce8a7ca21fc95afc721]) successfully deleted
I20260925 09:57:23.171169 11084 tablet_bootstrap.cc:654] T d5956be0da6441c79723e72f12dfe54a P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:23.173094 11082 tablet_bootstrap.cc:654] T 7360a869f3be45a884a0379f4ef78e85 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:23.177817 10392 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_3e77b19accd1463cb978e456128a53ef" } new_table_name: "default.Table_2d40588792754851bf107d378c5950dd" modify_external_catalogs: true
W20260925 09:57:23.175540 10619 raft_consensus.cc:2670] T ec1b78e61e8d46cb8b87f60786059658 P b8850818e2ee42a082802d0095c08d21: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260925 09:57:23.188210 10619 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 43632.9us
W20260925 09:57:23.192592 10621 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 46042.3us
I20260925 09:57:23.200085 11084 tablet_bootstrap.cc:492] T d5956be0da6441c79723e72f12dfe54a P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:23.200548 11084 ts_tablet_manager.cc:1397] T d5956be0da6441c79723e72f12dfe54a P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.135s user 0.010s sys 0.000s
I20260925 09:57:23.202826 11084 raft_consensus.cc:348] T d5956be0da6441c79723e72f12dfe54a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:23.203533 11084 raft_consensus.cc:374] T d5956be0da6441c79723e72f12dfe54a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:23.203840 11084 raft_consensus.cc:729] T d5956be0da6441c79723e72f12dfe54a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:23.204448 11084 consensus_queue.cc:260] T d5956be0da6441c79723e72f12dfe54a P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:23.207082 11084 ts_tablet_manager.cc:1428] T d5956be0da6441c79723e72f12dfe54a P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260925 09:57:23.209910 11709 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 181) took 1062 ms. Trace:
I20260925 09:57:23.216446 11084 tablet_bootstrap.cc:492] T 7a31b999f0fd447eacff13de71aa3fff P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:23.219089 11085 ts_tablet_manager.cc:1428] T 197f61525b0c4f9da441d98701d231a3 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.182s user 0.006s sys 0.000s
I20260925 09:57:23.219916 11085 tablet_bootstrap.cc:492] T b555eeba505740988c704df5c734e7c0 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:23.231010 11084 tablet_bootstrap.cc:654] T 7a31b999f0fd447eacff13de71aa3fff P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:23.231792 10839 raft_consensus.cc:3037] T 197f61525b0c4f9da441d98701d231a3 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:23.240393 11709 rpcz_store.cc:276] 0925 09:57:22.147726 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:22.147837 (+ 111us) service_pool.cc:227] Handling call
0925 09:57:23.209585 (+1061748us) ts_tablet_manager.cc:1145] Deleting tablet 60fdc299865446c89d3e5374d573029a
0925 09:57:23.209626 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:23.209844 (+ 218us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":988299}
W20260925 09:57:23.242219 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 60fdc299865446c89d3e5374d573029a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 60fdc299865446c89d3e5374d573029a already in progress: creating tablet
I20260925 09:57:23.244298 11709 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 182) took 1078 ms. Trace:
I20260925 09:57:23.244848 11709 rpcz_store.cc:276] 0925 09:57:22.165820 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:22.165919 (+ 99us) service_pool.cc:227] Handling call
0925 09:57:23.243656 (+1077737us) ts_tablet_manager.cc:1145] Deleting tablet 545ac025f2cb469da847979011a0b3ce
0925 09:57:23.243724 (+ 68us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:23.244217 (+ 493us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1022151}
I20260925 09:57:23.265197 11084 tablet_bootstrap.cc:492] T 7a31b999f0fd447eacff13de71aa3fff P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:23.265684 11084 ts_tablet_manager.cc:1397] T 7a31b999f0fd447eacff13de71aa3fff P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.049s user 0.006s sys 0.003s
I20260925 09:57:23.267942 11084 raft_consensus.cc:348] T 7a31b999f0fd447eacff13de71aa3fff P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:23.268636 11084 raft_consensus.cc:374] T 7a31b999f0fd447eacff13de71aa3fff P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:23.268927 11084 raft_consensus.cc:729] T 7a31b999f0fd447eacff13de71aa3fff P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:23.279057 11084 consensus_queue.cc:260] T 7a31b999f0fd447eacff13de71aa3fff P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:23.289347 11084 ts_tablet_manager.cc:1428] T 7a31b999f0fd447eacff13de71aa3fff P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.023s user 0.004s sys 0.000s
I20260925 09:57:23.293316 11084 tablet_bootstrap.cc:492] T 69a96874eea1409a9a5a06b5787fbd99 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:23.282719 11627 consensus_queue.cc:1035] T cfa619b699004882a82976f0ef8204ad P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:23.293860 10959 tablet_service.cc:1467] Processing CreateTablet for tablet defaef4d6b754ca5a2dfc21b7a596a5b (DEFAULT_TABLE table=default.Table_29e5f5a8cdbd472a9595a84716d0bfa1 [id=4b5e8269b97c400e867b6be4a5757587]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:23.295578 10959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet defaef4d6b754ca5a2dfc21b7a596a5b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:23.295624 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 545ac025f2cb469da847979011a0b3ce because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 545ac025f2cb469da847979011a0b3ce already in progress: creating tablet
I20260925 09:57:23.298856 10960 tablet_service.cc:1467] Processing CreateTablet for tablet 96a80382bb7d4395a14ad9ae972f4a25 (DEFAULT_TABLE table=default.Table_86a9e15122f04c9c9a111c49547520dc [id=89e2395c5a4e435baaa7abffa48ea910]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:23.299842 10960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96a80382bb7d4395a14ad9ae972f4a25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:23.298384 10384 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_580a41bad60e4da0aa7fe41a67dc8fe8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:23.302292 10384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_580a41bad60e4da0aa7fe41a67dc8fe8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 09:57:23.309769 10772 consensus_peers.cc:487] T 89a143a5623e4ef38027381a56c95227 P c43461c4ee854ae1ac52b873ccada318 -> Peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Couldn't send request to peer be97ab5d75934a648d4ab76341193ec0. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:23.309791 10958 tablet_service.cc:1467] Processing CreateTablet for tablet 92814437d53143a79f99199f8ee3a742 (DEFAULT_TABLE table=default.Table_755c3887714845d2bd8bb30bd444adef [id=d8fb4efa035549159e9b06af8723d715]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:23.310748 10958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92814437d53143a79f99199f8ee3a742. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:23.316020 10963 tablet_service.cc:1467] Processing CreateTablet for tablet b61444c31dbf4882ac748f861ea66818 (DEFAULT_TABLE table=default.Table_f7e141887342416f9c4468ca211d74e5 [id=71c61ce40b55462caefbda94b9ed6e44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:23.317009 10963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b61444c31dbf4882ac748f861ea66818. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:23.322108 10389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_aad29b2508b744aaba08bf51b98cd527"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:23.323642 10389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aad29b2508b744aaba08bf51b98cd527 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:23.324189 11084 tablet_bootstrap.cc:654] T 69a96874eea1409a9a5a06b5787fbd99 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:23.325412 10961 tablet_service.cc:1467] Processing CreateTablet for tablet a436fe287a0d48a1821191adcd031d74 (DEFAULT_TABLE table=default.Table_ae6c60e2023f4a48b6690dcd81dd81b5 [id=b65d342a66984996b482373256ab52d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:23.326395 10961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a436fe287a0d48a1821191adcd031d74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:23.332273 10388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_a2a96456c8804d1692ffbafab832340b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:23.333633 10388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a2a96456c8804d1692ffbafab832340b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:23.333664 10953 tablet_service.cc:1467] Processing CreateTablet for tablet ab6bdba5ca5d41ae921a90c1f77006c5 (DEFAULT_TABLE table=default.Table_14d12f6871c847daa860c3a0cfee4231 [id=f61297f0d10c4656bd06a752c9c7596b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:23.334677 10953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab6bdba5ca5d41ae921a90c1f77006c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:23.259660 11836 consensus_queue.cc:1035] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260925 09:57:23.288815 10664 tablet_service.cc:1467] Processing CreateTablet for tablet defaef4d6b754ca5a2dfc21b7a596a5b (DEFAULT_TABLE table=default.Table_29e5f5a8cdbd472a9595a84716d0bfa1 [id=4b5e8269b97c400e867b6be4a5757587]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:23.340682 10664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet defaef4d6b754ca5a2dfc21b7a596a5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:23.337953 11830 consensus_queue.cc:1035] T 3706253e53d841aeaa43889ead13af32 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:23.342180 10955 tablet_service.cc:1467] Processing CreateTablet for tablet cf775c9d9b114ddcb97dc6c6eca8e0fc (DEFAULT_TABLE table=default.Table_a749aff6c09e4425afc67db6a98196bd [id=9236e950b8d24547ba2ae47fabf2de6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:23.339807 10665 tablet_service.cc:1467] Processing CreateTablet for tablet 96a80382bb7d4395a14ad9ae972f4a25 (DEFAULT_TABLE table=default.Table_86a9e15122f04c9c9a111c49547520dc [id=89e2395c5a4e435baaa7abffa48ea910]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:23.343236 10955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf775c9d9b114ddcb97dc6c6eca8e0fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:23.343226 10665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96a80382bb7d4395a14ad9ae972f4a25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:23.345319 11752 consensus_queue.cc:1035] T 67ce2e3325214aa6bdc3347e9e8f82b0 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:23.350596 10668 tablet_service.cc:1467] Processing CreateTablet for tablet cf775c9d9b114ddcb97dc6c6eca8e0fc (DEFAULT_TABLE table=default.Table_a749aff6c09e4425afc67db6a98196bd [id=9236e950b8d24547ba2ae47fabf2de6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:23.350957 10666 tablet_service.cc:1467] Processing CreateTablet for tablet 92814437d53143a79f99199f8ee3a742 (DEFAULT_TABLE table=default.Table_755c3887714845d2bd8bb30bd444adef [id=d8fb4efa035549159e9b06af8723d715]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:23.351611 10668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf775c9d9b114ddcb97dc6c6eca8e0fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:23.352417 10659 tablet_service.cc:1467] Processing CreateTablet for tablet b61444c31dbf4882ac748f861ea66818 (DEFAULT_TABLE table=default.Table_f7e141887342416f9c4468ca211d74e5 [id=71c61ce40b55462caefbda94b9ed6e44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:23.353308 10659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b61444c31dbf4882ac748f861ea66818. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:23.356155 10661 tablet_service.cc:1467] Processing CreateTablet for tablet a436fe287a0d48a1821191adcd031d74 (DEFAULT_TABLE table=default.Table_ae6c60e2023f4a48b6690dcd81dd81b5 [id=b65d342a66984996b482373256ab52d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:23.357257 10661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a436fe287a0d48a1821191adcd031d74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:23.360013 10658 tablet_service.cc:1467] Processing CreateTablet for tablet ab6bdba5ca5d41ae921a90c1f77006c5 (DEFAULT_TABLE table=default.Table_14d12f6871c847daa860c3a0cfee4231 [id=f61297f0d10c4656bd06a752c9c7596b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:23.360967 10658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab6bdba5ca5d41ae921a90c1f77006c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:23.364078 11579 ts_tablet_manager.cc:1918] T 04fafb9dcf274806b11b2598e73853e9 P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:23.364666 11579 log.cc:1196] T 04fafb9dcf274806b11b2598e73853e9 P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/04fafb9dcf274806b11b2598e73853e9
I20260925 09:57:23.365772 10666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92814437d53143a79f99199f8ee3a742. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:23.365926 11579 ts_tablet_manager.cc:1939] T 04fafb9dcf274806b11b2598e73853e9 P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:23.372123 10959 tablet_service.cc:1467] Processing CreateTablet for tablet 350904aa5957496d90a2aa2e637b591c (DEFAULT_TABLE table=default.Table_836d63b3fbed4250b0757ef2d432a704 [id=f58c7edcfeb44c90968b95873346f919]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 09:57:23.375313 10621 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 96642.2us
I20260925 09:57:23.378389 11084 tablet_bootstrap.cc:492] T 69a96874eea1409a9a5a06b5787fbd99 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:23.378921 11084 ts_tablet_manager.cc:1397] T 69a96874eea1409a9a5a06b5787fbd99 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.086s user 0.009s sys 0.000s
I20260925 09:57:23.381093 11084 raft_consensus.cc:348] T 69a96874eea1409a9a5a06b5787fbd99 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:23.385555 11084 raft_consensus.cc:374] T 69a96874eea1409a9a5a06b5787fbd99 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:23.385869 11084 raft_consensus.cc:729] T 69a96874eea1409a9a5a06b5787fbd99 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
W20260925 09:57:23.388631 10619 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 43779.8us
I20260925 09:57:23.392033 11709 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 183) took 1027 ms. Trace:
I20260925 09:57:23.392989 11084 consensus_queue.cc:260] T 69a96874eea1409a9a5a06b5787fbd99 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:23.392913 11709 rpcz_store.cc:276] 0925 09:57:22.364438 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:22.364562 (+ 124us) service_pool.cc:227] Handling call
0925 09:57:23.296328 (+931766us) ts_tablet_manager.cc:1145] Deleting tablet d9e454f82a4e4c429b58f32926319b7c
0925 09:57:23.296372 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:23.391976 (+ 95604us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":931249}
I20260925 09:57:23.394884 11709 tablet_replica.cc:330] T 689ab2b6cf1640129f6db8b113cf2fc6 P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:23.385104 10961 tablet_service.cc:1467] Processing CreateTablet for tablet aca5987b14b64152887ac5ee584b257a (DEFAULT_TABLE table=default.Table_14fecfa780f146fba2e90a2751353aca [id=fd8ad4c81a8340c380a0150068fed9b2]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 09:57:23.389322 10619 consensus_peers.cc:487] T cf35f39bcd6a41128acb2a4936fcc316 P b8850818e2ee42a082802d0095c08d21 -> Peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Couldn't send request to peer be97ab5d75934a648d4ab76341193ec0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cf35f39bcd6a41128acb2a4936fcc316. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:23.395797 11709 raft_consensus.cc:2229] T 689ab2b6cf1640129f6db8b113cf2fc6 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:23.384815 10959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 350904aa5957496d90a2aa2e637b591c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:23.393646 11751 raft_consensus.cc:482] T 3b9cdad4f9144daea800553abd378e0b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:23.397145 11751 raft_consensus.cc:504] T 3b9cdad4f9144daea800553abd378e0b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:23.400389 11751 leader_election.cc:288] T 3b9cdad4f9144daea800553abd378e0b P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:23.401077 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b9cdad4f9144daea800553abd378e0b" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:23.402444 10918 leader_election.cc:341] T 3b9cdad4f9144daea800553abd378e0b P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:23.402256 10957 tablet_service.cc:1467] Processing CreateTablet for tablet 61aa6380d30a4b21b70ee2d321247bd8 (DEFAULT_TABLE table=default.Table_32e7bdf7eaf147ea90740b6b3cfb2fd3 [id=be059d72d5db4915ba767959c6a1c7b2]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 09:57:23.403709 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet d9e454f82a4e4c429b58f32926319b7c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d9e454f82a4e4c429b58f32926319b7c already in progress: creating tablet
I20260925 09:57:23.408686 11084 ts_tablet_manager.cc:1428] T 69a96874eea1409a9a5a06b5787fbd99 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.029s user 0.005s sys 0.000s
I20260925 09:57:23.411973 11084 tablet_bootstrap.cc:492] T 4e3120347c8e4a3eab57d6562ae40e8d P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:23.413959 10385 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 7a31b999f0fd447eacff13de71aa3fff
I20260925 09:57:23.405160 10957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61aa6380d30a4b21b70ee2d321247bd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:23.416604 10961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aca5987b14b64152887ac5ee584b257a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:23.417752 10385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_4688c2ec003a44218fedb9f17dc0f5e3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:23.425628 10621 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 49685.6us
I20260925 09:57:23.442059 10667 tablet_service.cc:1467] Processing CreateTablet for tablet aca5987b14b64152887ac5ee584b257a (DEFAULT_TABLE table=default.Table_14fecfa780f146fba2e90a2751353aca [id=fd8ad4c81a8340c380a0150068fed9b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:23.443243 10667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aca5987b14b64152887ac5ee584b257a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:23.454165 11709 raft_consensus.cc:2258] T 689ab2b6cf1640129f6db8b113cf2fc6 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:23.438735 11742 raft_consensus.cc:482] T d5956be0da6441c79723e72f12dfe54a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:23.440114 11605 raft_consensus.cc:482] T 55d14d4d8abe4ec9ac9aeef0d027e804 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:23.438922 11751 raft_consensus.cc:482] T e684798155d445bfb325a1b7c5c48753 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:23.465153 11579 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 175) took 3016 ms. Trace:
I20260925 09:57:23.465520 11579 rpcz_store.cc:276] 0925 09:57:20.448512 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:20.448621 (+ 109us) service_pool.cc:227] Handling call
0925 09:57:22.542425 (+2093804us) ts_tablet_manager.cc:1145] Deleting tablet 04fafb9dcf274806b11b2598e73853e9
0925 09:57:22.542513 (+ 88us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:23.004595 (+462082us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:23.363646 (+359051us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:23.465090 (+101444us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2093217}
I20260925 09:57:23.466703 11579 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 177) took 2760 ms. Trace:
I20260925 09:57:23.466871 10656 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a31b999f0fd447eacff13de71aa3fff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:23 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:23.468685 11085 tablet_bootstrap.cc:654] T b555eeba505740988c704df5c734e7c0 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:23.472318 10774 outbound_call.cc:322] RPC callback for RPC call kudu.master.MasterService.Ping -> {remote=127.10.9.60:36985, user_credentials={real_user=slave}} blocked reactor thread for 203886us
W20260925 09:57:23.476161 10385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4688c2ec003a44218fedb9f17dc0f5e3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:23.491641 10960 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a31b999f0fd447eacff13de71aa3fff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:23 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:23.493184 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 04fafb9dcf274806b11b2598e73853e9 (table default.Table_bc69fdbbaf5b4a4e8cdfe195eefb16ab [id=d76cef751d12440190294e86649e2639]) successfully deleted
I20260925 09:57:23.494437 11709 ts_tablet_manager.cc:1905] T 689ab2b6cf1640129f6db8b113cf2fc6 P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:23.456069 10393 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 3d652a76d96f4dbeafde15f3c16534a6 (table default.Table_be0f658aa41f4188bc16db37e96f4ad1 [id=73083c1560cb4db5a12964387cb49b73]). Expected version 1 got 0
I20260925 09:57:23.466981 11579 rpcz_store.cc:276] 0925 09:57:20.706228 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:20.706343 (+ 115us) service_pool.cc:227] Handling call
0925 09:57:23.466424 (+2760081us) ts_tablet_manager.cc:1145] Deleting tablet 4c4e8790282a49fd849a34ca11bef5bf
0925 09:57:23.466475 (+ 51us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:23.466658 (+ 183us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2759306}
I20260925 09:57:23.497785 10393 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet b555eeba505740988c704df5c734e7c0 (table default.Table_48f368d811494119be0dc534e2fdcfbd [id=3e32026d175b4b92a3ee036f17e54b9d]). Expected version 1 got 0
I20260925 09:57:23.499197 10393 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 26fae3fbfe374f348a1729c21ec751b9 (table default.Table_0011387b45014de4b24b466f4cb907f7 [id=e3f7aa5e988c4b6c98be5cd3d465245d]). Expected version 1 got 0
I20260925 09:57:23.500291 10393 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 42ec275c905343c4be1b72dc2bd0f6e3 (table default.Table_c22088d959e84c3cbabadde4d3445178 [id=cf07ed8756904011ab3dac900ea6705a]). Expected version 1 got 0
I20260925 09:57:23.500743 10807 tablet_service.cc:1467] Processing CreateTablet for tablet defaef4d6b754ca5a2dfc21b7a596a5b (DEFAULT_TABLE table=default.Table_29e5f5a8cdbd472a9595a84716d0bfa1 [id=4b5e8269b97c400e867b6be4a5757587]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:23.501771 10807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet defaef4d6b754ca5a2dfc21b7a596a5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:23.503719 11579 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 176) took 2859 ms. Trace:
I20260925 09:57:23.504021 11579 rpcz_store.cc:276] 0925 09:57:20.644329 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:20.711898 (+ 67569us) service_pool.cc:227] Handling call
0925 09:57:23.503363 (+2791465us) ts_tablet_manager.cc:1145] Deleting tablet 6161774f61ae48ab9764a05f2188a80b
0925 09:57:23.503425 (+ 62us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:23.503672 (+ 247us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2779431}
I20260925 09:57:23.511763 10819 tablet_service.cc:1467] Processing CreateTablet for tablet 92814437d53143a79f99199f8ee3a742 (DEFAULT_TABLE table=default.Table_755c3887714845d2bd8bb30bd444adef [id=d8fb4efa035549159e9b06af8723d715]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:23.512660 10819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92814437d53143a79f99199f8ee3a742. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:23.465795 11742 raft_consensus.cc:504] T d5956be0da6441c79723e72f12dfe54a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:23.468637 11751 raft_consensus.cc:504] T e684798155d445bfb325a1b7c5c48753 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:23.519564 10815 tablet_service.cc:1467] Processing CreateTablet for tablet 96a80382bb7d4395a14ad9ae972f4a25 (DEFAULT_TABLE table=default.Table_86a9e15122f04c9c9a111c49547520dc [id=89e2395c5a4e435baaa7abffa48ea910]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:23.520524 10815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96a80382bb7d4395a14ad9ae972f4a25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:23.523967 11751 leader_election.cc:288] T e684798155d445bfb325a1b7c5c48753 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:23.518370 10956 tablet_service.cc:1467] Processing CreateTablet for tablet f62c62715c9b42de826c1a66fd431eab (DEFAULT_TABLE table=default.Table_5062a73a7614459a8690df32165076dc [id=b3f7f21ab2214e4cb637bea27ce62abe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:23.526065 10810 tablet_service.cc:1467] Processing CreateTablet for tablet b61444c31dbf4882ac748f861ea66818 (DEFAULT_TABLE table=default.Table_f7e141887342416f9c4468ca211d74e5 [id=71c61ce40b55462caefbda94b9ed6e44]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 09:57:23.529019 10379 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 -> Peer b8be057e0b6d40a3a4b90f2b1f4a0fa2 (127.10.9.60:36985): Couldn't send request to peer b8be057e0b6d40a3a4b90f2b1f4a0fa2. Status: Network error: Client connection negotiation failed: client connection to 127.10.9.60:36985: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260925 09:57:23.529374 10956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f62c62715c9b42de826c1a66fd431eab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:23.468483 11605 raft_consensus.cc:504] T 55d14d4d8abe4ec9ac9aeef0d027e804 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:23.521744 11742 leader_election.cc:288] T d5956be0da6441c79723e72f12dfe54a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:23.536497 11605 leader_election.cc:288] T 55d14d4d8abe4ec9ac9aeef0d027e804 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:23.550016 10805 tablet_service.cc:1467] Processing CreateTablet for tablet a436fe287a0d48a1821191adcd031d74 (DEFAULT_TABLE table=default.Table_ae6c60e2023f4a48b6690dcd81dd81b5 [id=b65d342a66984996b482373256ab52d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:23.551683 10810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b61444c31dbf4882ac748f861ea66818. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:23.552418 11084 tablet_bootstrap.cc:654] T 4e3120347c8e4a3eab57d6562ae40e8d P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:23.555706 10833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55d14d4d8abe4ec9ac9aeef0d027e804" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:23.556773 10834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e684798155d445bfb325a1b7c5c48753" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:23.558440 10918 leader_election.cc:341] T 55d14d4d8abe4ec9ac9aeef0d027e804 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:23.559024 10918 leader_election.cc:341] T e684798155d445bfb325a1b7c5c48753 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:23.459066 11752 consensus_queue.cc:1035] T 67ce2e3325214aa6bdc3347e9e8f82b0 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.029s
I20260925 09:57:23.563984 10805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a436fe287a0d48a1821191adcd031d74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:23.564872 10645 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:37864 (request call id 12) took 1195 ms. Trace:
I20260925 09:57:23.565191 10645 rpcz_store.cc:276] 0925 09:57:22.369903 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:22.371456 (+ 1553us) service_pool.cc:227] Handling call
0925 09:57:22.513184 (+141728us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b5400141754. stack: 00007fb965334033 00007fb965333f30 00007fb964eb5534 00000000004db2b4 00000000004db249 00007fb96c240471 00007fb96c2b5268 00007fb96d2a16cd 00007fb96bb49e04 00007fb96bb49d92 00007fb96bb49cc1 00007fb96bb49c3c 00007fb96bb48ef2 00007fb96a06a94c 00007fb96a069d86 00007fb96a06972f
0925 09:57:22.520894 (+ 7710us) spinlock_profiling.cc:243] Waited 902 us on lock 0x7b20000a5978. stack: 00007fb965334033 00007fb965333f30 00007fb964eb5534 00000000004db2b4 00000000004db249 00007fb96c0e5f11 00007fb96c10777c 00007fb96c0d8e92 00007fb96c6dc522 00007fb96c6dc28d 00007fb96d2a16a0 00007fb96bb49e04 00007fb96bb49d92 00007fb96bb49cc1 00007fb96bb49c3c 00007fb96bb48ef2
0925 09:57:23.063178 (+542284us) spinlock_profiling.cc:243] Waited 541 ms on lock 0x7b54000f3554. stack: 00007fb965334033 00007fb965333f30 00007fb964eb5534 00000000004db2b4 00000000004db249 00007fb96c240471 00007fb96c2b5268 00007fb96d2a16cd 00007fb96bb49e04 00007fb96bb49d92 00007fb96bb49cc1 00007fb96bb49c3c 00007fb96bb48ef2 00007fb96a06a94c 00007fb96a069d86 00007fb96a06972f
0925 09:57:23.130196 (+ 67018us) spinlock_profiling.cc:243] Waited 44.5 ms on lock 0x7b54000e0e54. stack: 00007fb965334033 00007fb965333f30 00007fb964eb5534 00000000004db2b4 00000000004db249 00007fb96c240471 00007fb96c2b5268 00007fb96d2a16cd 00007fb96bb49e04 00007fb96bb49d92 00007fb96bb49cc1 00007fb96bb49c3c 00007fb96bb48ef2 00007fb96a06a94c 00007fb96a069d86 00007fb96a06972f
0925 09:57:23.470220 (+340024us) spinlock_profiling.cc:243] Waited 321 ms on lock 0x7b54000a4bd4. stack: 00007fb965334033 00007fb965333f30 00007fb964eb5534 00000000004db2b4 00000000004db249 00007fb96c240471 00007fb96c2b5268 00007fb96d2a16cd 00007fb96bb49e04 00007fb96bb49d92 00007fb96bb49cc1 00007fb96bb49c3c 00007fb96bb48ef2 00007fb96a06a94c 00007fb96a069d86 00007fb96a06972f
0925 09:57:23.564794 (+ 94574us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2400583296}
I20260925 09:57:23.566644 10685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e684798155d445bfb325a1b7c5c48753" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:23.571462 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5956be0da6441c79723e72f12dfe54a" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:23.572953 10918 leader_election.cc:341] T d5956be0da6441c79723e72f12dfe54a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:23.573165 10919 leader_election.cc:341] T e684798155d445bfb325a1b7c5c48753 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:23.573545 10919 leader_election.cc:302] T e684798155d445bfb325a1b7c5c48753 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:23.574676 11605 raft_consensus.cc:2726] T e684798155d445bfb325a1b7c5c48753 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:23.584712 11085 tablet_bootstrap.cc:492] T b555eeba505740988c704df5c734e7c0 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:23.585096 11085 ts_tablet_manager.cc:1397] T b555eeba505740988c704df5c734e7c0 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.365s user 0.009s sys 0.000s
I20260925 09:57:23.588055 11579 tablet_replica.cc:330] T 89a143a5623e4ef38027381a56c95227 P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:23.593809 11751 raft_consensus.cc:482] T 545ac025f2cb469da847979011a0b3ce P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:23.594254 11751 raft_consensus.cc:504] T 545ac025f2cb469da847979011a0b3ce P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:23.596197 10806 tablet_service.cc:1467] Processing CreateTablet for tablet ab6bdba5ca5d41ae921a90c1f77006c5 (DEFAULT_TABLE table=default.Table_14d12f6871c847daa860c3a0cfee4231 [id=f61297f0d10c4656bd06a752c9c7596b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:23.596632 11751 leader_election.cc:288] T 545ac025f2cb469da847979011a0b3ce P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:23.597493 10806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab6bdba5ca5d41ae921a90c1f77006c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:23.600545 10816 tablet_service.cc:1467] Processing CreateTablet for tablet cf775c9d9b114ddcb97dc6c6eca8e0fc (DEFAULT_TABLE table=default.Table_a749aff6c09e4425afc67db6a98196bd [id=9236e950b8d24547ba2ae47fabf2de6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:23.601670 10816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf775c9d9b114ddcb97dc6c6eca8e0fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:23.607805 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "545ac025f2cb469da847979011a0b3ce" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:23.609239 10918 leader_election.cc:341] T 545ac025f2cb469da847979011a0b3ce P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:23.611974 11579 raft_consensus.cc:2229] T 89a143a5623e4ef38027381a56c95227 P c43461c4ee854ae1ac52b873ccada318 [term 2 LEADER]: Raft consensus shutting down.
I20260925 09:57:23.613261 11579 raft_consensus.cc:2258] T 89a143a5623e4ef38027381a56c95227 P c43461c4ee854ae1ac52b873ccada318 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:23.617209 11751 raft_consensus.cc:482] T 2358acdaa5104e3eacbab8bf182b8f30 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:23.617782 11751 raft_consensus.cc:504] T 2358acdaa5104e3eacbab8bf182b8f30 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:23.618817 10385 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_5157be3699be4fa4ac153a69f6010ecc" } modify_external_catalogs: true
I20260925 09:57:23.619302 10385 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_5157be3699be4fa4ac153a69f6010ecc" } modify_external_catalogs: true
I20260925 09:57:23.621024 11751 leader_election.cc:288] T 2358acdaa5104e3eacbab8bf182b8f30 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:23.622831 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2358acdaa5104e3eacbab8bf182b8f30" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:23.624078 10918 leader_election.cc:341] T 2358acdaa5104e3eacbab8bf182b8f30 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:23.624706 11085 raft_consensus.cc:348] T b555eeba505740988c704df5c734e7c0 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:23.625425 11085 raft_consensus.cc:374] T b555eeba505740988c704df5c734e7c0 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:23.625710 11085 raft_consensus.cc:729] T b555eeba505740988c704df5c734e7c0 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:23.626418 11085 consensus_queue.cc:260] T b555eeba505740988c704df5c734e7c0 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:23.629760 11751 raft_consensus.cc:482] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:23.630277 11751 raft_consensus.cc:504] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:23.634141 11751 leader_election.cc:288] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:23.501992 10393 catalog_manager.cc:5495] T c588b0c05a1a4f3ea545066183e76502 P be97ab5d75934a648d4ab76341193ec0 reported cstate change: term changed from 0 to 1, leader changed from <none> to be97ab5d75934a648d4ab76341193ec0 (127.10.9.3). New cstate: current_term: 1 leader_uuid: "be97ab5d75934a648d4ab76341193ec0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: HEALTHY } } }
I20260925 09:57:23.643597 10393 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet e57065f2bb224a56ad1c3518a0c375da (table default.Table_6eff7b87b3b74ca4bab719140667a87b [id=7889081642604700bbc6d253ab105260]). Expected version 2 got 1
I20260925 09:57:23.609107 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55d14d4d8abe4ec9ac9aeef0d027e804" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:23.659915 11605 raft_consensus.cc:482] T 7a31b999f0fd447eacff13de71aa3fff P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:23.660377 11605 raft_consensus.cc:504] T 7a31b999f0fd447eacff13de71aa3fff P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:23.661224 11727 ts_tablet_manager.cc:1918] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.1
I20260925 09:57:23.661657 11727 log.cc:1196] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/89a143a5623e4ef38027381a56c95227
I20260925 09:57:23.658957 11084 tablet_bootstrap.cc:492] T 4e3120347c8e4a3eab57d6562ae40e8d P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:23.662519 11084 ts_tablet_manager.cc:1397] T 4e3120347c8e4a3eab57d6562ae40e8d P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.252s user 0.017s sys 0.002s
I20260925 09:57:23.662446 11605 leader_election.cc:288] T 7a31b999f0fd447eacff13de71aa3fff P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:23.664916 11084 raft_consensus.cc:348] T 4e3120347c8e4a3eab57d6562ae40e8d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:23.665601 11084 raft_consensus.cc:374] T 4e3120347c8e4a3eab57d6562ae40e8d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:23.665856 11084 raft_consensus.cc:729] T 4e3120347c8e4a3eab57d6562ae40e8d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:23.668252 10657 tablet_service.cc:1467] Processing CreateTablet for tablet 61aa6380d30a4b21b70ee2d321247bd8 (DEFAULT_TABLE table=default.Table_32e7bdf7eaf147ea90740b6b3cfb2fd3 [id=be059d72d5db4915ba767959c6a1c7b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:23.669435 10657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61aa6380d30a4b21b70ee2d321247bd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:23.672717 11085 ts_tablet_manager.cc:1428] T b555eeba505740988c704df5c734e7c0 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.087s user 0.006s sys 0.000s
I20260925 09:57:23.673640 11085 tablet_bootstrap.cc:492] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:23.673673 10810 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a31b999f0fd447eacff13de71aa3fff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:23 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:23.674015 11727 ts_tablet_manager.cc:1939] T 89a143a5623e4ef38027381a56c95227 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:23.633651 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aba253f22afa4200926f31a1b43e3970" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:23.636813 11579 ts_tablet_manager.cc:1905] T 89a143a5623e4ef38027381a56c95227 P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:23.637120 10812 tablet_service.cc:1467] Processing CreateTablet for tablet aca5987b14b64152887ac5ee584b257a (DEFAULT_TABLE table=default.Table_14fecfa780f146fba2e90a2751353aca [id=fd8ad4c81a8340c380a0150068fed9b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:23.675806 10812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aca5987b14b64152887ac5ee584b257a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:23.676096 10918 leader_election.cc:341] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:23.676318 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 175) took 2732 ms. Trace:
I20260925 09:57:23.676663 11727 rpcz_store.cc:276] 0925 09:57:20.944192 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:20.944331 (+ 139us) service_pool.cc:227] Handling call
0925 09:57:22.968898 (+2024567us) ts_tablet_manager.cc:1145] Deleting tablet 89a143a5623e4ef38027381a56c95227
0925 09:57:22.968946 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:23.507917 (+538971us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:23.660974 (+153057us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:23.676262 (+ 15288us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1924831}
I20260925 09:57:23.652874 10811 tablet_service.cc:1467] Processing CreateTablet for tablet f62c62715c9b42de826c1a66fd431eab (DEFAULT_TABLE table=default.Table_5062a73a7614459a8690df32165076dc [id=b3f7f21ab2214e4cb637bea27ce62abe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:23.678011 10655 tablet_service.cc:1467] Processing CreateTablet for tablet f62c62715c9b42de826c1a66fd431eab (DEFAULT_TABLE table=default.Table_5062a73a7614459a8690df32165076dc [id=b3f7f21ab2214e4cb637bea27ce62abe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:23.678757 10811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f62c62715c9b42de826c1a66fd431eab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:23.684815 11084 consensus_queue.cc:260] T 4e3120347c8e4a3eab57d6562ae40e8d P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:23.688165 10806 tablet_service.cc:1467] Processing CreateTablet for tablet 61aa6380d30a4b21b70ee2d321247bd8 (DEFAULT_TABLE table=default.Table_32e7bdf7eaf147ea90740b6b3cfb2fd3 [id=be059d72d5db4915ba767959c6a1c7b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:23.689186 10806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61aa6380d30a4b21b70ee2d321247bd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:23.667537 10834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a31b999f0fd447eacff13de71aa3fff" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:23.694947 11084 ts_tablet_manager.cc:1428] T 4e3120347c8e4a3eab57d6562ae40e8d P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.032s user 0.006s sys 0.001s
W20260925 09:57:23.694689 10918 leader_election.cc:341] T 7a31b999f0fd447eacff13de71aa3fff P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:23.697283 11084 tablet_bootstrap.cc:492] T 2f07dc9548364870923165f4ea524310 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:23.698139 10819 tablet_service.cc:1467] Processing CreateTablet for tablet 350904aa5957496d90a2aa2e637b591c (DEFAULT_TABLE table=default.Table_836d63b3fbed4250b0757ef2d432a704 [id=f58c7edcfeb44c90968b95873346f919]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:23.705883 11084 tablet_bootstrap.cc:654] T 2f07dc9548364870923165f4ea524310 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:23.706576 10819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 350904aa5957496d90a2aa2e637b591c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:23.715174 11085 tablet_bootstrap.cc:654] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:23.716982 11579 ts_tablet_manager.cc:1918] T 89a143a5623e4ef38027381a56c95227 P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.1
I20260925 09:57:23.717316 11579 log.cc:1196] T 89a143a5623e4ef38027381a56c95227 P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/89a143a5623e4ef38027381a56c95227
I20260925 09:57:23.717945 11579 ts_tablet_manager.cc:1939] T 89a143a5623e4ef38027381a56c95227 P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:23.663671 11752 raft_consensus.cc:482] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:23.683786 10682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aba253f22afa4200926f31a1b43e3970" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:23.709159 10663 tablet_service.cc:1467] Processing CreateTablet for tablet 350904aa5957496d90a2aa2e637b591c (DEFAULT_TABLE table=default.Table_836d63b3fbed4250b0757ef2d432a704 [id=f58c7edcfeb44c90968b95873346f919]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:23.725560 11752 raft_consensus.cc:504] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:23.729236 11752 leader_election.cc:288] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:23.711805 10677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2358acdaa5104e3eacbab8bf182b8f30" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:23.732138 11084 tablet_bootstrap.cc:492] T 2f07dc9548364870923165f4ea524310 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:23.732615 11084 ts_tablet_manager.cc:1397] T 2f07dc9548364870923165f4ea524310 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.037s user 0.011s sys 0.001s
I20260925 09:57:23.734824 11579 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 178) took 2772 ms. Trace:
I20260925 09:57:23.735054 11084 raft_consensus.cc:348] T 2f07dc9548364870923165f4ea524310 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:23.735149 11579 rpcz_store.cc:276] 0925 09:57:20.962611 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:20.962797 (+ 186us) service_pool.cc:227] Handling call
0925 09:57:23.516226 (+2553429us) ts_tablet_manager.cc:1145] Deleting tablet 89a143a5623e4ef38027381a56c95227
0925 09:57:23.587948 (+ 71722us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:23.681910 (+ 93962us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:23.716770 (+ 34860us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:23.734740 (+ 17970us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2551948}
I20260925 09:57:23.735827 11084 raft_consensus.cc:374] T 2f07dc9548364870923165f4ea524310 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:23.705219 10679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b9cdad4f9144daea800553abd378e0b" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:23.736136 11084 raft_consensus.cc:729] T 2f07dc9548364870923165f4ea524310 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:23.737313 11779 raft_consensus.cc:482] T b555eeba505740988c704df5c734e7c0 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:23.737694 11779 raft_consensus.cc:504] T b555eeba505740988c704df5c734e7c0 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:23.739717 11779 leader_election.cc:288] T b555eeba505740988c704df5c734e7c0 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
W20260925 09:57:23.740873 10919 leader_election.cc:341] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:23.741284 10919 leader_election.cc:302] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
W20260925 09:57:23.742376 10919 leader_election.cc:341] T 55d14d4d8abe4ec9ac9aeef0d027e804 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:23.742717 10919 leader_election.cc:302] T 55d14d4d8abe4ec9ac9aeef0d027e804 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:23.743048 11579 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 180) took 1652 ms. Trace:
I20260925 09:57:23.743325 11579 rpcz_store.cc:276] 0925 09:57:22.091004 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:22.097127 (+ 6123us) service_pool.cc:227] Handling call
0925 09:57:23.742619 (+1645492us) ts_tablet_manager.cc:1145] Deleting tablet ec1b78e61e8d46cb8b87f60786059658
0925 09:57:23.742658 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:23.742967 (+ 309us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1644445}
I20260925 09:57:23.743953 11742 raft_consensus.cc:2726] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:23.744504 11742 raft_consensus.cc:2726] T 55d14d4d8abe4ec9ac9aeef0d027e804 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:23.745597 10646 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:37864 (request call id 13) took 1316 ms. Trace:
I20260925 09:57:23.745471 10982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b555eeba505740988c704df5c734e7c0" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:23.745939 10646 rpcz_store.cc:276] 0925 09:57:22.421816 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:22.425124 (+ 3308us) service_pool.cc:227] Handling call
0925 09:57:22.516941 (+ 91817us) spinlock_profiling.cc:243] Waited 86 ms on lock 0x7b5400141754. stack: 00007fb965334033 00007fb965333f30 00007fb964eb5534 00000000004db2b4 00000000004db249 00007fb96c240471 00007fb96c2b5268 00007fb96d2a16cd 00007fb96bb49e04 00007fb96bb49d92 00007fb96bb49cc1 00007fb96bb49c3c 00007fb96bb48ef2 00007fb96a06a94c 00007fb96a069d86 00007fb96a06972f
0925 09:57:23.079514 (+562573us) spinlock_profiling.cc:243] Waited 542 ms on lock 0x7b54000f3554. stack: 00007fb965334033 00007fb965333f30 00007fb964eb5534 00000000004db2b4 00000000004db249 00007fb96c240471 00007fb96c2b5268 00007fb96d2a16cd 00007fb96bb49e04 00007fb96bb49d92 00007fb96bb49cc1 00007fb96bb49c3c 00007fb96bb48ef2 00007fb96a06a94c 00007fb96a069d86 00007fb96a06972f
0925 09:57:23.145085 (+ 65571us) spinlock_profiling.cc:243] Waited 49.1 ms on lock 0x7b54000e0e54. stack: 00007fb965334033 00007fb965333f30 00007fb964eb5534 00000000004db2b4 00000000004db249 00007fb96c240471 00007fb96c2b5268 00007fb96d2a16cd 00007fb96bb49e04 00007fb96bb49d92 00007fb96bb49cc1 00007fb96bb49c3c 00007fb96bb48ef2 00007fb96a06a94c 00007fb96a069d86 00007fb96a06972f
0925 09:57:23.550405 (+405320us) spinlock_profiling.cc:243] Waited 303 ms on lock 0x7b54000a4bd4. stack: 00007fb965334033 00007fb965333f30 00007fb964eb5534 00000000004db2b4 00000000004db249 00007fb96c240471 00007fb96c2b5268 00007fb96d2a16cd 00007fb96bb49e04 00007fb96bb49d92 00007fb96bb49cc1 00007fb96bb49c3c 00007fb96bb48ef2 00007fb96a06a94c 00007fb96a069d86 00007fb96a06972f
0925 09:57:23.738460 (+188055us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2254674048}
W20260925 09:57:23.746903 10919 leader_election.cc:341] T 2358acdaa5104e3eacbab8bf182b8f30 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:23.747242 10919 leader_election.cc:302] T 2358acdaa5104e3eacbab8bf182b8f30 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:23.748522 11742 raft_consensus.cc:2726] T 2358acdaa5104e3eacbab8bf182b8f30 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:23.748248 10919 leader_election.cc:341] T 3b9cdad4f9144daea800553abd378e0b P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:23.750186 10919 leader_election.cc:302] T 3b9cdad4f9144daea800553abd378e0b P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:23.751372 11605 raft_consensus.cc:2726] T 3b9cdad4f9144daea800553abd378e0b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:23.737030 11084 consensus_queue.cc:260] T 2f07dc9548364870923165f4ea524310 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:23.752027 10678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "545ac025f2cb469da847979011a0b3ce" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:23.754856 11579 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 181) took 1596 ms. Trace:
I20260925 09:57:23.755415 11084 ts_tablet_manager.cc:1428] T 2f07dc9548364870923165f4ea524310 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.022s user 0.006s sys 0.000s
I20260925 09:57:23.755160 11579 rpcz_store.cc:276] 0925 09:57:22.158253 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:22.165653 (+ 7400us) service_pool.cc:227] Handling call
0925 09:57:23.754485 (+1588832us) ts_tablet_manager.cc:1145] Deleting tablet 60fdc299865446c89d3e5374d573029a
0925 09:57:23.754526 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:23.754801 (+ 275us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1588120}
I20260925 09:57:23.755333 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5956be0da6441c79723e72f12dfe54a" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:23.756445 11084 tablet_bootstrap.cc:492] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:23.756577 11579 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 182) took 1589 ms. Trace:
I20260925 09:57:23.756842 11579 rpcz_store.cc:276] 0925 09:57:22.167630 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:22.168059 (+ 429us) service_pool.cc:227] Handling call
0925 09:57:23.756112 (+1588053us) ts_tablet_manager.cc:1145] Deleting tablet 545ac025f2cb469da847979011a0b3ce
0925 09:57:23.756146 (+ 34us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:23.756529 (+ 383us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1587538}
I20260925 09:57:23.757899 11579 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 184) took 1390 ms. Trace:
I20260925 09:57:23.758183 11579 rpcz_store.cc:276] 0925 09:57:22.367782 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:22.427912 (+ 60130us) service_pool.cc:227] Handling call
0925 09:57:23.757578 (+1329666us) ts_tablet_manager.cc:1145] Deleting tablet d9e454f82a4e4c429b58f32926319b7c
0925 09:57:23.757611 (+ 33us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:23.757853 (+ 242us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1329018}
I20260925 09:57:23.759091 11579 tablet_replica.cc:330] T 689ab2b6cf1640129f6db8b113cf2fc6 P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:23.701817 10683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a31b999f0fd447eacff13de71aa3fff" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:23.763222 10919 leader_election.cc:341] T 545ac025f2cb469da847979011a0b3ce P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:23.763742 10919 leader_election.cc:302] T 545ac025f2cb469da847979011a0b3ce P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:23.764909 11605 raft_consensus.cc:2726] T 545ac025f2cb469da847979011a0b3ce P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:23.765043 10919 leader_election.cc:341] T d5956be0da6441c79723e72f12dfe54a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:23.765394 10919 leader_election.cc:302] T d5956be0da6441c79723e72f12dfe54a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:23.766047 11605 raft_consensus.cc:2726] T d5956be0da6441c79723e72f12dfe54a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:23.771451 11085 tablet_bootstrap.cc:492] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:23.771876 11085 ts_tablet_manager.cc:1397] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.098s user 0.011s sys 0.000s
W20260925 09:57:23.766373 10919 leader_election.cc:341] T 7a31b999f0fd447eacff13de71aa3fff P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:23.773612 10919 leader_election.cc:302] T 7a31b999f0fd447eacff13de71aa3fff P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:23.774276 11605 raft_consensus.cc:2726] T 7a31b999f0fd447eacff13de71aa3fff P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:23.774544 11085 raft_consensus.cc:348] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:23.722486 10655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f62c62715c9b42de826c1a66fd431eab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:23.780972 11084 tablet_bootstrap.cc:654] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:23.694720 11727 tablet_replica.cc:330] T ec1b78e61e8d46cb8b87f60786059658 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:23.791339 11579 raft_consensus.cc:2229] T 689ab2b6cf1640129f6db8b113cf2fc6 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:23.791914 11579 raft_consensus.cc:2258] T 689ab2b6cf1640129f6db8b113cf2fc6 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 09:57:23.795316 10619 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 151045us
I20260925 09:57:23.775198 11085 raft_consensus.cc:374] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:23.798981 11082 tablet_bootstrap.cc:492] T 7360a869f3be45a884a0379f4ef78e85 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:23.799068 11085 raft_consensus.cc:729] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:23.799443 11082 ts_tablet_manager.cc:1397] T 7360a869f3be45a884a0379f4ef78e85 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.686s user 0.008s sys 0.018s
I20260925 09:57:23.799880 11085 consensus_queue.cc:260] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:23.805099 10663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 350904aa5957496d90a2aa2e637b591c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:23.806485 11727 raft_consensus.cc:2229] T ec1b78e61e8d46cb8b87f60786059658 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:23.807364 11727 raft_consensus.cc:2258] T ec1b78e61e8d46cb8b87f60786059658 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:23.809031 11085 ts_tablet_manager.cc:1428] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.037s user 0.006s sys 0.000s
I20260925 09:57:23.809705 11579 ts_tablet_manager.cc:1905] T 689ab2b6cf1640129f6db8b113cf2fc6 P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:23.812278 11727 ts_tablet_manager.cc:1905] T ec1b78e61e8d46cb8b87f60786059658 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:23.824731 11084 tablet_bootstrap.cc:492] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:23.825362 11084 ts_tablet_manager.cc:1397] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.069s user 0.012s sys 0.003s
I20260925 09:57:23.828224 11084 raft_consensus.cc:348] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:23.835523 11084 raft_consensus.cc:374] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:23.835911 11084 raft_consensus.cc:729] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:23.836700 11084 consensus_queue.cc:260] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:23.838003 11605 raft_consensus.cc:482] T a44d9f2f7e3e48aeb0f31fa29ae1988a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:23.838527 11605 raft_consensus.cc:504] T a44d9f2f7e3e48aeb0f31fa29ae1988a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:23.840900 11605 leader_election.cc:288] T a44d9f2f7e3e48aeb0f31fa29ae1988a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:23.842888 10834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a44d9f2f7e3e48aeb0f31fa29ae1988a" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:23.846288 11085 tablet_bootstrap.cc:492] T ec1b78e61e8d46cb8b87f60786059658 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
W20260925 09:57:23.847081 10918 leader_election.cc:341] T a44d9f2f7e3e48aeb0f31fa29ae1988a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:23.847294 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a44d9f2f7e3e48aeb0f31fa29ae1988a" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:23.848788 10919 leader_election.cc:341] T a44d9f2f7e3e48aeb0f31fa29ae1988a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:23.850246 10919 leader_election.cc:302] T a44d9f2f7e3e48aeb0f31fa29ae1988a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:23.852861 11605 raft_consensus.cc:2726] T a44d9f2f7e3e48aeb0f31fa29ae1988a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:23.860805 11084 ts_tablet_manager.cc:1428] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.035s user 0.005s sys 0.000s
I20260925 09:57:23.865900 11084 tablet_bootstrap.cc:492] T a5d872c3620646e5ac33ceb95de2cbd4 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:23.869047 11085 tablet_bootstrap.cc:654] T ec1b78e61e8d46cb8b87f60786059658 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:23.890793 11084 tablet_bootstrap.cc:654] T a5d872c3620646e5ac33ceb95de2cbd4 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:23.892259 11861 tablet.cc:1722] T e57065f2bb224a56ad1c3518a0c375da P b8850818e2ee42a082802d0095c08d21: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 09:57:23.904829 11605 raft_consensus.cc:482] T 9e697e7146e34841bd64660b45ebf231 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:23.905376 11605 raft_consensus.cc:504] T 9e697e7146e34841bd64660b45ebf231 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:23.907959 11605 leader_election.cc:288] T 9e697e7146e34841bd64660b45ebf231 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:23.908903 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e697e7146e34841bd64660b45ebf231" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:23.910097 10918 leader_election.cc:341] T 9e697e7146e34841bd64660b45ebf231 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:23.919883 11605 raft_consensus.cc:482] T 2f07dc9548364870923165f4ea524310 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:23.920542 11605 raft_consensus.cc:504] T 2f07dc9548364870923165f4ea524310 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:23.925410 10918 consensus_peers.cc:487] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:23.926319 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f07dc9548364870923165f4ea524310" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:23.928011 10918 leader_election.cc:341] T 2f07dc9548364870923165f4ea524310 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:23.928818 10833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26fae3fbfe374f348a1729c21ec751b9" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:23.929423 10833 raft_consensus.cc:2454] T 26fae3fbfe374f348a1729c21ec751b9 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:23.933197 11727 ts_tablet_manager.cc:1918] T ec1b78e61e8d46cb8b87f60786059658 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:23.933584 11727 log.cc:1196] T ec1b78e61e8d46cb8b87f60786059658 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/ec1b78e61e8d46cb8b87f60786059658
I20260925 09:57:23.934345 11727 ts_tablet_manager.cc:1939] T ec1b78e61e8d46cb8b87f60786059658 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:23.898222 11082 raft_consensus.cc:348] T 7360a869f3be45a884a0379f4ef78e85 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:23.936913 11082 raft_consensus.cc:374] T 7360a869f3be45a884a0379f4ef78e85 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:23.937249 11082 raft_consensus.cc:729] T 7360a869f3be45a884a0379f4ef78e85 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
W20260925 09:57:23.901520 10619 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 105526us
I20260925 09:57:23.910998 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e697e7146e34841bd64660b45ebf231" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:23.942281 10919 leader_election.cc:341] T 9e697e7146e34841bd64660b45ebf231 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:23.942688 10919 leader_election.cc:302] T 9e697e7146e34841bd64660b45ebf231 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:23.943708 11084 tablet_bootstrap.cc:492] T a5d872c3620646e5ac33ceb95de2cbd4 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:23.944160 11084 ts_tablet_manager.cc:1397] T a5d872c3620646e5ac33ceb95de2cbd4 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.078s user 0.011s sys 0.001s
I20260925 09:57:23.945698 11751 raft_consensus.cc:2726] T 9e697e7146e34841bd64660b45ebf231 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:23.946705 11084 raft_consensus.cc:348] T a5d872c3620646e5ac33ceb95de2cbd4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:23.947587 11084 raft_consensus.cc:374] T a5d872c3620646e5ac33ceb95de2cbd4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:23.947892 11084 raft_consensus.cc:729] T a5d872c3620646e5ac33ceb95de2cbd4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:23.923194 11815 raft_consensus.cc:482] T a21909e2e5794f57971f4e42795235c7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260925 09:57:23.657634 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 4c4e8790282a49fd849a34ca11bef5bf because the tablet was not found. No further retry: Not found: Tablet not found: 4c4e8790282a49fd849a34ca11bef5bf
I20260925 09:57:23.898759 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 89a143a5623e4ef38027381a56c95227 (table default.Table_bb742c618d96453aaabe897d54fc657b [id=68683893b91c4f0190a0050ecdc6d396]) successfully deleted
I20260925 09:57:23.668522 10384 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_e29783826d5c4556865f6b2b079a00bc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:23.949179 11815 raft_consensus.cc:504] T a21909e2e5794f57971f4e42795235c7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:23.950030 10377 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 51480.1us
I20260925 09:57:23.936632 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 178) took 1948 ms. Trace:
W20260925 09:57:23.951622 10384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e29783826d5c4556865f6b2b079a00bc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:23.951107 11727 rpcz_store.cc:276] 0925 09:57:21.988490 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:21.989032 (+ 542us) service_pool.cc:227] Handling call
0925 09:57:23.694393 (+1705361us) ts_tablet_manager.cc:1145] Deleting tablet ec1b78e61e8d46cb8b87f60786059658
0925 09:57:23.694477 (+ 84us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:23.901957 (+207480us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:23.932971 (+ 31014us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:23.936571 (+ 3600us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":11538,"tablet-delete.queue_time_us":1697998}
I20260925 09:57:23.953255 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet ec1b78e61e8d46cb8b87f60786059658 (table default.Table_09fe8f366734488c93a79eeeec2c96f9 [id=a7023f3c32884ce8a7ca21fc95afc721]) successfully deleted
I20260925 09:57:23.903317 10388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_4353b0aceb674088b1805b3da35d0c68"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:23.956792 10388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4353b0aceb674088b1805b3da35d0c68 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:23.957440 11727 tablet_replica.cc:330] T 60fdc299865446c89d3e5374d573029a P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:23.658243 10389 master_service.cc:946] ReplaceTablet: received request to replace tablet 359090244c4748e8a2d45e3c77e89498 from {username='slave'} at 127.0.0.1:35140
I20260925 09:57:23.959110 11727 raft_consensus.cc:2229] T 60fdc299865446c89d3e5374d573029a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:23.959640 11727 raft_consensus.cc:2258] T 60fdc299865446c89d3e5374d573029a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:23.960572 11605 leader_election.cc:288] T 2f07dc9548364870923165f4ea524310 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:23.956354 11084 consensus_queue.cc:260] T a5d872c3620646e5ac33ceb95de2cbd4 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:23.964120 11084 ts_tablet_manager.cc:1428] T a5d872c3620646e5ac33ceb95de2cbd4 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.020s user 0.006s sys 0.000s
I20260925 09:57:23.967226 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f07dc9548364870923165f4ea524310" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:23.967543 10379 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 309745us
I20260925 09:57:23.971644 11627 consensus_queue.cc:1035] T 3d652a76d96f4dbeafde15f3c16534a6 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 09:57:23.974175 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 6161774f61ae48ab9764a05f2188a80b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6161774f61ae48ab9764a05f2188a80b already in progress: creating tablet
I20260925 09:57:23.974519 11084 tablet_bootstrap.cc:492] T dba14a44a9de492886d629441606767e P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:23.978571 11579 ts_tablet_manager.cc:1918] T 689ab2b6cf1640129f6db8b113cf2fc6 P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:23.978969 11579 log.cc:1196] T 689ab2b6cf1640129f6db8b113cf2fc6 P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/689ab2b6cf1640129f6db8b113cf2fc6
W20260925 09:57:23.982328 10282 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 49355.6us
I20260925 09:57:23.985970 10978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26fae3fbfe374f348a1729c21ec751b9" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:23.986603 10978 raft_consensus.cc:2454] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:23.988428 11727 ts_tablet_manager.cc:1905] T 60fdc299865446c89d3e5374d573029a P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:23.991855 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a21909e2e5794f57971f4e42795235c7" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:23.992470 10984 raft_consensus.cc:2454] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
W20260925 09:57:23.980836 10919 leader_election.cc:341] T 2f07dc9548364870923165f4ea524310 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:23.995527 10919 leader_election.cc:302] T 2f07dc9548364870923165f4ea524310 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:23.995533 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b555eeba505740988c704df5c734e7c0" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:23.996735 11751 raft_consensus.cc:2726] T 2f07dc9548364870923165f4ea524310 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:24.000947 11084 tablet_bootstrap.cc:654] T dba14a44a9de492886d629441606767e P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:24.002760 11873 tablet.cc:1722] T e57065f2bb224a56ad1c3518a0c375da P be97ab5d75934a648d4ab76341193ec0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 09:57:23.938145 11082 consensus_queue.cc:260] T 7360a869f3be45a884a0379f4ef78e85 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.006587 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 89a143a5623e4ef38027381a56c95227 (table default.Table_bb742c618d96453aaabe897d54fc657b [id=68683893b91c4f0190a0050ecdc6d396]) successfully deleted
W20260925 09:57:24.006872 10621 consensus_peers.cc:487] T 3d652a76d96f4dbeafde15f3c16534a6 P b8850818e2ee42a082802d0095c08d21 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260925 09:57:24.007941 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet ec1b78e61e8d46cb8b87f60786059658 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ec1b78e61e8d46cb8b87f60786059658 already in progress: creating tablet
W20260925 09:57:24.008620 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 60fdc299865446c89d3e5374d573029a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 60fdc299865446c89d3e5374d573029a already in progress: creating tablet
W20260925 09:57:24.009261 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 545ac025f2cb469da847979011a0b3ce because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 545ac025f2cb469da847979011a0b3ce already in progress: creating tablet
W20260925 09:57:24.009869 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet d9e454f82a4e4c429b58f32926319b7c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d9e454f82a4e4c429b58f32926319b7c already in progress: creating tablet
I20260925 09:57:24.005056 11815 leader_election.cc:288] T a21909e2e5794f57971f4e42795235c7 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:24.031383 10619 leader_election.cc:302] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:24.035435 11815 raft_consensus.cc:2781] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 09:57:24.035832 11815 raft_consensus.cc:482] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:24.036145 11815 raft_consensus.cc:3037] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:57:23.981364 11579 ts_tablet_manager.cc:1939] T 689ab2b6cf1640129f6db8b113cf2fc6 P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:24.027522 10834 raft_consensus.cc:3037] T b555eeba505740988c704df5c734e7c0 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:24.045372 11579 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 191) took 1351 ms. Trace:
I20260925 09:57:24.050125 10619 leader_election.cc:302] T a21909e2e5794f57971f4e42795235c7 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:24.053328 11830 raft_consensus.cc:2781] T a21909e2e5794f57971f4e42795235c7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:24.053686 11830 raft_consensus.cc:482] T a21909e2e5794f57971f4e42795235c7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:24.054003 11830 raft_consensus.cc:3037] T a21909e2e5794f57971f4e42795235c7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:24.033938 11085 tablet_bootstrap.cc:492] T ec1b78e61e8d46cb8b87f60786059658 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:24.056525 11085 ts_tablet_manager.cc:1397] T ec1b78e61e8d46cb8b87f60786059658 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.210s user 0.015s sys 0.003s
I20260925 09:57:24.060161 11085 raft_consensus.cc:348] T ec1b78e61e8d46cb8b87f60786059658 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.069370 11085 raft_consensus.cc:374] T ec1b78e61e8d46cb8b87f60786059658 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:24.055871 11579 rpcz_store.cc:276] 0925 09:57:22.695159 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:22.695555 (+ 396us) service_pool.cc:227] Handling call
0925 09:57:23.758964 (+1063409us) ts_tablet_manager.cc:1145] Deleting tablet 689ab2b6cf1640129f6db8b113cf2fc6
0925 09:57:23.758997 (+ 33us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:23.870636 (+111639us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:23.977971 (+107335us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:24.045309 (+ 67338us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1062714}
I20260925 09:57:24.070917 11579 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 196) took 1057 ms. Trace:
I20260925 09:57:24.071316 11579 rpcz_store.cc:276] 0925 09:57:23.013261 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:23.013361 (+ 100us) service_pool.cc:227] Handling call
0925 09:57:24.070621 (+1057260us) ts_tablet_manager.cc:1145] Deleting tablet 4be3f6e81ddf418fabc7b694a2e62f35
0925 09:57:24.070666 (+ 45us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:24.070861 (+ 195us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1056733}
I20260925 09:57:24.072867 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a21909e2e5794f57971f4e42795235c7" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:24.072204 11085 raft_consensus.cc:729] T ec1b78e61e8d46cb8b87f60786059658 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:24.075779 11085 consensus_queue.cc:260] T ec1b78e61e8d46cb8b87f60786059658 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.087153 11085 ts_tablet_manager.cc:1428] T ec1b78e61e8d46cb8b87f60786059658 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.030s user 0.005s sys 0.000s
I20260925 09:57:24.090763 11085 tablet_bootstrap.cc:492] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:24.071338 11082 ts_tablet_manager.cc:1428] T 7360a869f3be45a884a0379f4ef78e85 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.272s user 0.006s sys 0.001s
I20260925 09:57:24.096057 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 689ab2b6cf1640129f6db8b113cf2fc6 (table default.Table_077f38b3329348828ef80c0f4e0c972f [id=51887beca2a54a4bb1e8f4262bc4fb4a]) successfully deleted
I20260925 09:57:24.096570 11082 tablet_bootstrap.cc:492] T 55d14d4d8abe4ec9ac9aeef0d027e804 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:24.077585 11876 tablet.cc:1722] T e57065f2bb224a56ad1c3518a0c375da P c43461c4ee854ae1ac52b873ccada318: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 09:57:24.077317 11579 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 197) took 1063 ms. Trace:
I20260925 09:57:24.097922 11579 rpcz_store.cc:276] 0925 09:57:23.014252 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:23.014345 (+ 93us) service_pool.cc:227] Handling call
0925 09:57:24.076999 (+1062654us) ts_tablet_manager.cc:1145] Deleting tablet 41df75fe4b8e4856bdbb5ead3cf4fd24
0925 09:57:24.077045 (+ 46us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:24.077258 (+ 213us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1062159}
I20260925 09:57:24.099509 11579 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 198) took 1084 ms. Trace:
I20260925 09:57:24.099835 11579 rpcz_store.cc:276] 0925 09:57:23.015171 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:23.015251 (+ 80us) service_pool.cc:227] Handling call
0925 09:57:24.099183 (+1083932us) ts_tablet_manager.cc:1145] Deleting tablet c07c6205d1284a6fbb369c010a226da7
0925 09:57:24.099231 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:24.099451 (+ 220us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1083406}
I20260925 09:57:24.101286 11579 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 194) took 1088 ms. Trace:
I20260925 09:57:24.101600 11579 rpcz_store.cc:276] 0925 09:57:23.012707 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:23.018146 (+ 5439us) service_pool.cc:227] Handling call
0925 09:57:24.100997 (+1082851us) ts_tablet_manager.cc:1145] Deleting tablet c16e096038084b2b9148390b6e773294
0925 09:57:24.101036 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:24.101233 (+ 197us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1082344}
I20260925 09:57:24.103005 11579 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 195) took 1090 ms. Trace:
I20260925 09:57:24.103448 11579 rpcz_store.cc:276] 0925 09:57:23.013015 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:23.018767 (+ 5752us) service_pool.cc:227] Handling call
0925 09:57:24.102744 (+1083977us) ts_tablet_manager.cc:1145] Deleting tablet 2d55154ceb064f22bd6f37c8ae5dce74
0925 09:57:24.102786 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:24.102957 (+ 171us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1083484}
W20260925 09:57:24.107832 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 4be3f6e81ddf418fabc7b694a2e62f35 because the tablet was not found. No further retry: Not found: Tablet not found: 4be3f6e81ddf418fabc7b694a2e62f35
I20260925 09:57:24.108533 11085 tablet_bootstrap.cc:654] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:24.108865 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 41df75fe4b8e4856bdbb5ead3cf4fd24 because the tablet was not found. No further retry: Not found: Tablet not found: 41df75fe4b8e4856bdbb5ead3cf4fd24
W20260925 09:57:24.109727 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet c07c6205d1284a6fbb369c010a226da7 because the tablet was not found. No further retry: Not found: Tablet not found: c07c6205d1284a6fbb369c010a226da7
W20260925 09:57:24.110530 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet c16e096038084b2b9148390b6e773294 because the tablet was not found. No further retry: Not found: Tablet not found: c16e096038084b2b9148390b6e773294
W20260925 09:57:24.111315 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 2d55154ceb064f22bd6f37c8ae5dce74 because the tablet was not found. No further retry: Not found: Tablet not found: 2d55154ceb064f22bd6f37c8ae5dce74
I20260925 09:57:24.112751 11579 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 193) took 1100 ms. Trace:
I20260925 09:57:24.112874 11830 raft_consensus.cc:504] T a21909e2e5794f57971f4e42795235c7 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.113107 11579 rpcz_store.cc:276] 0925 09:57:23.012450 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:23.026686 (+ 14236us) service_pool.cc:227] Handling call
0925 09:57:24.112095 (+1085409us) ts_tablet_manager.cc:1145] Deleting tablet 752105c1514f4c1db2a11b95b413b4f6
0925 09:57:24.112496 (+ 401us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:24.112693 (+ 197us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1084945}
I20260925 09:57:24.121237 11815 raft_consensus.cc:504] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.124336 11085 tablet_bootstrap.cc:492] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:24.124787 11085 ts_tablet_manager.cc:1397] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.034s user 0.006s sys 0.004s
I20260925 09:57:24.131920 11085 raft_consensus.cc:348] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.133895 11085 raft_consensus.cc:374] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:24.134258 11085 raft_consensus.cc:729] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:24.136085 11085 consensus_queue.cc:260] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.138640 11709 ts_tablet_manager.cc:1918] T 689ab2b6cf1640129f6db8b113cf2fc6 P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:24.139158 11709 log.cc:1196] T 689ab2b6cf1640129f6db8b113cf2fc6 P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/689ab2b6cf1640129f6db8b113cf2fc6
I20260925 09:57:24.145905 11085 ts_tablet_manager.cc:1428] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.021s user 0.002s sys 0.004s
W20260925 09:57:24.149160 10619 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 86902.2us
I20260925 09:57:24.150014 11085 tablet_bootstrap.cc:492] T 3706253e53d841aeaa43889ead13af32 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:24.151652 11709 ts_tablet_manager.cc:1939] T 689ab2b6cf1640129f6db8b113cf2fc6 P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
W20260925 09:57:24.159749 10379 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 48459.5us
I20260925 09:57:24.164196 11881 tablet.cc:1722] T b555eeba505740988c704df5c734e7c0 P c43461c4ee854ae1ac52b873ccada318: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:24.174528 11085 tablet_bootstrap.cc:654] T 3706253e53d841aeaa43889ead13af32 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:24.195719 11709 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 191) took 1345 ms. Trace:
I20260925 09:57:24.196185 11709 rpcz_store.cc:276] 0925 09:57:22.849842 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:22.853563 (+ 3721us) service_pool.cc:227] Handling call
0925 09:57:23.394727 (+541164us) ts_tablet_manager.cc:1145] Deleting tablet 689ab2b6cf1640129f6db8b113cf2fc6
0925 09:57:23.394772 (+ 45us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:23.924819 (+530047us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:24.138322 (+213503us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:24.195622 (+ 57300us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":129707,"tablet-delete.queue_time_us":540592}
I20260925 09:57:24.197993 11085 tablet_bootstrap.cc:492] T 3706253e53d841aeaa43889ead13af32 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:24.198383 11085 ts_tablet_manager.cc:1397] T 3706253e53d841aeaa43889ead13af32 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.049s user 0.005s sys 0.009s
I20260925 09:57:24.200942 11085 raft_consensus.cc:348] T 3706253e53d841aeaa43889ead13af32 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.201602 11085 raft_consensus.cc:374] T 3706253e53d841aeaa43889ead13af32 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:24.201858 11085 raft_consensus.cc:729] T 3706253e53d841aeaa43889ead13af32 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:24.202579 11085 consensus_queue.cc:260] T 3706253e53d841aeaa43889ead13af32 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.203701 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 689ab2b6cf1640129f6db8b113cf2fc6 (table default.Table_077f38b3329348828ef80c0f4e0c972f [id=51887beca2a54a4bb1e8f4262bc4fb4a]) successfully deleted
I20260925 09:57:24.203657 11815 leader_election.cc:288] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 2 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:24.204974 11830 leader_election.cc:288] T a21909e2e5794f57971f4e42795235c7 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:24.206076 11779 raft_consensus.cc:482] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:24.206552 11779 raft_consensus.cc:504] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.208563 11779 leader_election.cc:288] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:24.211582 11085 ts_tablet_manager.cc:1428] T 3706253e53d841aeaa43889ead13af32 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.013s user 0.003s sys 0.003s
I20260925 09:57:24.215222 11085 tablet_bootstrap.cc:492] T 3d652a76d96f4dbeafde15f3c16534a6 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:24.220810 11861 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:37876 (request call id 192) took 1370 ms. Trace:
I20260925 09:57:24.224444 10775 leader_election.cc:302] T b555eeba505740988c704df5c734e7c0 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:24.225975 11779 raft_consensus.cc:2726] T b555eeba505740988c704df5c734e7c0 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:24.227237 10379 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.62:34455, user_credentials={real_user=slave}} blocked reactor thread for 66658.3us
W20260925 09:57:24.227919 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 752105c1514f4c1db2a11b95b413b4f6 because the tablet was not found. No further retry: Not found: Tablet not found: 752105c1514f4c1db2a11b95b413b4f6
I20260925 09:57:24.230268 10834 raft_consensus.cc:3037] T 229aaa1fc7b84fb887a8c0d8a53bf0ae P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:24.234740 10833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a21909e2e5794f57971f4e42795235c7" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:24.235713 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26fae3fbfe374f348a1729c21ec751b9" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:24.236011 11085 tablet_bootstrap.cc:654] T 3d652a76d96f4dbeafde15f3c16534a6 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:24.236287 10832 raft_consensus.cc:3037] T 26fae3fbfe374f348a1729c21ec751b9 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:57:24.236150 11709 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 196) took 1361 ms. Trace:
I20260925 09:57:24.236685 11709 rpcz_store.cc:276] 0925 09:57:22.874640 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:22.874829 (+ 189us) service_pool.cc:227] Handling call
0925 09:57:24.235763 (+1360934us) ts_tablet_manager.cc:1145] Deleting tablet 04fafb9dcf274806b11b2598e73853e9
0925 09:57:24.235834 (+ 71us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:24.236072 (+ 238us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1360247}
W20260925 09:57:24.238730 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 04fafb9dcf274806b11b2598e73853e9 because the tablet was not found. No further retry: Not found: Tablet not found: 04fafb9dcf274806b11b2598e73853e9
I20260925 09:57:24.240650 10835 raft_consensus.cc:3037] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:57:24.228602 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 7a31b999f0fd447eacff13de71aa3fff because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7a31b999f0fd447eacff13de71aa3fff already in progress: creating tablet
I20260925 09:57:24.246160 10688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93e1b5837aff48d7907890c40dfcd0d0" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:24.256013 11082 tablet_bootstrap.cc:654] T 55d14d4d8abe4ec9ac9aeef0d027e804 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:24.260823 10621 leader_election.cc:341] T a21909e2e5794f57971f4e42795235c7 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:24.221280 11861 rpcz_store.cc:276] 0925 09:57:22.850087 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:22.857774 (+ 7687us) service_pool.cc:227] Handling call
0925 09:57:24.220725 (+1362951us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:22.879962 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0925 09:57:22.911770 (+ 31808us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0925 09:57:22.911832 (+ 62us) alter_schema_op.cc:65] Acquired schema lock
0925 09:57:22.911927 (+ 95us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0925 09:57:22.949545 (+ 37618us) spinlock_profiling.cc:243] Waited 36 ms on lock 0x7b180006b940. stack: 00007fb965334033 00007fb965333f30 00007fb964eb5534 00000000004db2b4 00000000004db249 00007fb96c240471 00007fb96c2f694d 00007fb96c7187a0 00007fb96c7182b8 00007fb96c71cba1 00007fb96c71cb59 00007fb96c71cae9 00007fb96c71cab1 00007fb96c71bdad 0000000000580784 00000000005805b9
0925 09:57:22.949613 (+ 68us) alter_schema_op.cc:126] START. Timestamp: 1790330242947849
0925 09:57:22.949636 (+ 23us) op_driver.cc:331] REPLICATION: starting
0925 09:57:22.950192 (+ 556us) log.cc:842] Serialized 173 byte log entry
0925 09:57:23.748923 (+798731us) op_driver.cc:444] REPLICATION: finished
0925 09:57:23.891935 (+143012us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0925 09:57:23.936684 (+ 44749us) log.cc:842] Serialized 5 byte log entry
0925 09:57:24.187072 (+250388us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:24.220425 (+ 33353us) log.cc:842] Serialized 14 byte log entry
0925 09:57:24.220575 (+ 150us) alter_schema_op.cc:71] Released schema lock
0925 09:57:24.220607 (+ 32us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":142768,"prepare.queue_time_us":12966,"prepare.run_cpu_time_us":14383,"prepare.run_wall_time_us":250508,"replication_time_us":799195,"spinlock_wait_cycles":82775424,"thread_start_us":316187,"threads_started":2,"wal-append.queue_time_us":170399}]]}
I20260925 09:57:24.292629 11709 tablet_replica.cc:330] T cf35f39bcd6a41128acb2a4936fcc316 P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:24.293768 11709 raft_consensus.cc:2229] T cf35f39bcd6a41128acb2a4936fcc316 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:24.294745 11709 raft_consensus.cc:2258] T cf35f39bcd6a41128acb2a4936fcc316 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:24.295691 11861 tablet.cc:1722] T 42ec275c905343c4be1b72dc2bd0f6e3 P b8850818e2ee42a082802d0095c08d21: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260925 09:57:24.308406 10621 leader_election.cc:341] T a21909e2e5794f57971f4e42795235c7 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:24.315361 11082 tablet_bootstrap.cc:492] T 55d14d4d8abe4ec9ac9aeef0d027e804 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:24.315900 11082 ts_tablet_manager.cc:1397] T 55d14d4d8abe4ec9ac9aeef0d027e804 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.220s user 0.014s sys 0.000s
I20260925 09:57:24.311725 10832 raft_consensus.cc:2454] T 26fae3fbfe374f348a1729c21ec751b9 P c43461c4ee854ae1ac52b873ccada318 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 2.
I20260925 09:57:24.329034 11709 ts_tablet_manager.cc:1905] T cf35f39bcd6a41128acb2a4936fcc316 P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:24.335379 10390 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 3706253e53d841aeaa43889ead13af32
I20260925 09:57:24.337368 10390 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:35140 (request call id 215) took 1772 ms. Trace:
I20260925 09:57:24.337957 10663 tablet_service.cc:1514] Processing DeleteTablet for tablet 3706253e53d841aeaa43889ead13af32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:23 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:24.337759 10390 rpcz_store.cc:276] 0925 09:57:22.564560 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:22.957957 (+393397us) service_pool.cc:227] Handling call
0925 09:57:23.066901 (+108944us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:57:23.067485 (+ 584us) catalog_manager.cc:2752] Modifying in-memory table state
0925 09:57:23.067602 (+ 117us) catalog_manager.cc:2761] Locking tablets
0925 09:57:24.196354 (+1128752us) catalog_manager.cc:2775] Removing table and tablets from system table
0925 09:57:24.334714 (+138360us) catalog_manager.cc:2793] Removing table from by-name map
0925 09:57:24.335003 (+ 289us) catalog_manager.cc:2809] Committing in-memory state
0925 09:57:24.335150 (+ 147us) catalog_manager.cc:2813] Aborting table tasks
0925 09:57:24.337210 (+ 2060us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:24.201501 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:24.201886 (+ 385us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:24.201939 (+ 53us) write_op.cc:435] Acquired schema lock
0925 09:57:24.201954 (+ 15us) tablet.cc:660] Decoding operations
0925 09:57:24.202298 (+ 344us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:24.202342 (+ 44us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:24.202371 (+ 29us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:24.202654 (+ 283us) tablet.cc:699] Row locks acquired
0925 09:57:24.202669 (+ 15us) write_op.cc:260] PREPARE: finished
0925 09:57:24.202787 (+ 118us) write_op.cc:270] Start()
0925 09:57:24.202874 (+ 87us) write_op.cc:276] Timestamp: P: 1790330244202671 usec, L: 0
0925 09:57:24.202891 (+ 17us) op_driver.cc:331] REPLICATION: starting
0925 09:57:24.284054 (+ 81163us) spinlock_profiling.cc:243] Waited 6.27 ms on lock 0x7b5400012320. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90471 00007fabf0aa0dc0 00007fabf0b11bd2 00007fabf0b072d7 00007fabf0b07114 00007fabf0f68ade 00007fabf0f682b8 00007fabf0f6cba1 00007fabf0f6cb59 00007fabf0f6cae9 00007fabf0f6cab1
0925 09:57:24.284624 (+ 570us) log.cc:842] Serialized 756 byte log entry
0925 09:57:24.285492 (+ 868us) spinlock_profiling.cc:243] Waited 73 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90471 00007fabf0b07198 00007fabf0f68ade 00007fabf0f682b8 00007fabf0f6cba1 00007fabf0f6cb59 00007fabf0f6cae9 00007fabf0f6cab1 00007fabf0f6bdad 0000000000580784 00000000005805b9
0925 09:57:24.299219 (+ 13727us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b58000b03b0. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0a85834 00007fabf0a8a771 00007fabf0a8a6f9 00007fabf0a8a689 00007fabf0a8a651 00007fabf0a8981d 0000000000580784 00000000005805b9 00007fabe9bb66b6 00007fabe9bb9b11
0925 09:57:24.329961 (+ 30742us) op_driver.cc:444] REPLICATION: finished
0925 09:57:24.331105 (+ 1144us) write_op.cc:301] APPLY: starting
0925 09:57:24.331179 (+ 74us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:24.331335 (+ 156us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:24.331366 (+ 31us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:24.331742 (+ 376us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:24.331908 (+ 166us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 09:57:24.331975 (+ 67us) write_op.cc:312] APPLY: finished
0925 09:57:24.332210 (+ 235us) log.cc:842] Serialized 29 byte log entry
0925 09:57:24.332400 (+ 190us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:24.332490 (+ 90us) write_op.cc:454] Released schema lock
0925 09:57:24.332533 (+ 43us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:24.332739 (+ 206us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:24.332777 (+ 38us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":982,"apply.run_cpu_time_us":1863,"apply.run_wall_time_us":1862,"num_ops":2,"prepare.queue_time_us":2913,"prepare.run_cpu_time_us":10113,"prepare.run_wall_time_us":103384,"raft.queue_time_us":15186,"raft.run_cpu_time_us":2101,"raft.run_wall_time_us":12917,"replication_time_us":126983,"spinlock_wait_cycles":207903232,"thread_start_us":19090,"threads_started":1}]]}
I20260925 09:57:24.350432 10819 tablet_service.cc:1514] Processing DeleteTablet for tablet 3706253e53d841aeaa43889ead13af32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:23 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:24.351148 11579 tablet_replica.cc:330] T 3706253e53d841aeaa43889ead13af32 P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:24.351979 11579 raft_consensus.cc:2229] T 3706253e53d841aeaa43889ead13af32 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:24.352415 11579 raft_consensus.cc:2258] T 3706253e53d841aeaa43889ead13af32 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:24.354944 11579 ts_tablet_manager.cc:1905] T 3706253e53d841aeaa43889ead13af32 P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:24.361686 11830 raft_consensus.cc:482] T 7360a869f3be45a884a0379f4ef78e85 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:24.362311 11830 raft_consensus.cc:504] T 7360a869f3be45a884a0379f4ef78e85 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.368360 11830 leader_election.cc:288] T 7360a869f3be45a884a0379f4ef78e85 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
W20260925 09:57:24.382249 10775 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 69586.9us
I20260925 09:57:24.386515 10956 tablet_service.cc:1514] Processing DeleteTablet for tablet 3706253e53d841aeaa43889ead13af32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:23 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:24.388443 10972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7360a869f3be45a884a0379f4ef78e85" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:24.389926 10972 raft_consensus.cc:2454] T 7360a869f3be45a884a0379f4ef78e85 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:24.392092 10984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93e1b5837aff48d7907890c40dfcd0d0" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:24.394493 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26fae3fbfe374f348a1729c21ec751b9" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:24.395090 10976 raft_consensus.cc:3037] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:57:24.395898 10619 leader_election.cc:302] T 7360a869f3be45a884a0379f4ef78e85 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:24.397673 11815 raft_consensus.cc:2781] T 7360a869f3be45a884a0379f4ef78e85 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:24.398000 11815 raft_consensus.cc:482] T 7360a869f3be45a884a0379f4ef78e85 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:24.398317 11815 raft_consensus.cc:3037] T 7360a869f3be45a884a0379f4ef78e85 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:24.399344 10978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a21909e2e5794f57971f4e42795235c7" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:24.408870 10978 raft_consensus.cc:3037] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:24.408772 10772 leader_election.cc:302] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:24.409615 11889 raft_consensus.cc:2726] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:24.411979 11085 tablet_bootstrap.cc:492] T 3d652a76d96f4dbeafde15f3c16534a6 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:24.412438 11085 ts_tablet_manager.cc:1397] T 3d652a76d96f4dbeafde15f3c16534a6 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.197s user 0.008s sys 0.006s
I20260925 09:57:24.415086 11085 raft_consensus.cc:348] T 3d652a76d96f4dbeafde15f3c16534a6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.415807 11085 raft_consensus.cc:374] T 3d652a76d96f4dbeafde15f3c16534a6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:24.416074 11085 raft_consensus.cc:729] T 3d652a76d96f4dbeafde15f3c16534a6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:24.424815 11815 raft_consensus.cc:504] T 7360a869f3be45a884a0379f4ef78e85 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.427563 10978 raft_consensus.cc:2454] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:24.428138 10981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7360a869f3be45a884a0379f4ef78e85" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:24.429563 10981 raft_consensus.cc:3037] T 7360a869f3be45a884a0379f4ef78e85 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:24.439549 11579 ts_tablet_manager.cc:1918] T 3706253e53d841aeaa43889ead13af32 P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:24.440008 11579 log.cc:1196] T 3706253e53d841aeaa43889ead13af32 P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/3706253e53d841aeaa43889ead13af32
I20260925 09:57:24.440701 11579 ts_tablet_manager.cc:1939] T 3706253e53d841aeaa43889ead13af32 P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:24.443269 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 3706253e53d841aeaa43889ead13af32 (table default.Table_bbc3c50c06634d119ae96dcf649f44e7 [id=0260f0e6cdd6431db06788bde54ec38c]) successfully deleted
W20260925 09:57:24.451983 11875 log.cc:924] Time spent T 3d652a76d96f4dbeafde15f3c16534a6 P be97ab5d75934a648d4ab76341193ec0: Append to log took a long time: real 0.424s user 0.000s sys 0.001s
I20260925 09:57:24.453794 11084 tablet_bootstrap.cc:492] T dba14a44a9de492886d629441606767e P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
W20260925 09:57:24.454545 11873 tablet_replica.cc:1402] Time spent flushing the Log queue took a long time: real 0.450s user 0.000s sys 0.001s
I20260925 09:57:24.454659 11084 ts_tablet_manager.cc:1397] T dba14a44a9de492886d629441606767e P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.480s user 0.014s sys 0.001s
I20260925 09:57:24.463686 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7360a869f3be45a884a0379f4ef78e85" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:24.429932 11082 raft_consensus.cc:348] T 55d14d4d8abe4ec9ac9aeef0d027e804 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.464860 11082 raft_consensus.cc:374] T 55d14d4d8abe4ec9ac9aeef0d027e804 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:24.465180 11082 raft_consensus.cc:729] T 55d14d4d8abe4ec9ac9aeef0d027e804 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:24.465246 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7360a869f3be45a884a0379f4ef78e85" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:24.466338 11082 consensus_queue.cc:260] T 55d14d4d8abe4ec9ac9aeef0d027e804 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.467762 10389 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 359090244c4748e8a2d45e3c77e89498
W20260925 09:57:24.469393 10380 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 4e3120347c8e4a3eab57d6562ae40e8d (table default.Table_8e186a99ccba425b9b588041d2364a40 [id=2d6bc0e2aaee4b1483b01fc678505bb0], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 4e3120347c8e4a3eab57d6562ae40e8d consensus state has no leader
I20260925 09:57:24.470117 10819 tablet_service.cc:1514] Processing DeleteTablet for tablet 359090244c4748e8a2d45e3c77e89498 with delete_type TABLET_DATA_DELETED (replaced by tablet b39f9e7102ff4a9c96a2cc5d2676f03d) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:24.470978 10663 tablet_service.cc:1514] Processing DeleteTablet for tablet 359090244c4748e8a2d45e3c77e89498 with delete_type TABLET_DATA_DELETED (replaced by tablet b39f9e7102ff4a9c96a2cc5d2676f03d) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:24.472435 10389 catalog_manager.cc:6565] ReplaceTablet: tablet 359090244c4748e8a2d45e3c77e89498 deleted and replaced by tablet b39f9e7102ff4a9c96a2cc5d2676f03d
I20260925 09:57:24.416666 11085 consensus_queue.cc:260] T 3d652a76d96f4dbeafde15f3c16534a6 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.476102 11751 raft_consensus.cc:482] T 69a96874eea1409a9a5a06b5787fbd99 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:24.476647 10956 tablet_service.cc:1514] Processing DeleteTablet for tablet 359090244c4748e8a2d45e3c77e89498 with delete_type TABLET_DATA_DELETED (replaced by tablet b39f9e7102ff4a9c96a2cc5d2676f03d) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:24.476688 11751 raft_consensus.cc:504] T 69a96874eea1409a9a5a06b5787fbd99 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.034446 11727 ts_tablet_manager.cc:1918] T 60fdc299865446c89d3e5374d573029a P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:24.479377 10386 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 26fae3fbfe374f348a1729c21ec751b9 (table default.Table_0011387b45014de4b24b466f4cb907f7 [id=e3f7aa5e988c4b6c98be5cd3d465245d]). Expected version 1 got 0
I20260925 09:57:24.479862 11727 log.cc:1196] T 60fdc299865446c89d3e5374d573029a P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/60fdc299865446c89d3e5374d573029a
I20260925 09:57:24.480808 11727 ts_tablet_manager.cc:1939] T 60fdc299865446c89d3e5374d573029a P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:24.482398 11085 ts_tablet_manager.cc:1428] T 3d652a76d96f4dbeafde15f3c16534a6 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.070s user 0.005s sys 0.000s
I20260925 09:57:24.482702 10389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_b1ed2ff673484b57bc734908b3c0ddf3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:24.483951 10619 leader_election.cc:302] T a21909e2e5794f57971f4e42795235c7 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
W20260925 09:57:24.484654 10389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b1ed2ff673484b57bc734908b3c0ddf3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:24.488523 10976 raft_consensus.cc:2454] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 2.
I20260925 09:57:24.494738 10981 raft_consensus.cc:2454] T 7360a869f3be45a884a0379f4ef78e85 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:24.479151 11751 leader_election.cc:288] T 69a96874eea1409a9a5a06b5787fbd99 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:24.495473 11085 tablet_bootstrap.cc:492] T 7360a869f3be45a884a0379f4ef78e85 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:24.497130 10386 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet e57065f2bb224a56ad1c3518a0c375da (table default.Table_6eff7b87b3b74ca4bab719140667a87b [id=7889081642604700bbc6d253ab105260]). Expected version 2 got 1
I20260925 09:57:24.501542 11815 leader_election.cc:288] T 7360a869f3be45a884a0379f4ef78e85 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:24.502285 10386 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 42ec275c905343c4be1b72dc2bd0f6e3 (table default.Table_c22088d959e84c3cbabadde4d3445178 [id=cf07ed8756904011ab3dac900ea6705a]). Expected version 1 got 0
I20260925 09:57:24.502956 10619 leader_election.cc:302] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:24.503746 10619 leader_election.cc:302] T 7360a869f3be45a884a0379f4ef78e85 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:24.504449 11801 raft_consensus.cc:2781] T a21909e2e5794f57971f4e42795235c7 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:24.504843 11801 raft_consensus.cc:686] T a21909e2e5794f57971f4e42795235c7 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:24.505589 11801 consensus_queue.cc:237] T a21909e2e5794f57971f4e42795235c7 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.506378 11579 tablet_replica.cc:330] T 359090244c4748e8a2d45e3c77e89498 P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:24.508912 11836 raft_consensus.cc:2781] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 09:57:24.509370 11836 raft_consensus.cc:686] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21 [term 2 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:24.509713 11579 raft_consensus.cc:2229] T 359090244c4748e8a2d45e3c77e89498 P c43461c4ee854ae1ac52b873ccada318 [term 2 LEADER]: Raft consensus shutting down.
I20260925 09:57:24.524490 11742 raft_consensus.cc:482] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:24.510152 11836 consensus_queue.cc:237] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.524956 11742 raft_consensus.cc:504] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.523096 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 179) took 2374 ms. Trace:
I20260925 09:57:24.526835 11742 leader_election.cc:288] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:24.524820 11851 raft_consensus.cc:2781] T 7360a869f3be45a884a0379f4ef78e85 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:24.528915 11851 raft_consensus.cc:686] T 7360a869f3be45a884a0379f4ef78e85 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:24.530397 11084 raft_consensus.cc:348] T dba14a44a9de492886d629441606767e P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.531117 11084 raft_consensus.cc:374] T dba14a44a9de492886d629441606767e P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:24.531366 11084 raft_consensus.cc:729] T dba14a44a9de492886d629441606767e P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:24.531961 11084 consensus_queue.cc:260] T dba14a44a9de492886d629441606767e P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.527827 11727 rpcz_store.cc:276] 0925 09:57:22.148597 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:22.148697 (+ 100us) service_pool.cc:227] Handling call
0925 09:57:23.957271 (+1808574us) ts_tablet_manager.cc:1145] Deleting tablet 60fdc299865446c89d3e5374d573029a
0925 09:57:23.957321 (+ 50us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:24.014006 (+ 56685us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:24.034182 (+ 20176us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:24.522962 (+488780us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":38797,"tablet-delete.queue_time_us":1808032}
I20260925 09:57:24.533596 11579 raft_consensus.cc:2258] T 359090244c4748e8a2d45e3c77e89498 P c43461c4ee854ae1ac52b873ccada318 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:24.534003 11084 ts_tablet_manager.cc:1428] T dba14a44a9de492886d629441606767e P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.079s user 0.001s sys 0.008s
I20260925 09:57:24.535620 11084 tablet_bootstrap.cc:492] T 37bab60afe7a4c758a08ef7c9c24aba0 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:24.537128 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 60fdc299865446c89d3e5374d573029a (table default.Table_32e7bdf7eaf147ea90740b6b3cfb2fd3 [id=be059d72d5db4915ba767959c6a1c7b2]) successfully deleted
I20260925 09:57:24.529686 11851 consensus_queue.cc:237] T 7360a869f3be45a884a0379f4ef78e85 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.540676 11579 ts_tablet_manager.cc:1905] T 359090244c4748e8a2d45e3c77e89498 P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:24.544129 11085 tablet_bootstrap.cc:654] T 7360a869f3be45a884a0379f4ef78e85 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:24.544055 10389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_62f952bfd0724ec792a45a6eb99a82c7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:24.551991 10389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_62f952bfd0724ec792a45a6eb99a82c7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:24.574447 11895 raft_consensus.cc:482] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:24.574935 11895 raft_consensus.cc:504] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.576864 11895 leader_election.cc:288] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:24.583751 11727 tablet_replica.cc:330] T 545ac025f2cb469da847979011a0b3ce P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:24.583901 10389 master_service.cc:946] ReplaceTablet: received request to replace tablet 89a143a5623e4ef38027381a56c95227 from {username='slave'} at 127.0.0.1:35140
I20260925 09:57:24.583415 11082 ts_tablet_manager.cc:1428] T 55d14d4d8abe4ec9ac9aeef0d027e804 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.252s user 0.005s sys 0.003s
I20260925 09:57:24.587154 11084 tablet_bootstrap.cc:654] T 37bab60afe7a4c758a08ef7c9c24aba0 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:24.588078 10389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_175fa05d29fa47b8a45cd68e94d78800"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:24.588649 11082 tablet_bootstrap.cc:492] T d9e454f82a4e4c429b58f32926319b7c P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
W20260925 09:57:24.592100 10621 leader_election.cc:341] T 7360a869f3be45a884a0379f4ef78e85 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:24.592962 10621 leader_election.cc:341] T 7360a869f3be45a884a0379f4ef78e85 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:24.593395 11892 raft_consensus.cc:482] T 4e3120347c8e4a3eab57d6562ae40e8d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:24.593843 11892 raft_consensus.cc:504] T 4e3120347c8e4a3eab57d6562ae40e8d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.595927 11892 leader_election.cc:288] T 4e3120347c8e4a3eab57d6562ae40e8d P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
W20260925 09:57:24.595889 10389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_175fa05d29fa47b8a45cd68e94d78800 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:24.604406 10688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69a96874eea1409a9a5a06b5787fbd99" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:24.607067 11085 tablet_bootstrap.cc:492] T 7360a869f3be45a884a0379f4ef78e85 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:24.609117 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ff4cf556eee494d8cd590884466f09f" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:24.623835 10919 leader_election.cc:341] T 69a96874eea1409a9a5a06b5787fbd99 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:24.624718 10919 consensus_peers.cc:487] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0 -> Peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Couldn't send request to peer b8850818e2ee42a082802d0095c08d21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 689ab2b6cf1640129f6db8b113cf2fc6. This is attempt 1: this message will repeat every 5th retry.
W20260925 09:57:24.625512 10919 leader_election.cc:341] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:24.626262 10918 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 207459us
I20260925 09:57:24.628086 11084 tablet_bootstrap.cc:492] T 37bab60afe7a4c758a08ef7c9c24aba0 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:24.631626 11084 ts_tablet_manager.cc:1397] T 37bab60afe7a4c758a08ef7c9c24aba0 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.096s user 0.012s sys 0.003s
I20260925 09:57:24.633704 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e3120347c8e4a3eab57d6562ae40e8d" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:24.636163 10688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb9c51f7e7fd4c029be2a710cb9462eb" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:24.637082 11084 raft_consensus.cc:348] T 37bab60afe7a4c758a08ef7c9c24aba0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.641404 11082 tablet_bootstrap.cc:654] T d9e454f82a4e4c429b58f32926319b7c P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:24.641707 11084 raft_consensus.cc:374] T 37bab60afe7a4c758a08ef7c9c24aba0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:24.647372 11085 ts_tablet_manager.cc:1397] T 7360a869f3be45a884a0379f4ef78e85 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.152s user 0.006s sys 0.004s
I20260925 09:57:24.647449 11084 raft_consensus.cc:729] T 37bab60afe7a4c758a08ef7c9c24aba0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:24.648061 11084 consensus_queue.cc:260] T 37bab60afe7a4c758a08ef7c9c24aba0 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.650223 11084 ts_tablet_manager.cc:1428] T 37bab60afe7a4c758a08ef7c9c24aba0 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.015s user 0.004s sys 0.000s
I20260925 09:57:24.649866 11085 raft_consensus.cc:348] T 7360a869f3be45a884a0379f4ef78e85 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.650542 11085 raft_consensus.cc:374] T 7360a869f3be45a884a0379f4ef78e85 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:24.650892 11085 raft_consensus.cc:729] T 7360a869f3be45a884a0379f4ef78e85 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:24.651113 11084 tablet_bootstrap.cc:492] T 8aff53cfd4994c629a408ee8250280c5 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:24.652624 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69a96874eea1409a9a5a06b5787fbd99" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:24.654548 11727 raft_consensus.cc:2229] T 545ac025f2cb469da847979011a0b3ce P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:24.655093 11727 raft_consensus.cc:2258] T 545ac025f2cb469da847979011a0b3ce P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:24.657356 11727 ts_tablet_manager.cc:1905] T 545ac025f2cb469da847979011a0b3ce P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:24.666669 11084 tablet_bootstrap.cc:654] T 8aff53cfd4994c629a408ee8250280c5 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:24.655095 10919 leader_election.cc:341] T 4e3120347c8e4a3eab57d6562ae40e8d P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:24.669193 11709 ts_tablet_manager.cc:1918] T cf35f39bcd6a41128acb2a4936fcc316 P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260925 09:57:24.669070 10919 leader_election.cc:341] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:24.670130 11709 log.cc:1196] T cf35f39bcd6a41128acb2a4936fcc316 P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/cf35f39bcd6a41128acb2a4936fcc316
I20260925 09:57:24.670930 11709 ts_tablet_manager.cc:1939] T cf35f39bcd6a41128acb2a4936fcc316 P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:24.678927 10389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_757681cc28db4f3695a9df63d0260883"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:24.680510 10389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_757681cc28db4f3695a9df63d0260883 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:24.689690 11085 consensus_queue.cc:260] T 7360a869f3be45a884a0379f4ef78e85 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.692068 11085 ts_tablet_manager.cc:1428] T 7360a869f3be45a884a0379f4ef78e85 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.044s user 0.006s sys 0.000s
I20260925 09:57:24.702469 11082 tablet_bootstrap.cc:492] T d9e454f82a4e4c429b58f32926319b7c P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:24.703008 11082 ts_tablet_manager.cc:1397] T d9e454f82a4e4c429b58f32926319b7c P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.115s user 0.010s sys 0.000s
I20260925 09:57:24.708724 11901 raft_consensus.cc:482] T dba14a44a9de492886d629441606767e P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:24.709151 11901 raft_consensus.cc:504] T dba14a44a9de492886d629441606767e P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.711184 11901 leader_election.cc:288] T dba14a44a9de492886d629441606767e P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:24.712097 11709 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 197) took 1833 ms. Trace:
W20260925 09:57:24.714560 10918 leader_election.cc:341] T 69a96874eea1409a9a5a06b5787fbd99 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:24.715137 10918 leader_election.cc:302] T 69a96874eea1409a9a5a06b5787fbd99 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:24.712785 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dba14a44a9de492886d629441606767e" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:24.716279 11901 raft_consensus.cc:2726] T 69a96874eea1409a9a5a06b5787fbd99 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:24.716714 10919 leader_election.cc:341] T dba14a44a9de492886d629441606767e P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:24.717183 11709 rpcz_store.cc:276] 0925 09:57:22.878380 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:22.878513 (+ 133us) service_pool.cc:227] Handling call
0925 09:57:24.284871 (+1406358us) ts_tablet_manager.cc:1145] Deleting tablet cf35f39bcd6a41128acb2a4936fcc316
0925 09:57:24.284930 (+ 59us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:24.595702 (+310772us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:24.668963 (+ 73261us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:24.712043 (+ 43080us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1405870}
I20260925 09:57:24.718348 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet cf35f39bcd6a41128acb2a4936fcc316 (table default.Table_fea25102a7f84d58927645a0feae51e3 [id=61a4c50b2c6e43c1a471f63b59d218cb]) successfully deleted
I20260925 09:57:24.715186 11082 raft_consensus.cc:348] T d9e454f82a4e4c429b58f32926319b7c P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.719609 11082 raft_consensus.cc:374] T d9e454f82a4e4c429b58f32926319b7c P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:24.719851 11082 raft_consensus.cc:729] T d9e454f82a4e4c429b58f32926319b7c P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:24.720610 11082 consensus_queue.cc:260] T d9e454f82a4e4c429b58f32926319b7c P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.724772 11709 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 200) took 1825 ms. Trace:
I20260925 09:57:24.728891 10389 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_ce09d3a8d8e84973894f33b18bbfdb22" } modify_external_catalogs: true
I20260925 09:57:24.729470 10389 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_ce09d3a8d8e84973894f33b18bbfdb22" } modify_external_catalogs: true
I20260925 09:57:24.745471 11084 tablet_bootstrap.cc:492] T 8aff53cfd4994c629a408ee8250280c5 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:24.745857 11084 ts_tablet_manager.cc:1397] T 8aff53cfd4994c629a408ee8250280c5 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.095s user 0.011s sys 0.004s
I20260925 09:57:24.725172 11709 rpcz_store.cc:276] 0925 09:57:22.899652 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:22.899818 (+ 166us) service_pool.cc:227] Handling call
0925 09:57:24.724490 (+1824672us) ts_tablet_manager.cc:1145] Deleting tablet 89a143a5623e4ef38027381a56c95227
0925 09:57:24.724545 (+ 55us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:24.724718 (+ 173us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1824014}
I20260925 09:57:24.750864 11082 ts_tablet_manager.cc:1428] T d9e454f82a4e4c429b58f32926319b7c P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.048s user 0.005s sys 0.000s
I20260925 09:57:24.752032 11815 raft_consensus.cc:482] T d9e454f82a4e4c429b58f32926319b7c P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:24.752458 11815 raft_consensus.cc:504] T d9e454f82a4e4c429b58f32926319b7c P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.753295 11709 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 195) took 1893 ms. Trace:
I20260925 09:57:24.753595 11709 rpcz_store.cc:276] 0925 09:57:22.860209 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:22.892929 (+ 32720us) service_pool.cc:227] Handling call
0925 09:57:24.747982 (+1855053us) ts_tablet_manager.cc:1145] Deleting tablet 752105c1514f4c1db2a11b95b413b4f6
0925 09:57:24.748020 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:24.753237 (+ 5217us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1840472}
I20260925 09:57:24.754225 11815 leader_election.cc:288] T d9e454f82a4e4c429b58f32926319b7c P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
W20260925 09:57:24.755650 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 89a143a5623e4ef38027381a56c95227 because the tablet was not found. No further retry: Not found: Tablet not found: 89a143a5623e4ef38027381a56c95227
I20260925 09:57:24.757969 10981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9e454f82a4e4c429b58f32926319b7c" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:24.758829 10981 raft_consensus.cc:2454] T d9e454f82a4e4c429b58f32926319b7c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:24.760980 10619 leader_election.cc:302] T d9e454f82a4e4c429b58f32926319b7c P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:24.764364 11709 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 199) took 1877 ms. Trace:
I20260925 09:57:24.764636 11835 raft_consensus.cc:2781] T d9e454f82a4e4c429b58f32926319b7c P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:24.764981 11835 raft_consensus.cc:482] T d9e454f82a4e4c429b58f32926319b7c P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:24.764685 11709 rpcz_store.cc:276] 0925 09:57:22.886469 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:22.886569 (+ 100us) service_pool.cc:227] Handling call
0925 09:57:24.764128 (+1877559us) ts_tablet_manager.cc:1145] Deleting tablet 41df75fe4b8e4856bdbb5ead3cf4fd24
0925 09:57:24.764168 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:24.764317 (+ 149us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1849615}
I20260925 09:57:24.765290 11835 raft_consensus.cc:3037] T d9e454f82a4e4c429b58f32926319b7c P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:24.767477 11082 tablet_bootstrap.cc:492] T 9e697e7146e34841bd64660b45ebf231 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:24.729750 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ff4cf556eee494d8cd590884466f09f" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:24.729787 10834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e3120347c8e4a3eab57d6562ae40e8d" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:24.729900 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dba14a44a9de492886d629441606767e" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:24.767340 11780 raft_consensus.cc:482] T 3d652a76d96f4dbeafde15f3c16534a6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:24.770267 11780 raft_consensus.cc:504] T 3d652a76d96f4dbeafde15f3c16534a6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:24.771751 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 752105c1514f4c1db2a11b95b413b4f6 because the tablet was not found. No further retry: Not found: Tablet not found: 752105c1514f4c1db2a11b95b413b4f6
I20260925 09:57:24.772284 11780 leader_election.cc:288] T 3d652a76d96f4dbeafde15f3c16534a6 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
W20260925 09:57:24.772396 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 41df75fe4b8e4856bdbb5ead3cf4fd24 because the tablet was not found. No further retry: Not found: Tablet not found: 41df75fe4b8e4856bdbb5ead3cf4fd24
I20260925 09:57:24.729684 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb9c51f7e7fd4c029be2a710cb9462eb" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:24.776293 11709 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 198) took 1894 ms. Trace:
I20260925 09:57:24.776211 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9e454f82a4e4c429b58f32926319b7c" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:24.776567 11709 rpcz_store.cc:276] 0925 09:57:22.881621 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:22.884371 (+ 2750us) service_pool.cc:227] Handling call
0925 09:57:24.776073 (+1891702us) ts_tablet_manager.cc:1145] Deleting tablet 2d55154ceb064f22bd6f37c8ae5dce74
0925 09:57:24.776109 (+ 36us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:24.776235 (+ 126us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1843376}
I20260925 09:57:24.732652 11085 tablet_bootstrap.cc:492] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
W20260925 09:57:24.777563 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 2d55154ceb064f22bd6f37c8ae5dce74 because the tablet was not found. No further retry: Not found: Tablet not found: 2d55154ceb064f22bd6f37c8ae5dce74
I20260925 09:57:24.778537 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d652a76d96f4dbeafde15f3c16534a6" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:24.779424 11901 raft_consensus.cc:482] T 977a59175e5b43eeb6f23b938afd00a3 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:24.779857 11901 raft_consensus.cc:504] T 977a59175e5b43eeb6f23b938afd00a3 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.780566 10981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d652a76d96f4dbeafde15f3c16534a6" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:24.781716 11901 leader_election.cc:288] T 977a59175e5b43eeb6f23b938afd00a3 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
W20260925 09:57:24.782617 10621 leader_election.cc:341] T d9e454f82a4e4c429b58f32926319b7c P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:24.782992 10772 leader_election.cc:302] T 3d652a76d96f4dbeafde15f3c16534a6 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:24.783437 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "977a59175e5b43eeb6f23b938afd00a3" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:24.784377 11709 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 211) took 1326 ms. Trace:
W20260925 09:57:24.784602 10919 leader_election.cc:341] T 977a59175e5b43eeb6f23b938afd00a3 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:24.785216 11780 raft_consensus.cc:2726] T 3d652a76d96f4dbeafde15f3c16534a6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:24.787725 11709 rpcz_store.cc:276] 0925 09:57:23.459991 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:23.466719 (+ 6728us) service_pool.cc:227] Handling call
0925 09:57:24.784074 (+1317355us) ts_tablet_manager.cc:1145] Deleting tablet 7a31b999f0fd447eacff13de71aa3fff
0925 09:57:24.784113 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:24.784332 (+ 219us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1316709}
W20260925 09:57:24.788897 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 7a31b999f0fd447eacff13de71aa3fff because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7a31b999f0fd447eacff13de71aa3fff already in progress: creating tablet
I20260925 09:57:24.791288 11085 tablet_bootstrap.cc:654] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:24.794028 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "977a59175e5b43eeb6f23b938afd00a3" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:24.796674 11082 tablet_bootstrap.cc:654] T 9e697e7146e34841bd64660b45ebf231 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:24.796525 10918 leader_election.cc:341] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:24.796988 11727 ts_tablet_manager.cc:1918] T 545ac025f2cb469da847979011a0b3ce P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:24.797350 11727 log.cc:1196] T 545ac025f2cb469da847979011a0b3ce P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/545ac025f2cb469da847979011a0b3ce
I20260925 09:57:24.797176 10918 leader_election.cc:302] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:24.797956 11727 ts_tablet_manager.cc:1939] T 545ac025f2cb469da847979011a0b3ce P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:24.798334 11901 raft_consensus.cc:2726] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:24.798602 10918 leader_election.cc:341] T 4e3120347c8e4a3eab57d6562ae40e8d P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:24.799060 10918 leader_election.cc:302] T 4e3120347c8e4a3eab57d6562ae40e8d P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:24.798974 11084 raft_consensus.cc:348] T 8aff53cfd4994c629a408ee8250280c5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.799748 11901 raft_consensus.cc:2726] T 4e3120347c8e4a3eab57d6562ae40e8d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:24.800206 10918 leader_election.cc:341] T dba14a44a9de492886d629441606767e P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:24.800549 10918 leader_election.cc:302] T dba14a44a9de492886d629441606767e P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:24.801209 11901 raft_consensus.cc:2726] T dba14a44a9de492886d629441606767e P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:24.699364 11579 ts_tablet_manager.cc:1918] T 359090244c4748e8a2d45e3c77e89498 P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.1
I20260925 09:57:24.799750 11084 raft_consensus.cc:374] T 8aff53cfd4994c629a408ee8250280c5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:24.804087 11084 raft_consensus.cc:729] T 8aff53cfd4994c629a408ee8250280c5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:24.803363 11579 log.cc:1196] T 359090244c4748e8a2d45e3c77e89498 P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/359090244c4748e8a2d45e3c77e89498
I20260925 09:57:24.806278 11579 ts_tablet_manager.cc:1939] T 359090244c4748e8a2d45e3c77e89498 P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:24.791117 11835 raft_consensus.cc:504] T d9e454f82a4e4c429b58f32926319b7c P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.811426 11709 tablet_replica.cc:330] T 3706253e53d841aeaa43889ead13af32 P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:24.811689 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 359090244c4748e8a2d45e3c77e89498 (table default.Table_3f0e11249b90447595581ed433d20c5b [id=d0cfc580b2c64772b90d93e44f1fdb85]) successfully deleted
I20260925 09:57:24.812363 11709 raft_consensus.cc:2229] T 3706253e53d841aeaa43889ead13af32 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:24.807420 11084 consensus_queue.cc:260] T 8aff53cfd4994c629a408ee8250280c5 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.815551 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9e454f82a4e4c429b58f32926319b7c" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:24.815762 11835 leader_election.cc:288] T d9e454f82a4e4c429b58f32926319b7c P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:24.815915 10981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9e454f82a4e4c429b58f32926319b7c" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:24.816473 10981 raft_consensus.cc:3037] T d9e454f82a4e4c429b58f32926319b7c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:57:24.801514 10918 leader_election.cc:341] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:24.820890 10621 leader_election.cc:341] T d9e454f82a4e4c429b58f32926319b7c P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:24.822399 11709 raft_consensus.cc:2258] T 3706253e53d841aeaa43889ead13af32 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:24.836634 11709 ts_tablet_manager.cc:1905] T 3706253e53d841aeaa43889ead13af32 P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:24.831182 10981 raft_consensus.cc:2454] T d9e454f82a4e4c429b58f32926319b7c P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:24.838171 10387 catalog_manager.cc:5495] T 3d652a76d96f4dbeafde15f3c16534a6 P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:24.839434 10619 leader_election.cc:302] T d9e454f82a4e4c429b58f32926319b7c P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:24.839680 10387 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 3d652a76d96f4dbeafde15f3c16534a6 (table default.Table_be0f658aa41f4188bc16db37e96f4ad1 [id=73083c1560cb4db5a12964387cb49b73]). Expected version 1 got 0
I20260925 09:57:24.840181 11835 raft_consensus.cc:2781] T d9e454f82a4e4c429b58f32926319b7c P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:24.841017 11835 raft_consensus.cc:686] T d9e454f82a4e4c429b58f32926319b7c P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:24.843845 10819 tablet_service.cc:1467] Processing CreateTablet for tablet 9744b9814b69470396ce039348fe5e45 (DEFAULT_TABLE table=default.Table_876195e38a7349cc94043563f026f7f6 [id=cec6a532068b4ce7802c0f4fe5de6ed9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:24.844005 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 180) took 2676 ms. Trace:
I20260925 09:57:24.843506 11835 consensus_queue.cc:237] T d9e454f82a4e4c429b58f32926319b7c P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.844848 10819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9744b9814b69470396ce039348fe5e45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:24.844342 11727 rpcz_store.cc:276] 0925 09:57:22.168030 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:22.168130 (+ 100us) service_pool.cc:227] Handling call
0925 09:57:24.571715 (+2403585us) ts_tablet_manager.cc:1145] Deleting tablet 545ac025f2cb469da847979011a0b3ce
0925 09:57:24.583586 (+ 11871us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:24.665888 (+ 82302us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:24.796769 (+130881us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:24.843948 (+ 47179us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":113792,"tablet-delete.queue_time_us":2388883}
I20260925 09:57:24.845551 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 545ac025f2cb469da847979011a0b3ce (table default.Table_14d12f6871c847daa860c3a0cfee4231 [id=f61297f0d10c4656bd06a752c9c7596b]) successfully deleted
I20260925 09:57:24.850865 11082 tablet_bootstrap.cc:492] T 9e697e7146e34841bd64660b45ebf231 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:24.819635 10918 leader_election.cc:302] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:24.851481 11082 ts_tablet_manager.cc:1397] T 9e697e7146e34841bd64660b45ebf231 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.100s user 0.010s sys 0.000s
I20260925 09:57:24.851827 11901 raft_consensus.cc:2726] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:24.852718 10918 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 51224.6us
W20260925 09:57:24.853358 10918 leader_election.cc:341] T 977a59175e5b43eeb6f23b938afd00a3 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:24.853860 10918 leader_election.cc:302] T 977a59175e5b43eeb6f23b938afd00a3 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:24.853847 11082 raft_consensus.cc:348] T 9e697e7146e34841bd64660b45ebf231 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.854552 11082 raft_consensus.cc:374] T 9e697e7146e34841bd64660b45ebf231 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:24.854521 11901 raft_consensus.cc:2726] T 977a59175e5b43eeb6f23b938afd00a3 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:24.854812 11082 raft_consensus.cc:729] T 9e697e7146e34841bd64660b45ebf231 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:24.871143 10390 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 3d652a76d96f4dbeafde15f3c16534a6 (table default.Table_be0f658aa41f4188bc16db37e96f4ad1 [id=73083c1560cb4db5a12964387cb49b73]). Expected version 1 got 0
I20260925 09:57:24.875725 11082 consensus_queue.cc:260] T 9e697e7146e34841bd64660b45ebf231 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.878613 11082 ts_tablet_manager.cc:1428] T 9e697e7146e34841bd64660b45ebf231 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.027s user 0.005s sys 0.000s
I20260925 09:57:24.879423 11082 tablet_bootstrap.cc:492] T aba253f22afa4200926f31a1b43e3970 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:24.881412 11085 tablet_bootstrap.cc:492] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:24.881840 11085 ts_tablet_manager.cc:1397] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.149s user 0.010s sys 0.000s
I20260925 09:57:24.882288 10663 tablet_service.cc:1467] Processing CreateTablet for tablet 9744b9814b69470396ce039348fe5e45 (DEFAULT_TABLE table=default.Table_876195e38a7349cc94043563f026f7f6 [id=cec6a532068b4ce7802c0f4fe5de6ed9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:24.883322 10663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9744b9814b69470396ce039348fe5e45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:24.884061 11085 raft_consensus.cc:348] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.884692 11085 raft_consensus.cc:374] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:24.884979 11085 raft_consensus.cc:729] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:24.885524 11085 consensus_queue.cc:260] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.886678 10388 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet e57065f2bb224a56ad1c3518a0c375da (table default.Table_6eff7b87b3b74ca4bab719140667a87b [id=7889081642604700bbc6d253ab105260]). Expected version 2 got 1
I20260925 09:57:24.890058 10388 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 26fae3fbfe374f348a1729c21ec751b9 (table default.Table_0011387b45014de4b24b466f4cb907f7 [id=e3f7aa5e988c4b6c98be5cd3d465245d]). Expected version 1 got 0
I20260925 09:57:24.894455 11084 ts_tablet_manager.cc:1428] T 8aff53cfd4994c629a408ee8250280c5 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.148s user 0.005s sys 0.002s
I20260925 09:57:24.900029 10956 tablet_service.cc:1467] Processing CreateTablet for tablet 9744b9814b69470396ce039348fe5e45 (DEFAULT_TABLE table=default.Table_876195e38a7349cc94043563f026f7f6 [id=cec6a532068b4ce7802c0f4fe5de6ed9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:24.901422 10956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9744b9814b69470396ce039348fe5e45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:24.902151 11727 tablet_replica.cc:330] T d9e454f82a4e4c429b58f32926319b7c P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:24.902495 10819 tablet_service.cc:1467] Processing CreateTablet for tablet 69028aa7908c45b896da4ba8deef07b1 (DEFAULT_TABLE table=default.Table_fad50b13f9024bf8b4df345241f8c32b [id=351cf96e6f86425cb5077c576bc006db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:24.902944 11727 raft_consensus.cc:2229] T d9e454f82a4e4c429b58f32926319b7c P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:24.903366 11727 raft_consensus.cc:2258] T d9e454f82a4e4c429b58f32926319b7c P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:24.903518 10819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69028aa7908c45b896da4ba8deef07b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:24.906874 10655 tablet_service.cc:1467] Processing CreateTablet for tablet 69028aa7908c45b896da4ba8deef07b1 (DEFAULT_TABLE table=default.Table_fad50b13f9024bf8b4df345241f8c32b [id=351cf96e6f86425cb5077c576bc006db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:24.908041 10655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69028aa7908c45b896da4ba8deef07b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:24.909003 11085 ts_tablet_manager.cc:1428] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.027s user 0.005s sys 0.000s
I20260925 09:57:24.909441 10388 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet b555eeba505740988c704df5c734e7c0 (table default.Table_48f368d811494119be0dc534e2fdcfbd [id=3e32026d175b4b92a3ee036f17e54b9d]). Expected version 1 got 0
I20260925 09:57:24.922081 11085 tablet_bootstrap.cc:492] T 55d14d4d8abe4ec9ac9aeef0d027e804 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:24.931602 11780 raft_consensus.cc:482] T 7360a869f3be45a884a0379f4ef78e85 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:24.932051 11780 raft_consensus.cc:504] T 7360a869f3be45a884a0379f4ef78e85 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.933882 11780 leader_election.cc:288] T 7360a869f3be45a884a0379f4ef78e85 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:24.934439 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7360a869f3be45a884a0379f4ef78e85" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:24.935195 11901 raft_consensus.cc:482] T a5d872c3620646e5ac33ceb95de2cbd4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:24.935638 11901 raft_consensus.cc:504] T a5d872c3620646e5ac33ceb95de2cbd4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.935890 10981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7360a869f3be45a884a0379f4ef78e85" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:24.936609 10981 raft_consensus.cc:2379] T 7360a869f3be45a884a0379f4ef78e85 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c43461c4ee854ae1ac52b873ccada318 in current term 1: Already voted for candidate b8850818e2ee42a082802d0095c08d21 in this term.
I20260925 09:57:24.937474 11901 leader_election.cc:288] T a5d872c3620646e5ac33ceb95de2cbd4 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:24.938719 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5d872c3620646e5ac33ceb95de2cbd4" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:24.939975 10919 leader_election.cc:341] T a5d872c3620646e5ac33ceb95de2cbd4 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:24.940361 11084 tablet_bootstrap.cc:492] T 6540aba5854a44419addc5d454e473b7 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:24.944929 10772 leader_election.cc:302] T 7360a869f3be45a884a0379f4ef78e85 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:24.945947 11889 raft_consensus.cc:3037] T 7360a869f3be45a884a0379f4ef78e85 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:24.945909 11901 raft_consensus.cc:482] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:24.946334 11901 raft_consensus.cc:504] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.948807 11901 leader_election.cc:288] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:24.948688 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7cd1d6a603642afa2c8c737079961b5" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:24.949854 10919 leader_election.cc:341] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:24.950524 10957 tablet_service.cc:1467] Processing CreateTablet for tablet 69028aa7908c45b896da4ba8deef07b1 (DEFAULT_TABLE table=default.Table_fad50b13f9024bf8b4df345241f8c32b [id=351cf96e6f86425cb5077c576bc006db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:24.949999 11709 ts_tablet_manager.cc:1918] T 3706253e53d841aeaa43889ead13af32 P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:24.951649 10957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69028aa7908c45b896da4ba8deef07b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:24.951804 11709 log.cc:1196] T 3706253e53d841aeaa43889ead13af32 P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/3706253e53d841aeaa43889ead13af32
I20260925 09:57:24.952275 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5d872c3620646e5ac33ceb95de2cbd4" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:24.952744 10812 tablet_service.cc:1467] Processing CreateTablet for tablet 9a73a1f242e7430cbf52229c07b4dc82 (DEFAULT_TABLE table=default.Table_c720bf4c23b44a94b73ef67ad45560ce [id=8256a2288a304db39502a0369c9ab58e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:24.954574 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7cd1d6a603642afa2c8c737079961b5" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:24.956149 10918 leader_election.cc:341] T a5d872c3620646e5ac33ceb95de2cbd4 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:24.956511 10918 leader_election.cc:302] T a5d872c3620646e5ac33ceb95de2cbd4 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
W20260925 09:57:24.957415 10918 leader_election.cc:341] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:24.957747 10918 leader_election.cc:302] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:24.957649 10657 tablet_service.cc:1467] Processing CreateTablet for tablet 9a73a1f242e7430cbf52229c07b4dc82 (DEFAULT_TABLE table=default.Table_c720bf4c23b44a94b73ef67ad45560ce [id=8256a2288a304db39502a0369c9ab58e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:24.958730 10657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a73a1f242e7430cbf52229c07b4dc82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:24.959334 10811 tablet_service.cc:1467] Processing CreateTablet for tablet 1e76f9e0ba554e1cad0cf18863393608 (DEFAULT_TABLE table=default.Table_a2a96456c8804d1692ffbafab832340b [id=5dc3fd1666fb4431b1c3bbf27b02ba54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:24.959616 10961 tablet_service.cc:1467] Processing CreateTablet for tablet 9a73a1f242e7430cbf52229c07b4dc82 (DEFAULT_TABLE table=default.Table_c720bf4c23b44a94b73ef67ad45560ce [id=8256a2288a304db39502a0369c9ab58e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:24.960538 10961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a73a1f242e7430cbf52229c07b4dc82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:24.960798 11892 raft_consensus.cc:2726] T a5d872c3620646e5ac33ceb95de2cbd4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:24.961331 11892 raft_consensus.cc:2726] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:24.961756 10667 tablet_service.cc:1467] Processing CreateTablet for tablet 1e76f9e0ba554e1cad0cf18863393608 (DEFAULT_TABLE table=default.Table_a2a96456c8804d1692ffbafab832340b [id=5dc3fd1666fb4431b1c3bbf27b02ba54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:24.962628 10667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e76f9e0ba554e1cad0cf18863393608. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:24.969234 11889 raft_consensus.cc:2726] T 7360a869f3be45a884a0379f4ef78e85 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:24.976560 10812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a73a1f242e7430cbf52229c07b4dc82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:24.979858 10811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e76f9e0ba554e1cad0cf18863393608. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:24.986996 11085 tablet_bootstrap.cc:654] T 55d14d4d8abe4ec9ac9aeef0d027e804 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:24.988164 11835 raft_consensus.cc:482] T 9e697e7146e34841bd64660b45ebf231 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:24.988581 11835 raft_consensus.cc:504] T 9e697e7146e34841bd64660b45ebf231 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:24.990326 11835 leader_election.cc:288] T 9e697e7146e34841bd64660b45ebf231 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:24.990954 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e697e7146e34841bd64660b45ebf231" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:24.991262 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e697e7146e34841bd64660b45ebf231" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:24.991839 10976 raft_consensus.cc:2454] T 9e697e7146e34841bd64660b45ebf231 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
W20260925 09:57:24.992177 10621 leader_election.cc:341] T 9e697e7146e34841bd64660b45ebf231 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:24.992888 10619 leader_election.cc:302] T 9e697e7146e34841bd64660b45ebf231 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:24.993875 11835 raft_consensus.cc:2781] T 9e697e7146e34841bd64660b45ebf231 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:24.994205 11835 raft_consensus.cc:482] T 9e697e7146e34841bd64660b45ebf231 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:24.994652 11835 raft_consensus.cc:3037] T 9e697e7146e34841bd64660b45ebf231 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:24.999848 11727 ts_tablet_manager.cc:1905] T d9e454f82a4e4c429b58f32926319b7c P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:25.001957 11835 raft_consensus.cc:504] T 9e697e7146e34841bd64660b45ebf231 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:25.004323 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e697e7146e34841bd64660b45ebf231" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
W20260925 09:57:25.005980 10621 leader_election.cc:341] T 9e697e7146e34841bd64660b45ebf231 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:25.012627 10806 tablet_service.cc:1467] Processing CreateTablet for tablet a81530200cf14717bc7cbf67bb73f4e5 (DEFAULT_TABLE table=default.Table_4f4ab1f0170a42e2826f1e3143803a6b [id=86ea83e3afc04657ad2aff23057b36d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:25.013695 10806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a81530200cf14717bc7cbf67bb73f4e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:25.014436 10663 tablet_service.cc:1467] Processing CreateTablet for tablet a81530200cf14717bc7cbf67bb73f4e5 (DEFAULT_TABLE table=default.Table_4f4ab1f0170a42e2826f1e3143803a6b [id=86ea83e3afc04657ad2aff23057b36d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:25.015457 10663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a81530200cf14717bc7cbf67bb73f4e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:25.020596 11780 raft_consensus.cc:482] T ec1b78e61e8d46cb8b87f60786059658 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:25.021019 11780 raft_consensus.cc:504] T ec1b78e61e8d46cb8b87f60786059658 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:25.023204 11780 leader_election.cc:288] T ec1b78e61e8d46cb8b87f60786059658 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:25.023600 11709 ts_tablet_manager.cc:1939] T 3706253e53d841aeaa43889ead13af32 P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
W20260925 09:57:25.025342 10775 leader_election.cc:341] T ec1b78e61e8d46cb8b87f60786059658 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Not found: Tablet not found: ec1b78e61e8d46cb8b87f60786059658
I20260925 09:57:25.026047 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 3706253e53d841aeaa43889ead13af32 (table default.Table_bbc3c50c06634d119ae96dcf649f44e7 [id=0260f0e6cdd6431db06788bde54ec38c]) successfully deleted
I20260925 09:57:25.027073 11082 tablet_bootstrap.cc:654] T aba253f22afa4200926f31a1b43e3970 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:25.030349 10807 tablet_service.cc:1467] Processing CreateTablet for tablet 05e2c7f9820f41679c4d58343ff3f829 (DEFAULT_TABLE table=default.Table_4066ae2ab4534ff58ac51f74bef05db9 [id=653cef93edf64ea5aad4122799ed9082]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:25.031347 10807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05e2c7f9820f41679c4d58343ff3f829. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:25.032222 10661 tablet_service.cc:1467] Processing CreateTablet for tablet 05e2c7f9820f41679c4d58343ff3f829 (DEFAULT_TABLE table=default.Table_4066ae2ab4534ff58ac51f74bef05db9 [id=653cef93edf64ea5aad4122799ed9082]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:25.033700 10661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05e2c7f9820f41679c4d58343ff3f829. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:25.034744 10816 tablet_service.cc:1467] Processing CreateTablet for tablet 3039ec3febf848dba7c4e6fd38bb8053 (DEFAULT_TABLE table=default.Table_908fa06a45334bf7a6700063843028b1 [id=5032e01ba3404dec8925a55ea1afed26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:25.035660 10816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3039ec3febf848dba7c4e6fd38bb8053. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:25.036059 10655 tablet_service.cc:1467] Processing CreateTablet for tablet 3039ec3febf848dba7c4e6fd38bb8053 (DEFAULT_TABLE table=default.Table_908fa06a45334bf7a6700063843028b1 [id=5032e01ba3404dec8925a55ea1afed26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:25.037066 10655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3039ec3febf848dba7c4e6fd38bb8053. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:25.061697 10958 tablet_service.cc:1467] Processing CreateTablet for tablet a81530200cf14717bc7cbf67bb73f4e5 (DEFAULT_TABLE table=default.Table_4f4ab1f0170a42e2826f1e3143803a6b [id=86ea83e3afc04657ad2aff23057b36d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:25.061892 10955 tablet_service.cc:1467] Processing CreateTablet for tablet 1e76f9e0ba554e1cad0cf18863393608 (DEFAULT_TABLE table=default.Table_a2a96456c8804d1692ffbafab832340b [id=5dc3fd1666fb4431b1c3bbf27b02ba54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:25.064011 10963 tablet_service.cc:1467] Processing CreateTablet for tablet 05e2c7f9820f41679c4d58343ff3f829 (DEFAULT_TABLE table=default.Table_4066ae2ab4534ff58ac51f74bef05db9 [id=653cef93edf64ea5aad4122799ed9082]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 09:57:25.082365 10772 leader_election.cc:341] T ec1b78e61e8d46cb8b87f60786059658 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: ec1b78e61e8d46cb8b87f60786059658
I20260925 09:57:25.082772 10772 leader_election.cc:302] T ec1b78e61e8d46cb8b87f60786059658 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:25.083427 11889 raft_consensus.cc:2726] T ec1b78e61e8d46cb8b87f60786059658 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:25.085402 10958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a81530200cf14717bc7cbf67bb73f4e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:25.085249 10959 tablet_service.cc:1467] Processing CreateTablet for tablet 3039ec3febf848dba7c4e6fd38bb8053 (DEFAULT_TABLE table=default.Table_908fa06a45334bf7a6700063843028b1 [id=5032e01ba3404dec8925a55ea1afed26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:25.088411 11835 leader_election.cc:288] T 9e697e7146e34841bd64660b45ebf231 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:25.093240 10955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e76f9e0ba554e1cad0cf18863393608. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:25.101858 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e697e7146e34841bd64660b45ebf231" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:25.102458 10976 raft_consensus.cc:3037] T 9e697e7146e34841bd64660b45ebf231 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:25.103871 10963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05e2c7f9820f41679c4d58343ff3f829. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:25.104422 10959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3039ec3febf848dba7c4e6fd38bb8053. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:25.106791 10839 raft_consensus.cc:3037] T 3d652a76d96f4dbeafde15f3c16534a6 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:25.115906 10384 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet e57065f2bb224a56ad1c3518a0c375da (table default.Table_6eff7b87b3b74ca4bab719140667a87b [id=7889081642604700bbc6d253ab105260]). Expected version 2 got 1
I20260925 09:57:25.125386 11085 tablet_bootstrap.cc:492] T 55d14d4d8abe4ec9ac9aeef0d027e804 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:25.125839 11085 ts_tablet_manager.cc:1397] T 55d14d4d8abe4ec9ac9aeef0d027e804 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.204s user 0.008s sys 0.000s
I20260925 09:57:25.127175 11709 tablet_replica.cc:330] T 359090244c4748e8a2d45e3c77e89498 P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:25.128120 11709 raft_consensus.cc:2229] T 359090244c4748e8a2d45e3c77e89498 P b8850818e2ee42a082802d0095c08d21 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:25.128592 11709 raft_consensus.cc:2258] T 359090244c4748e8a2d45e3c77e89498 P b8850818e2ee42a082802d0095c08d21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:25.130434 11709 ts_tablet_manager.cc:1905] T 359090244c4748e8a2d45e3c77e89498 P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:25.132771 11082 tablet_bootstrap.cc:492] T aba253f22afa4200926f31a1b43e3970 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:25.133175 11082 ts_tablet_manager.cc:1397] T aba253f22afa4200926f31a1b43e3970 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.254s user 0.009s sys 0.000s
I20260925 09:57:25.135941 11085 raft_consensus.cc:348] T 55d14d4d8abe4ec9ac9aeef0d027e804 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:25.136593 11085 raft_consensus.cc:374] T 55d14d4d8abe4ec9ac9aeef0d027e804 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:25.136842 11085 raft_consensus.cc:729] T 55d14d4d8abe4ec9ac9aeef0d027e804 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:25.137286 11780 raft_consensus.cc:482] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:25.137396 11085 consensus_queue.cc:260] T 55d14d4d8abe4ec9ac9aeef0d027e804 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:25.156275 10384 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 3d652a76d96f4dbeafde15f3c16534a6 (table default.Table_be0f658aa41f4188bc16db37e96f4ad1 [id=73083c1560cb4db5a12964387cb49b73]). Expected version 1 got 0
I20260925 09:57:25.157830 10384 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 26fae3fbfe374f348a1729c21ec751b9 (table default.Table_0011387b45014de4b24b466f4cb907f7 [id=e3f7aa5e988c4b6c98be5cd3d465245d]). Expected version 1 got 0
I20260925 09:57:25.137693 11780 raft_consensus.cc:504] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:25.177500 11780 leader_election.cc:288] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:25.180490 11085 ts_tablet_manager.cc:1428] T 55d14d4d8abe4ec9ac9aeef0d027e804 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.054s user 0.007s sys 0.000s
I20260925 09:57:25.181409 11085 tablet_bootstrap.cc:492] T c2110d4ada2d45c2b07259374465a8df P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:25.182643 10981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a21909e2e5794f57971f4e42795235c7" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:25.183429 10981 raft_consensus.cc:2379] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c43461c4ee854ae1ac52b873ccada318 in current term 1: Already voted for candidate b8850818e2ee42a082802d0095c08d21 in this term.
I20260925 09:57:25.178252 10688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a21909e2e5794f57971f4e42795235c7" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:25.198182 10392 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:35140 (request call id 221) took 2039 ms. Trace:
I20260925 09:57:25.198459 10775 leader_election.cc:302] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:25.199640 11889 raft_consensus.cc:3037] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:25.201615 10391 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:35140 (request call id 220) took 2046 ms. Trace:
I20260925 09:57:25.198484 10392 rpcz_store.cc:276] 0925 09:57:23.158946 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:23.167539 (+ 8593us) service_pool.cc:227] Handling call
0925 09:57:23.178434 (+ 10895us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:57:23.179061 (+ 627us) catalog_manager.cc:3616] Apply alter schema
0925 09:57:23.179668 (+ 607us) catalog_manager.cc:3639] Acquired catalog manager lock
0925 09:57:23.179855 (+ 187us) catalog_manager.cc:3786] Updating metadata on disk
0925 09:57:23.243095 (+ 63240us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:23.243120 (+ 25us) write_op.cc:454] Released schema lock
0925 09:57:23.243321 (+ 201us) catalog_manager.cc:3817] Committing alterations to in-memory state
0925 09:57:25.198003 (+1954682us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:23.180883 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:23.181186 (+ 303us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:23.181208 (+ 22us) write_op.cc:435] Acquired schema lock
0925 09:57:23.181220 (+ 12us) tablet.cc:660] Decoding operations
0925 09:57:23.181461 (+ 241us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:23.181497 (+ 36us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:23.181518 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:23.181640 (+ 122us) tablet.cc:699] Row locks acquired
0925 09:57:23.181653 (+ 13us) write_op.cc:260] PREPARE: finished
0925 09:57:23.181748 (+ 95us) write_op.cc:270] Start()
0925 09:57:23.181816 (+ 68us) write_op.cc:276] Timestamp: P: 1790330243181675 usec, L: 0
0925 09:57:23.181831 (+ 15us) op_driver.cc:331] REPLICATION: starting
0925 09:57:23.182248 (+ 417us) log.cc:842] Serialized 405 byte log entry
0925 09:57:23.188615 (+ 6367us) op_driver.cc:444] REPLICATION: finished
0925 09:57:23.203884 (+ 15269us) write_op.cc:301] APPLY: starting
0925 09:57:23.203938 (+ 54us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:23.204029 (+ 91us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:23.204126 (+ 97us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:23.204320 (+ 194us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:23.204436 (+ 116us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:23.204478 (+ 42us) write_op.cc:312] APPLY: finished
0925 09:57:23.204647 (+ 169us) log.cc:842] Serialized 23 byte log entry
0925 09:57:23.204785 (+ 138us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:23.204847 (+ 62us) write_op.cc:454] Released schema lock
0925 09:57:23.204877 (+ 30us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":15105,"apply.run_cpu_time_us":1182,"apply.run_wall_time_us":1182,"num_ops":1,"prepare.queue_time_us":81,"prepare.run_cpu_time_us":1656,"prepare.run_wall_time_us":3213,"raft.queue_time_us":125,"raft.run_cpu_time_us":545,"raft.run_wall_time_us":542,"replication_time_us":6723}]]}
I20260925 09:57:25.207787 10886 heartbeater.cc:347] Connected to a master server at 127.10.9.61:33047
I20260925 09:57:25.209081 11889 raft_consensus.cc:2726] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:25.135367 11082 raft_consensus.cc:348] T aba253f22afa4200926f31a1b43e3970 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:25.159305 11084 tablet_bootstrap.cc:654] T 6540aba5854a44419addc5d454e473b7 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:25.210880 11082 raft_consensus.cc:374] T aba253f22afa4200926f31a1b43e3970 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:25.216023 11082 raft_consensus.cc:729] T aba253f22afa4200926f31a1b43e3970 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:25.220113 11082 consensus_queue.cc:260] T aba253f22afa4200926f31a1b43e3970 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:25.225083 10389 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 28d60531970a4ab58ce5ef99cb3a96bf
I20260925 09:57:25.225320 10385 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 67ce2e3325214aa6bdc3347e9e8f82b0
I20260925 09:57:25.226400 10385 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:35140 (request call id 226) took 1608 ms. Trace:
I20260925 09:57:25.228116 11830 raft_consensus.cc:482] T 55d14d4d8abe4ec9ac9aeef0d027e804 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:25.231457 10961 tablet_service.cc:1514] Processing DeleteTablet for tablet 28d60531970a4ab58ce5ef99cb3a96bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:24 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:25.232072 10956 tablet_service.cc:1514] Processing DeleteTablet for tablet 67ce2e3325214aa6bdc3347e9e8f82b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:23 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:25.233470 10667 tablet_service.cc:1514] Processing DeleteTablet for tablet 67ce2e3325214aa6bdc3347e9e8f82b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:23 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:25.201895 10391 rpcz_store.cc:276] 0925 09:57:23.155027 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:23.155148 (+ 121us) service_pool.cc:227] Handling call
0925 09:57:23.156394 (+ 1246us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:57:23.157279 (+ 885us) catalog_manager.cc:3616] Apply alter schema
0925 09:57:23.159041 (+ 1762us) catalog_manager.cc:3639] Acquired catalog manager lock
0925 09:57:23.159230 (+ 189us) catalog_manager.cc:3786] Updating metadata on disk
0925 09:57:23.261562 (+102332us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:23.261586 (+ 24us) write_op.cc:454] Released schema lock
0925 09:57:23.263139 (+ 1553us) catalog_manager.cc:3817] Committing alterations to in-memory state
0925 09:57:25.201439 (+1938300us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:23.244819 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:23.245157 (+ 338us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:23.245180 (+ 23us) write_op.cc:435] Acquired schema lock
0925 09:57:23.245192 (+ 12us) tablet.cc:660] Decoding operations
0925 09:57:23.245433 (+ 241us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:23.245472 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:23.245493 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:23.245624 (+ 131us) tablet.cc:699] Row locks acquired
0925 09:57:23.245637 (+ 13us) write_op.cc:260] PREPARE: finished
0925 09:57:23.245733 (+ 96us) write_op.cc:270] Start()
0925 09:57:23.245805 (+ 72us) write_op.cc:276] Timestamp: P: 1790330243245699 usec, L: 0
0925 09:57:23.245820 (+ 15us) op_driver.cc:331] REPLICATION: starting
0925 09:57:23.246247 (+ 427us) log.cc:842] Serialized 405 byte log entry
0925 09:57:23.253552 (+ 7305us) op_driver.cc:444] REPLICATION: finished
0925 09:57:23.254492 (+ 940us) write_op.cc:301] APPLY: starting
0925 09:57:23.254542 (+ 50us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:23.254634 (+ 92us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:23.255527 (+ 893us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:23.255681 (+ 154us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:23.255801 (+ 120us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:23.255996 (+ 195us) write_op.cc:312] APPLY: finished
0925 09:57:23.256173 (+ 177us) log.cc:842] Serialized 23 byte log entry
0925 09:57:23.256305 (+ 132us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:23.256363 (+ 58us) write_op.cc:454] Released schema lock
0925 09:57:23.256392 (+ 29us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":4007,"child_traces":[["op",{"apply.queue_time_us":802,"apply.run_cpu_time_us":1177,"apply.run_wall_time_us":2070,"num_ops":1,"prepare.queue_time_us":19453,"prepare.run_cpu_time_us":1721,"prepare.run_wall_time_us":5554,"raft.queue_time_us":89,"raft.run_cpu_time_us":563,"raft.run_wall_time_us":563,"replication_time_us":7662}]]}
I20260925 09:57:25.235663 10391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_0478cdcfe9d64b66ba0e3fbc638f9a3b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:25.228502 11830 raft_consensus.cc:504] T 55d14d4d8abe4ec9ac9aeef0d027e804 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:25.237092 10391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0478cdcfe9d64b66ba0e3fbc638f9a3b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:25.237437 11830 leader_election.cc:288] T 55d14d4d8abe4ec9ac9aeef0d027e804 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:25.237304 10389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_2b0b0b9f52034c03aa2c4f041b28ec3d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:25.238613 10389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2b0b0b9f52034c03aa2c4f041b28ec3d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:25.226735 10385 rpcz_store.cc:276] 0925 09:57:23.618372 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:23.618496 (+ 124us) service_pool.cc:227] Handling call
0925 09:57:23.619832 (+ 1336us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:57:23.621355 (+ 1523us) catalog_manager.cc:2752] Modifying in-memory table state
0925 09:57:23.621475 (+ 120us) catalog_manager.cc:2761] Locking tablets
0925 09:57:24.199421 (+577946us) catalog_manager.cc:2775] Removing table and tablets from system table
0925 09:57:24.339996 (+140575us) catalog_manager.cc:2793] Removing table from by-name map
0925 09:57:24.340229 (+ 233us) catalog_manager.cc:2809] Committing in-memory state
0925 09:57:25.225048 (+884819us) catalog_manager.cc:2813] Aborting table tasks
0925 09:57:25.226352 (+ 1304us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:24.305032 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:24.305423 (+ 391us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:24.305446 (+ 23us) write_op.cc:435] Acquired schema lock
0925 09:57:24.305461 (+ 15us) tablet.cc:660] Decoding operations
0925 09:57:24.305806 (+ 345us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:24.305844 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:24.305866 (+ 22us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:24.306141 (+ 275us) tablet.cc:699] Row locks acquired
0925 09:57:24.306156 (+ 15us) write_op.cc:260] PREPARE: finished
0925 09:57:24.306274 (+ 118us) write_op.cc:270] Start()
0925 09:57:24.306357 (+ 83us) write_op.cc:276] Timestamp: P: 1790330244306151 usec, L: 0
0925 09:57:24.306374 (+ 17us) op_driver.cc:331] REPLICATION: starting
0925 09:57:24.306892 (+ 518us) log.cc:842] Serialized 722 byte log entry
0925 09:57:24.330850 (+ 23958us) op_driver.cc:444] REPLICATION: finished
0925 09:57:24.333126 (+ 2276us) write_op.cc:301] APPLY: starting
0925 09:57:24.333185 (+ 59us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:24.333327 (+ 142us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:24.333439 (+ 112us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:24.333739 (+ 300us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:24.333917 (+ 178us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 09:57:24.333989 (+ 72us) write_op.cc:312] APPLY: finished
0925 09:57:24.334205 (+ 216us) log.cc:842] Serialized 29 byte log entry
0925 09:57:24.334351 (+ 146us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:24.334425 (+ 74us) write_op.cc:454] Released schema lock
0925 09:57:24.334469 (+ 44us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:24.340372 (+ 5903us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:24.340398 (+ 26us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2119,"apply.run_cpu_time_us":1796,"apply.run_wall_time_us":7465,"mutex_wait_us":3314,"num_ops":2,"prepare.queue_time_us":103801,"prepare.run_cpu_time_us":2099,"prepare.run_wall_time_us":13764,"replication_time_us":24386}]]}
I20260925 09:57:25.246096 10385 master_service.cc:946] ReplaceTablet: received request to replace tablet 3be0eccc65564bfba3d6e5c398c61fb6 from {username='slave'} at 127.0.0.1:35140
I20260925 09:57:25.248030 10981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55d14d4d8abe4ec9ac9aeef0d027e804" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:25.252494 10981 raft_consensus.cc:2454] T 55d14d4d8abe4ec9ac9aeef0d027e804 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:25.253708 10657 tablet_service.cc:1514] Processing DeleteTablet for tablet 28d60531970a4ab58ce5ef99cb3a96bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:24 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:25.234042 10976 raft_consensus.cc:2454] T 9e697e7146e34841bd64660b45ebf231 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:25.256367 10619 leader_election.cc:302] T 55d14d4d8abe4ec9ac9aeef0d027e804 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:25.257187 10619 leader_election.cc:302] T 9e697e7146e34841bd64660b45ebf231 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:25.259115 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55d14d4d8abe4ec9ac9aeef0d027e804" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:25.259714 10835 raft_consensus.cc:2454] T 55d14d4d8abe4ec9ac9aeef0d027e804 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:25.262414 10390 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: default.Table_48f368d811494119be0dc534e2fdcfbd [id=3e32026d175b4b92a3ee036f17e54b9d] alter complete (version 1)
I20260925 09:57:25.262713 11815 raft_consensus.cc:2781] T 55d14d4d8abe4ec9ac9aeef0d027e804 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:25.263083 11815 raft_consensus.cc:482] T 55d14d4d8abe4ec9ac9aeef0d027e804 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:25.263372 11815 raft_consensus.cc:3037] T 55d14d4d8abe4ec9ac9aeef0d027e804 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:25.268993 11830 raft_consensus.cc:2781] T 9e697e7146e34841bd64660b45ebf231 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:25.269657 11830 raft_consensus.cc:686] T 9e697e7146e34841bd64660b45ebf231 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:25.272163 11830 consensus_queue.cc:237] T 9e697e7146e34841bd64660b45ebf231 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:25.282354 10387 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.1:56563 (request call id 32) took 2138 ms (client timeout 1000 ms). Trace:
I20260925 09:57:25.299196 10812 tablet_service.cc:1514] Processing DeleteTablet for tablet 67ce2e3325214aa6bdc3347e9e8f82b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:23 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:25.300022 11579 tablet_replica.cc:330] T 67ce2e3325214aa6bdc3347e9e8f82b0 P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:25.300628 10811 tablet_service.cc:1514] Processing DeleteTablet for tablet 28d60531970a4ab58ce5ef99cb3a96bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:24 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:25.300781 11579 raft_consensus.cc:2229] T 67ce2e3325214aa6bdc3347e9e8f82b0 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:25.301247 11579 raft_consensus.cc:2258] T 67ce2e3325214aa6bdc3347e9e8f82b0 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:25.303009 11579 ts_tablet_manager.cc:1905] T 67ce2e3325214aa6bdc3347e9e8f82b0 P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:25.308921 10388 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: default.Table_c22088d959e84c3cbabadde4d3445178 [id=cf07ed8756904011ab3dac900ea6705a] alter complete (version 1)
I20260925 09:57:25.314844 11815 raft_consensus.cc:504] T 55d14d4d8abe4ec9ac9aeef0d027e804 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:25.323820 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55d14d4d8abe4ec9ac9aeef0d027e804" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:25.324555 10976 raft_consensus.cc:3037] T 55d14d4d8abe4ec9ac9aeef0d027e804 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:57:25.327984 10621 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 44803.3us
W20260925 09:57:25.283099 10387 rpcz_store.cc:269] 0925 09:57:23.144305 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:23.184781 (+ 40476us) service_pool.cc:227] Handling call
0925 09:57:25.282294 (+2097513us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:24.845711 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:24.846062 (+ 351us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:24.846093 (+ 31us) write_op.cc:435] Acquired schema lock
0925 09:57:24.846111 (+ 18us) tablet.cc:660] Decoding operations
0925 09:57:24.846368 (+ 257us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:24.846407 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:24.846430 (+ 23us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:24.846583 (+ 153us) tablet.cc:699] Row locks acquired
0925 09:57:24.846600 (+ 17us) write_op.cc:260] PREPARE: finished
0925 09:57:24.846718 (+ 118us) write_op.cc:270] Start()
0925 09:57:24.846809 (+ 91us) write_op.cc:276] Timestamp: P: 1790330244846599 usec, L: 0
0925 09:57:24.846825 (+ 16us) op_driver.cc:331] REPLICATION: starting
0925 09:57:24.847271 (+ 446us) log.cc:842] Serialized 524 byte log entry
0925 09:57:24.857023 (+ 9752us) op_driver.cc:444] REPLICATION: finished
0925 09:57:24.857456 (+ 433us) write_op.cc:301] APPLY: starting
0925 09:57:24.857506 (+ 50us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:24.857602 (+ 96us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:24.857709 (+ 107us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:24.857909 (+ 200us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:24.858042 (+ 133us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:24.858099 (+ 57us) write_op.cc:312] APPLY: finished
0925 09:57:24.858263 (+ 164us) log.cc:842] Serialized 23 byte log entry
0925 09:57:24.858409 (+ 146us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:24.858488 (+ 79us) write_op.cc:454] Released schema lock
0925 09:57:24.858519 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:24.858700 (+ 181us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:24.858725 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":13671,"reported_tablets":13,"child_traces":[["op",{"apply.queue_time_us":291,"apply.run_cpu_time_us":1420,"apply.run_wall_time_us":1419,"num_ops":1,"prepare.queue_time_us":4199,"prepare.run_cpu_time_us":1954,"prepare.run_wall_time_us":2972,"raft.queue_time_us":161,"raft.run_cpu_time_us":661,"raft.run_wall_time_us":661,"replication_time_us":10128}]]}
I20260925 09:57:25.333081 10657 tablet_service.cc:1514] Processing DeleteTablet for tablet 689ab2b6cf1640129f6db8b113cf2fc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:20 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:25.335625 10663 tablet_service.cc:1514] Processing DeleteTablet for tablet cf35f39bcd6a41128acb2a4936fcc316 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:16 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:25.335220 10657 tablet_service.cc:1514] Processing DeleteTablet for tablet 3706253e53d841aeaa43889ead13af32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:23 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:25.339056 11082 ts_tablet_manager.cc:1428] T aba253f22afa4200926f31a1b43e3970 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.206s user 0.004s sys 0.002s
I20260925 09:57:25.339613 10807 tablet_service.cc:1514] Processing DeleteTablet for tablet 359090244c4748e8a2d45e3c77e89498 with delete_type TABLET_DATA_DELETED (replaced by tablet b39f9e7102ff4a9c96a2cc5d2676f03d) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:25.340131 11082 tablet_bootstrap.cc:492] T 2d087d3e35814c90a03eccd90eeea86c P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:25.369405 11085 tablet_bootstrap.cc:654] T c2110d4ada2d45c2b07259374465a8df P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:25.370240 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55d14d4d8abe4ec9ac9aeef0d027e804" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:25.370790 10832 raft_consensus.cc:3037] T 55d14d4d8abe4ec9ac9aeef0d027e804 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:25.372231 11084 tablet_bootstrap.cc:492] T 6540aba5854a44419addc5d454e473b7 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:25.372628 11084 ts_tablet_manager.cc:1397] T 6540aba5854a44419addc5d454e473b7 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.477s user 0.011s sys 0.005s
I20260925 09:57:25.376917 10812 tablet_service.cc:1514] Processing DeleteTablet for tablet 4be3f6e81ddf418fabc7b694a2e62f35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:18 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:25.379462 10832 raft_consensus.cc:2454] T 55d14d4d8abe4ec9ac9aeef0d027e804 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:25.380226 10811 tablet_service.cc:1514] Processing DeleteTablet for tablet 04fafb9dcf274806b11b2598e73853e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:19 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:25.379230 10807 tablet_service.cc:1514] Processing DeleteTablet for tablet 689ab2b6cf1640129f6db8b113cf2fc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:20 UTC) from {username='slave'} at 127.0.0.1:33344
W20260925 09:57:25.334082 10390 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.1:56563 (request call id 33) took 1085 ms (client timeout 1000 ms). Trace:
I20260925 09:57:25.387555 11815 leader_election.cc:288] T 55d14d4d8abe4ec9ac9aeef0d027e804 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:25.396313 11922 consensus_queue.cc:1035] T 7360a869f3be45a884a0379f4ef78e85 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
W20260925 09:57:25.390980 10390 rpcz_store.cc:269] 0925 09:57:24.248324 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:24.345758 (+ 97434us) service_pool.cc:227] Handling call
0925 09:57:24.411999 (+ 66241us) spinlock_profiling.cc:243] Waited 61.6 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2ad9c3e 00007fabf0610fb4 00007fabf0610f42 00007fabf0610e71 00007fabf0610dec 00007fabf06100a2 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:25.334021 (+922022us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:25.213830 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:25.214204 (+ 374us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:25.214232 (+ 28us) write_op.cc:435] Acquired schema lock
0925 09:57:25.214243 (+ 11us) tablet.cc:660] Decoding operations
0925 09:57:25.219907 (+ 5664us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:25.219947 (+ 40us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:25.219968 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:25.220101 (+ 133us) tablet.cc:699] Row locks acquired
0925 09:57:25.220113 (+ 12us) write_op.cc:260] PREPARE: finished
0925 09:57:25.220223 (+ 110us) write_op.cc:270] Start()
0925 09:57:25.220300 (+ 77us) write_op.cc:276] Timestamp: P: 1790330245220161 usec, L: 0
0925 09:57:25.220335 (+ 35us) op_driver.cc:331] REPLICATION: starting
0925 09:57:25.220796 (+ 461us) log.cc:842] Serialized 356 byte log entry
0925 09:57:25.255159 (+ 34363us) op_driver.cc:444] REPLICATION: finished
0925 09:57:25.255589 (+ 430us) write_op.cc:301] APPLY: starting
0925 09:57:25.255639 (+ 50us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:25.255727 (+ 88us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:25.255827 (+ 100us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:25.256038 (+ 211us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:25.256154 (+ 116us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:25.256195 (+ 41us) write_op.cc:312] APPLY: finished
0925 09:57:25.256376 (+ 181us) log.cc:842] Serialized 23 byte log entry
0925 09:57:25.256531 (+ 155us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:25.256586 (+ 55us) write_op.cc:454] Released schema lock
0925 09:57:25.256621 (+ 35us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:25.256782 (+ 161us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:25.256804 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":267,"reported_tablets":14,"spinlock_wait_cycles":141628160,"child_traces":[["op",{"apply.queue_time_us":291,"apply.run_cpu_time_us":1374,"apply.run_wall_time_us":1375,"num_ops":1,"prepare.queue_time_us":13597,"prepare.run_cpu_time_us":2037,"prepare.run_wall_time_us":7266,"raft.queue_time_us":5069,"raft.run_cpu_time_us":590,"raft.run_wall_time_us":591,"replication_time_us":34754}]]}
I20260925 09:57:25.414212 10807 tablet_service.cc:1514] Processing DeleteTablet for tablet 89a143a5623e4ef38027381a56c95227 with delete_type TABLET_DATA_DELETED (replaced by tablet dba14a44a9de492886d629441606767e) from {username='slave'} at 127.0.0.1:33344
W20260925 09:57:25.416579 10386 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.2:40761 (request call id 29) took 2593 ms (client timeout 1000 ms). Trace:
W20260925 09:57:25.416951 10386 rpcz_store.cc:269] 0925 09:57:22.822811 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:22.875412 (+ 52601us) service_pool.cc:227] Handling call
0925 09:57:25.416481 (+2541069us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":96480,"reported_tablets":26}
I20260925 09:57:25.408819 10621 leader_election.cc:302] T 55d14d4d8abe4ec9ac9aeef0d027e804 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:25.424975 11727 ts_tablet_manager.cc:1918] T d9e454f82a4e4c429b58f32926319b7c P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:25.425359 11727 log.cc:1196] T d9e454f82a4e4c429b58f32926319b7c P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/d9e454f82a4e4c429b58f32926319b7c
I20260925 09:57:25.426077 11727 ts_tablet_manager.cc:1939] T d9e454f82a4e4c429b58f32926319b7c P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:25.375000 11084 raft_consensus.cc:348] T 6540aba5854a44419addc5d454e473b7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:25.431779 11084 raft_consensus.cc:374] T 6540aba5854a44419addc5d454e473b7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:25.432058 11084 raft_consensus.cc:729] T 6540aba5854a44419addc5d454e473b7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:25.432785 11084 consensus_queue.cc:260] T 6540aba5854a44419addc5d454e473b7 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:25.433769 11579 ts_tablet_manager.cc:1918] T 67ce2e3325214aa6bdc3347e9e8f82b0 P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:25.434206 11579 log.cc:1196] T 67ce2e3325214aa6bdc3347e9e8f82b0 P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/67ce2e3325214aa6bdc3347e9e8f82b0
I20260925 09:57:25.435019 11579 ts_tablet_manager.cc:1939] T 67ce2e3325214aa6bdc3347e9e8f82b0 P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:25.437203 11084 ts_tablet_manager.cc:1428] T 6540aba5854a44419addc5d454e473b7 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.064s user 0.006s sys 0.000s
I20260925 09:57:25.438623 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 67ce2e3325214aa6bdc3347e9e8f82b0 (table default.Table_5157be3699be4fa4ac153a69f6010ecc [id=9cd1024c9d784df9af5f8239e39d1be7]) successfully deleted
I20260925 09:57:25.439893 10976 raft_consensus.cc:2454] T 55d14d4d8abe4ec9ac9aeef0d027e804 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:25.444501 11085 tablet_bootstrap.cc:492] T c2110d4ada2d45c2b07259374465a8df P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:25.444972 11085 ts_tablet_manager.cc:1397] T c2110d4ada2d45c2b07259374465a8df P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.264s user 0.007s sys 0.005s
I20260925 09:57:25.445649 11926 raft_consensus.cc:2781] T 55d14d4d8abe4ec9ac9aeef0d027e804 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:25.446164 11926 raft_consensus.cc:686] T 55d14d4d8abe4ec9ac9aeef0d027e804 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:25.447186 11926 consensus_queue.cc:237] T 55d14d4d8abe4ec9ac9aeef0d027e804 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:25.470084 10388 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.2:40761 (request call id 30) took 1497 ms (client timeout 1000 ms). Trace:
I20260925 09:57:25.470892 11901 raft_consensus.cc:482] T 37bab60afe7a4c758a08ef7c9c24aba0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:25.471665 10807 tablet_service.cc:1514] Processing DeleteTablet for tablet cf35f39bcd6a41128acb2a4936fcc316 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:16 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:25.471505 11901 raft_consensus.cc:504] T 37bab60afe7a4c758a08ef7c9c24aba0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:25.472337 10811 tablet_service.cc:1514] Processing DeleteTablet for tablet 67ce2e3325214aa6bdc3347e9e8f82b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:23 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:25.473291 11901 leader_election.cc:288] T 37bab60afe7a4c758a08ef7c9c24aba0 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:25.476430 10834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37bab60afe7a4c758a08ef7c9c24aba0" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:25.477068 11919 consensus_queue.cc:1035] T a21909e2e5794f57971f4e42795235c7 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 09:57:25.477860 10918 leader_election.cc:341] T 37bab60afe7a4c758a08ef7c9c24aba0 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:25.480540 10981 raft_consensus.cc:1261] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0 [term 2 FOLLOWER]: Refusing update from remote peer b8850818e2ee42a082802d0095c08d21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260925 09:57:25.481794 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 181) took 3119 ms. Trace:
I20260925 09:57:25.482120 11727 rpcz_store.cc:276] 0925 09:57:22.362157 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:22.362660 (+ 503us) service_pool.cc:227] Handling call
0925 09:57:24.901994 (+2539334us) ts_tablet_manager.cc:1145] Deleting tablet d9e454f82a4e4c429b58f32926319b7c
0925 09:57:24.902053 (+ 59us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:25.257913 (+355860us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:25.424744 (+166831us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:25.481735 (+ 56991us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2538825}
I20260925 09:57:25.483114 11727 tablet_replica.cc:330] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:25.485031 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet d9e454f82a4e4c429b58f32926319b7c (table default.Table_95a4c3e1f6014e63811447ba4a0e34cf [id=626e547271e6456e81ccaa3b1e59bd97]) successfully deleted
I20260925 09:57:25.488724 11921 consensus_queue.cc:1035] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:25.490051 11084 tablet_bootstrap.cc:492] T 10657a81058241fc988d37e5ea7ff8cf P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:25.492559 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37bab60afe7a4c758a08ef7c9c24aba0" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:25.495415 10919 leader_election.cc:341] T 37bab60afe7a4c758a08ef7c9c24aba0 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:25.495806 10919 leader_election.cc:302] T 37bab60afe7a4c758a08ef7c9c24aba0 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:25.497308 11892 raft_consensus.cc:2726] T 37bab60afe7a4c758a08ef7c9c24aba0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:25.496824 11921 consensus_queue.cc:1035] T a21909e2e5794f57971f4e42795235c7 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:25.500118 10386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_ec0c08cc69f14fe5b33b93b0ca637da2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:25.501614 10386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ec0c08cc69f14fe5b33b93b0ca637da2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:25.506585 10390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_1c14b5a0f00f43c38e3c5e0abea90e0b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:25.521173 10385 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 3be0eccc65564bfba3d6e5c398c61fb6
I20260925 09:57:25.523838 10389 master_service.cc:946] ReplaceTablet: received request to replace tablet 4e3120347c8e4a3eab57d6562ae40e8d from {username='slave'} at 127.0.0.1:35140
W20260925 09:57:25.470456 10388 rpcz_store.cc:269] 0925 09:57:23.973311 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:24.050554 (+ 77243us) service_pool.cc:227] Handling call
0925 09:57:25.470014 (+1419460us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:25.238328 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:25.238647 (+ 319us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:25.238669 (+ 22us) write_op.cc:435] Acquired schema lock
0925 09:57:25.238682 (+ 13us) tablet.cc:660] Decoding operations
0925 09:57:25.238910 (+ 228us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:25.238943 (+ 33us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:25.238963 (+ 20us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:25.239088 (+ 125us) tablet.cc:699] Row locks acquired
0925 09:57:25.239104 (+ 16us) write_op.cc:260] PREPARE: finished
0925 09:57:25.239205 (+ 101us) write_op.cc:270] Start()
0925 09:57:25.239281 (+ 76us) write_op.cc:276] Timestamp: P: 1790330245239143 usec, L: 0
0925 09:57:25.239298 (+ 17us) op_driver.cc:331] REPLICATION: starting
0925 09:57:25.239842 (+ 544us) log.cc:842] Serialized 356 byte log entry
0925 09:57:25.291604 (+ 51762us) op_driver.cc:444] REPLICATION: finished
0925 09:57:25.291982 (+ 378us) write_op.cc:301] APPLY: starting
0925 09:57:25.292038 (+ 56us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:25.292131 (+ 93us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:25.292242 (+ 111us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:25.292439 (+ 197us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:25.292567 (+ 128us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:25.292612 (+ 45us) write_op.cc:312] APPLY: finished
0925 09:57:25.292780 (+ 168us) log.cc:842] Serialized 23 byte log entry
0925 09:57:25.292920 (+ 140us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:25.292984 (+ 64us) write_op.cc:454] Released schema lock
0925 09:57:25.293016 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:25.293180 (+ 164us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:25.293208 (+ 28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":111486,"reported_tablets":27,"child_traces":[["op",{"apply.queue_time_us":241,"apply.run_cpu_time_us":1387,"apply.run_wall_time_us":1388,"num_ops":1,"prepare.queue_time_us":101,"prepare.run_cpu_time_us":1767,"prepare.run_wall_time_us":1906,"raft.queue_time_us":81,"raft.run_cpu_time_us":14,"raft.run_wall_time_us":14,"replication_time_us":52235}]]}
I20260925 09:57:25.526621 11085 raft_consensus.cc:348] T c2110d4ada2d45c2b07259374465a8df P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:25.527339 11085 raft_consensus.cc:374] T c2110d4ada2d45c2b07259374465a8df P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:25.527287 11901 raft_consensus.cc:482] T 5246648245e04e26be2abf1dbf706688 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:25.527623 11085 raft_consensus.cc:729] T c2110d4ada2d45c2b07259374465a8df P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:25.527813 11901 raft_consensus.cc:504] T 5246648245e04e26be2abf1dbf706688 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:25.528190 11085 consensus_queue.cc:260] T c2110d4ada2d45c2b07259374465a8df P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:25.531136 11901 leader_election.cc:288] T 5246648245e04e26be2abf1dbf706688 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:25.532441 10388 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_256aab1fb8b742d6b04709d930f59bce" } new_table_name: "default.Table_44ddd064d56640a0a0e3dd8fd38344fd" modify_external_catalogs: true
I20260925 09:57:25.533321 11892 raft_consensus.cc:482] T 8aff53cfd4994c629a408ee8250280c5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:25.533627 10834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5246648245e04e26be2abf1dbf706688" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:25.533825 11892 raft_consensus.cc:504] T 8aff53cfd4994c629a408ee8250280c5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:25.535562 10918 leader_election.cc:341] T 5246648245e04e26be2abf1dbf706688 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:25.535997 11892 leader_election.cc:288] T 8aff53cfd4994c629a408ee8250280c5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:25.536540 11892 raft_consensus.cc:482] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:25.536991 11579 tablet_replica.cc:330] T 28d60531970a4ab58ce5ef99cb3a96bf P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:25.536984 11892 raft_consensus.cc:504] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:25.537881 11579 raft_consensus.cc:2229] T 28d60531970a4ab58ce5ef99cb3a96bf P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:25.543381 11892 leader_election.cc:288] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:25.552938 10963 tablet_service.cc:1514] Processing DeleteTablet for tablet d9e454f82a4e4c429b58f32926319b7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:22 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:25.554083 10955 tablet_service.cc:1514] Processing DeleteTablet for tablet 60fdc299865446c89d3e5374d573029a with delete_type TABLET_DATA_DELETED (replaced by tablet 61aa6380d30a4b21b70ee2d321247bd8) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:25.557269 10385 catalog_manager.cc:6565] ReplaceTablet: tablet 3be0eccc65564bfba3d6e5c398c61fb6 deleted and replaced by tablet cc295f7a9169475f9a9235e657be629f
I20260925 09:57:25.555044 10959 tablet_service.cc:1514] Processing DeleteTablet for tablet 6161774f61ae48ab9764a05f2188a80b with delete_type TABLET_DATA_DELETED (replaced by tablet 10657a81058241fc988d37e5ea7ff8cf) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:25.558379 10959 tablet_service.cc:1514] Processing DeleteTablet for tablet 3be0eccc65564bfba3d6e5c398c61fb6 with delete_type TABLET_DATA_DELETED (replaced by tablet cc295f7a9169475f9a9235e657be629f) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:25.558913 10811 tablet_service.cc:1514] Processing DeleteTablet for tablet 3be0eccc65564bfba3d6e5c398c61fb6 with delete_type TABLET_DATA_DELETED (replaced by tablet cc295f7a9169475f9a9235e657be629f) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:25.559222 10385 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_64ba123898174e72a6884dbb9ad07de3" } modify_external_catalogs: true
I20260925 09:57:25.559824 10385 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_64ba123898174e72a6884dbb9ad07de3" } modify_external_catalogs: true
I20260925 09:57:25.561758 10834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aff53cfd4994c629a408ee8250280c5" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:25.564664 10688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aff53cfd4994c629a408ee8250280c5" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:25.566561 11085 ts_tablet_manager.cc:1428] T c2110d4ada2d45c2b07259374465a8df P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.121s user 0.007s sys 0.000s
I20260925 09:57:25.567680 11085 tablet_bootstrap.cc:492] T 60fdc299865446c89d3e5374d573029a P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
W20260925 09:57:25.567278 10918 leader_election.cc:341] T 8aff53cfd4994c629a408ee8250280c5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:25.568414 10834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5608c5f9b65b4da7b2fbb56417c92d88" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:25.570616 10918 leader_election.cc:341] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:25.571667 11579 raft_consensus.cc:2258] T 28d60531970a4ab58ce5ef99cb3a96bf P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:25.583477 11579 ts_tablet_manager.cc:1905] T 28d60531970a4ab58ce5ef99cb3a96bf P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 09:57:25.587206 10919 leader_election.cc:341] T 8aff53cfd4994c629a408ee8250280c5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:25.587726 10919 leader_election.cc:302] T 8aff53cfd4994c629a408ee8250280c5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:25.588924 11901 raft_consensus.cc:2726] T 8aff53cfd4994c629a408ee8250280c5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:25.590442 10390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1c14b5a0f00f43c38e3c5e0abea90e0b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:25.592900 10386 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_08049956307d45478571dcca6a46356f" } new_table_name: "default.Table_1b4a040b8d60467a869df4b51c09cf46" modify_external_catalogs: true
I20260925 09:57:25.603127 10959 tablet_service.cc:1514] Processing DeleteTablet for tablet cf35f39bcd6a41128acb2a4936fcc316 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:16 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:25.604192 10959 tablet_service.cc:1514] Processing DeleteTablet for tablet ec1b78e61e8d46cb8b87f60786059658 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:21 UTC) from {username='slave'} at 127.0.0.1:53560
W20260925 09:57:25.604630 10393 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.3:41469 (request call id 22) took 3130 ms (client timeout 1000 ms). Trace:
I20260925 09:57:25.605208 10959 tablet_service.cc:1514] Processing DeleteTablet for tablet 89a143a5623e4ef38027381a56c95227 with delete_type TABLET_DATA_DELETED (replaced by tablet dba14a44a9de492886d629441606767e) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:25.606989 10661 tablet_service.cc:1514] Processing DeleteTablet for tablet 3be0eccc65564bfba3d6e5c398c61fb6 with delete_type TABLET_DATA_DELETED (replaced by tablet cc295f7a9169475f9a9235e657be629f) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:25.615108 10688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5608c5f9b65b4da7b2fbb56417c92d88" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:25.621661 10919 leader_election.cc:341] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:25.551718 11921 consensus_queue.cc:1035] T 7360a869f3be45a884a0379f4ef78e85 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.056s
I20260925 09:57:25.625952 10919 leader_election.cc:302] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
W20260925 09:57:25.627235 11939 log.cc:924] Time spent T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0: Append to log took a long time: real 0.050s user 0.000s sys 0.000s
I20260925 09:57:25.633898 11892 raft_consensus.cc:2726] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:25.637688 11727 raft_consensus.cc:2229] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:25.638844 11727 raft_consensus.cc:2258] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 09:57:25.604966 10393 rpcz_store.cc:269] 0925 09:57:22.474331 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:22.627452 (+153121us) service_pool.cc:227] Handling call
0925 09:57:25.604474 (+2977022us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:24.036822 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:24.037200 (+ 378us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:24.037224 (+ 24us) write_op.cc:435] Acquired schema lock
0925 09:57:24.037249 (+ 25us) tablet.cc:660] Decoding operations
0925 09:57:24.037542 (+ 293us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:24.037583 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:24.037605 (+ 22us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:24.037774 (+ 169us) tablet.cc:699] Row locks acquired
0925 09:57:24.037788 (+ 14us) write_op.cc:260] PREPARE: finished
0925 09:57:24.037908 (+ 120us) write_op.cc:270] Start()
0925 09:57:24.037993 (+ 85us) write_op.cc:276] Timestamp: P: 1790330244037801 usec, L: 0
0925 09:57:24.038012 (+ 19us) op_driver.cc:331] REPLICATION: starting
0925 09:57:24.038664 (+ 652us) log.cc:842] Serialized 524 byte log entry
0925 09:57:24.161582 (+122918us) op_driver.cc:444] REPLICATION: finished
0925 09:57:24.164491 (+ 2909us) write_op.cc:301] APPLY: starting
0925 09:57:24.164546 (+ 55us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:24.164648 (+ 102us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:24.164752 (+ 104us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:24.164921 (+ 169us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:24.165050 (+ 129us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:24.165092 (+ 42us) write_op.cc:312] APPLY: finished
0925 09:57:24.173272 (+ 8180us) log.cc:842] Serialized 23 byte log entry
0925 09:57:24.173421 (+ 149us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:24.173492 (+ 71us) write_op.cc:454] Released schema lock
0925 09:57:24.174147 (+ 655us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:24.174368 (+ 221us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:24.174397 (+ 29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":38204,"reported_tablets":55,"child_traces":[["op",{"apply.queue_time_us":2772,"apply.run_cpu_time_us":1531,"apply.run_wall_time_us":10073,"num_ops":1,"prepare.queue_time_us":125,"prepare.run_cpu_time_us":2008,"prepare.run_wall_time_us":2269,"raft.queue_time_us":85,"raft.run_cpu_time_us":11,"raft.run_wall_time_us":12,"replication_time_us":123492,"thread_start_us":103103,"threads_started":1}]]}
I20260925 09:57:25.641469 11727 ts_tablet_manager.cc:1905] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:25.641322 10393 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_d1b168dcbc624b88980354edd6eff43a" } modify_external_catalogs: true
I20260925 09:57:25.642096 10393 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_d1b168dcbc624b88980354edd6eff43a" } modify_external_catalogs: true
I20260925 09:57:25.655948 11085 tablet_bootstrap.cc:654] T 60fdc299865446c89d3e5374d573029a P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:25.598999 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5246648245e04e26be2abf1dbf706688" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:25.665416 11709 ts_tablet_manager.cc:1918] T 359090244c4748e8a2d45e3c77e89498 P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.1
I20260925 09:57:25.666308 11709 log.cc:1196] T 359090244c4748e8a2d45e3c77e89498 P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/359090244c4748e8a2d45e3c77e89498
I20260925 09:57:25.667539 11709 ts_tablet_manager.cc:1939] T 359090244c4748e8a2d45e3c77e89498 P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:25.670557 11709 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 214) took 1200 ms. Trace:
W20260925 09:57:25.674638 10919 leader_election.cc:341] T 5246648245e04e26be2abf1dbf706688 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:25.675151 10919 leader_election.cc:302] T 5246648245e04e26be2abf1dbf706688 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:25.694223 11892 raft_consensus.cc:2726] T 5246648245e04e26be2abf1dbf706688 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:25.695997 10811 tablet_service.cc:1467] Processing CreateTablet for tablet d54ab15dd054489ab50b7f0b26b79a50 (DEFAULT_TABLE table=default.Table_62f952bfd0724ec792a45a6eb99a82c7 [id=dfc2a72fb633484f97a46deada8a84e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:25.697216 10811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d54ab15dd054489ab50b7f0b26b79a50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:25.700675 11084 tablet_bootstrap.cc:654] T 10657a81058241fc988d37e5ea7ff8cf P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:25.726013 10389 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 4e3120347c8e4a3eab57d6562ae40e8d
I20260925 09:57:25.727948 10389 catalog_manager.cc:6565] ReplaceTablet: tablet 4e3120347c8e4a3eab57d6562ae40e8d deleted and replaced by tablet c52745a8cdee461aba7485b0531245d6
I20260925 09:57:25.728868 10807 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e3120347c8e4a3eab57d6562ae40e8d with delete_type TABLET_DATA_DELETED (replaced by tablet c52745a8cdee461aba7485b0531245d6) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:25.732072 11780 raft_consensus.cc:482] T c2110d4ada2d45c2b07259374465a8df P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:25.732496 11780 raft_consensus.cc:504] T c2110d4ada2d45c2b07259374465a8df P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:25.734220 11780 leader_election.cc:288] T c2110d4ada2d45c2b07259374465a8df P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:25.736222 10385 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 10657a81058241fc988d37e5ea7ff8cf
I20260925 09:57:25.738668 10389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_e22cff6ba0484ecf82c555938f037b69"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:25.739982 10807 tablet_service.cc:1514] Processing DeleteTablet for tablet 10657a81058241fc988d37e5ea7ff8cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:25 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:25.741535 10959 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e3120347c8e4a3eab57d6562ae40e8d with delete_type TABLET_DATA_DELETED (replaced by tablet c52745a8cdee461aba7485b0531245d6) from {username='slave'} at 127.0.0.1:53560
W20260925 09:57:25.742453 10389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e22cff6ba0484ecf82c555938f037b69 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:25.743502 10983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2110d4ada2d45c2b07259374465a8df" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:25.744136 10983 raft_consensus.cc:2454] T c2110d4ada2d45c2b07259374465a8df P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
I20260925 09:57:25.744103 10390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_0f18e1fd434a47d2828c80f67ab5107f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:25.745514 10390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0f18e1fd434a47d2828c80f67ab5107f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:25.758877 10839 raft_consensus.cc:1261] T 26fae3fbfe374f348a1729c21ec751b9 P c43461c4ee854ae1ac52b873ccada318 [term 2 FOLLOWER]: Refusing update from remote peer b8850818e2ee42a082802d0095c08d21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260925 09:57:25.779073 11085 tablet_bootstrap.cc:492] T 60fdc299865446c89d3e5374d573029a P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:25.760722 11950 tablet.cc:1722] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:25.779536 11085 ts_tablet_manager.cc:1397] T 60fdc299865446c89d3e5374d573029a P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.212s user 0.005s sys 0.008s
I20260925 09:57:25.752409 11892 raft_consensus.cc:482] T fe91b39f91274a7ca61e9618d8cbb475 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:25.761868 10959 tablet_service.cc:1514] Processing DeleteTablet for tablet 10657a81058241fc988d37e5ea7ff8cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:25 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:25.781282 11892 raft_consensus.cc:504] T fe91b39f91274a7ca61e9618d8cbb475 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:25.805948 10385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_8ccc930f32b84671ab46826d4a9fceef"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:25.808204 10385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8ccc930f32b84671ab46826d4a9fceef in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:25.809677 10772 leader_election.cc:302] T c2110d4ada2d45c2b07259374465a8df P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:25.810897 11780 raft_consensus.cc:2781] T c2110d4ada2d45c2b07259374465a8df P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:25.811417 11780 raft_consensus.cc:482] T c2110d4ada2d45c2b07259374465a8df P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:25.811993 11780 raft_consensus.cc:3037] T c2110d4ada2d45c2b07259374465a8df P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:25.819568 11709 rpcz_store.cc:276] 0925 09:57:24.470546 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:24.470815 (+ 269us) service_pool.cc:227] Handling call
0925 09:57:25.127010 (+656195us) ts_tablet_manager.cc:1145] Deleting tablet 359090244c4748e8a2d45e3c77e89498
0925 09:57:25.127058 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:25.381989 (+254931us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:25.665160 (+283171us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:25.670499 (+ 5339us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":655540}
I20260925 09:57:25.821081 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 359090244c4748e8a2d45e3c77e89498 (table default.Table_3f0e11249b90447595581ed433d20c5b [id=d0cfc580b2c64772b90d93e44f1fdb85]) successfully deleted
I20260925 09:57:25.822012 11709 tablet_replica.cc:330] T 67ce2e3325214aa6bdc3347e9e8f82b0 P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:25.705957 11942 tablet.cc:1722] T 93e1b5837aff48d7907890c40dfcd0d0 P b8850818e2ee42a082802d0095c08d21: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:25.824156 11579 ts_tablet_manager.cc:1918] T 28d60531970a4ab58ce5ef99cb3a96bf P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:57:25.729130 10655 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e3120347c8e4a3eab57d6562ae40e8d with delete_type TABLET_DATA_DELETED (replaced by tablet c52745a8cdee461aba7485b0531245d6) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:25.735750 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2110d4ada2d45c2b07259374465a8df" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:25.757555 10661 tablet_service.cc:1514] Processing DeleteTablet for tablet 10657a81058241fc988d37e5ea7ff8cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:25 UTC) from {username='slave'} at 127.0.0.1:37876
W20260925 09:57:25.826206 10775 leader_election.cc:341] T c2110d4ada2d45c2b07259374465a8df P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:25.827152 11579 log.cc:1196] T 28d60531970a4ab58ce5ef99cb3a96bf P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/28d60531970a4ab58ce5ef99cb3a96bf
I20260925 09:57:25.838335 11579 ts_tablet_manager.cc:1939] T 28d60531970a4ab58ce5ef99cb3a96bf P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:25.843262 10959 tablet_service.cc:1467] Processing CreateTablet for tablet d54ab15dd054489ab50b7f0b26b79a50 (DEFAULT_TABLE table=default.Table_62f952bfd0724ec792a45a6eb99a82c7 [id=dfc2a72fb633484f97a46deada8a84e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:25.844657 10959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d54ab15dd054489ab50b7f0b26b79a50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:25.846580 11085 raft_consensus.cc:348] T 60fdc299865446c89d3e5374d573029a P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:25.847801 10621 consensus_peers.cc:487] T 28d60531970a4ab58ce5ef99cb3a96bf P b8850818e2ee42a082802d0095c08d21 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 28d60531970a4ab58ce5ef99cb3a96bf. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:25.848579 11085 raft_consensus.cc:374] T 60fdc299865446c89d3e5374d573029a P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:25.848915 11085 raft_consensus.cc:729] T 60fdc299865446c89d3e5374d573029a P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:25.760914 11919 consensus_queue.cc:1035] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:25.786296 10688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe91b39f91274a7ca61e9618d8cbb475" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:25.854458 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 28d60531970a4ab58ce5ef99cb3a96bf (table default.Table_ce09d3a8d8e84973894f33b18bbfdb22 [id=6851500c7dba44f1bf353bb79bafd97a]) successfully deleted
W20260925 09:57:25.855612 10919 leader_election.cc:341] T fe91b39f91274a7ca61e9618d8cbb475 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:25.788661 11082 tablet_bootstrap.cc:654] T 2d087d3e35814c90a03eccd90eeea86c P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:25.856423 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 359090244c4748e8a2d45e3c77e89498 because the tablet was not found. No further retry: Not found: Tablet not found: 359090244c4748e8a2d45e3c77e89498
I20260925 09:57:25.816053 10657 tablet_service.cc:1467] Processing CreateTablet for tablet d54ab15dd054489ab50b7f0b26b79a50 (DEFAULT_TABLE table=default.Table_62f952bfd0724ec792a45a6eb99a82c7 [id=dfc2a72fb633484f97a46deada8a84e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:25.858162 10657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d54ab15dd054489ab50b7f0b26b79a50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:25.858855 11085 consensus_queue.cc:260] T 60fdc299865446c89d3e5374d573029a P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:25.871213 10392 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 59f0c6489a33433899bb6e62088121b6 (table default.Table_2d40588792754851bf107d378c5950dd [id=64032d286bf94c4cbdddde18adfb739e]). Expected version 1 got 0
W20260925 09:57:25.877470 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 4be3f6e81ddf418fabc7b694a2e62f35 because the tablet was not found. No further retry: Not found: Tablet not found: 4be3f6e81ddf418fabc7b694a2e62f35
W20260925 09:57:25.879573 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 04fafb9dcf274806b11b2598e73853e9 because the tablet was not found. No further retry: Not found: Tablet not found: 04fafb9dcf274806b11b2598e73853e9
W20260925 09:57:25.880928 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 689ab2b6cf1640129f6db8b113cf2fc6 because the tablet was not found. No further retry: Not found: Tablet not found: 689ab2b6cf1640129f6db8b113cf2fc6
I20260925 09:57:25.884876 11579 tablet_replica.cc:330] T cf35f39bcd6a41128acb2a4936fcc316 P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:25.895112 11892 leader_election.cc:288] T fe91b39f91274a7ca61e9618d8cbb475 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:25.921597 11084 tablet_bootstrap.cc:492] T 10657a81058241fc988d37e5ea7ff8cf P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:25.894586 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe91b39f91274a7ca61e9618d8cbb475" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:25.863166 11780 raft_consensus.cc:504] T c2110d4ada2d45c2b07259374465a8df P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:25.953289 10619 consensus_peers.cc:487] T d9e454f82a4e4c429b58f32926319b7c P b8850818e2ee42a082802d0095c08d21 -> Peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Couldn't send request to peer be97ab5d75934a648d4ab76341193ec0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d9e454f82a4e4c429b58f32926319b7c. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:25.954113 11780 leader_election.cc:288] T c2110d4ada2d45c2b07259374465a8df P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:25.947261 11579 raft_consensus.cc:2229] T cf35f39bcd6a41128acb2a4936fcc316 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260925 09:57:25.940119 10918 leader_election.cc:341] T fe91b39f91274a7ca61e9618d8cbb475 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:25.963603 10918 leader_election.cc:302] T fe91b39f91274a7ca61e9618d8cbb475 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:25.964681 11084 ts_tablet_manager.cc:1397] T 10657a81058241fc988d37e5ea7ff8cf P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.475s user 0.008s sys 0.004s
I20260925 09:57:25.965420 10392 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 93e1b5837aff48d7907890c40dfcd0d0 (table default.Table_f8b067e4e4cf498bbf44d7bf24c29363 [id=23d14ed1d2d647fc8b3b5ef40c7dc0bb]). Expected version 1 got 0
I20260925 09:57:25.967651 11892 raft_consensus.cc:2726] T fe91b39f91274a7ca61e9618d8cbb475 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:25.972154 10392 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 3d652a76d96f4dbeafde15f3c16534a6 (table default.Table_be0f658aa41f4188bc16db37e96f4ad1 [id=73083c1560cb4db5a12964387cb49b73]). Expected version 1 got 0
I20260925 09:57:25.975008 10392 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 26fae3fbfe374f348a1729c21ec751b9 (table default.Table_0011387b45014de4b24b466f4cb907f7 [id=e3f7aa5e988c4b6c98be5cd3d465245d]). Expected version 1 got 0
I20260925 09:57:25.978647 10983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2110d4ada2d45c2b07259374465a8df" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
W20260925 09:57:25.979450 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 89a143a5623e4ef38027381a56c95227 because the tablet was not found. No further retry: Not found: Tablet not found: 89a143a5623e4ef38027381a56c95227
I20260925 09:57:25.980064 10393 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 417058493c6547c69e72347cef373292
I20260925 09:57:25.980160 10807 tablet_service.cc:1467] Processing CreateTablet for tablet 17bd30416fe14c0e9ab2e0c0bf5f2fb5 (DEFAULT_TABLE table=default.Table_4688c2ec003a44218fedb9f17dc0f5e3 [id=0e904cb2f51c411aa5fbdde8907a695d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:25.981120 10807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17bd30416fe14c0e9ab2e0c0bf5f2fb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:25.983001 10811 tablet_service.cc:1467] Processing CreateTablet for tablet f9c259867cf44c4c90e50864475a4970 (DEFAULT_TABLE table=default.Table_b1ed2ff673484b57bc734908b3c0ddf3 [id=5eec7096e6534be19f9a4a55ab0438b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:25.983551 10983 raft_consensus.cc:3037] T c2110d4ada2d45c2b07259374465a8df P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:25.984666 10811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9c259867cf44c4c90e50864475a4970. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:25.986001 11085 ts_tablet_manager.cc:1428] T 60fdc299865446c89d3e5374d573029a P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.206s user 0.013s sys 0.010s
I20260925 09:57:25.988775 10812 tablet_service.cc:1467] Processing CreateTablet for tablet c9fccada64d1411eb1f75480c8001c49 (DEFAULT_TABLE table=default.Table_757681cc28db4f3695a9df63d0260883 [id=3634ee6acb3b44258b099e7885f6e47d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:25.990430 10812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9fccada64d1411eb1f75480c8001c49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:25.995253 11085 tablet_bootstrap.cc:492] T 9e697e7146e34841bd64660b45ebf231 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:26.002015 11579 raft_consensus.cc:2258] T cf35f39bcd6a41128acb2a4936fcc316 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:26.011543 10955 tablet_service.cc:1467] Processing CreateTablet for tablet f9c259867cf44c4c90e50864475a4970 (DEFAULT_TABLE table=default.Table_b1ed2ff673484b57bc734908b3c0ddf3 [id=5eec7096e6534be19f9a4a55ab0438b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:26.012185 10816 tablet_service.cc:1467] Processing CreateTablet for tablet 54157e776efa4c6c8e2bd4c58321fcda (DEFAULT_TABLE table=default.Table_175fa05d29fa47b8a45cd68e94d78800 [id=edbacd029b4442a29e3e9ca78cc0b745]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:26.013105 10816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54157e776efa4c6c8e2bd4c58321fcda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:26.013238 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2110d4ada2d45c2b07259374465a8df" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:26.014444 10963 tablet_service.cc:1467] Processing CreateTablet for tablet 17bd30416fe14c0e9ab2e0c0bf5f2fb5 (DEFAULT_TABLE table=default.Table_4688c2ec003a44218fedb9f17dc0f5e3 [id=0e904cb2f51c411aa5fbdde8907a695d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:26.002525 10983 raft_consensus.cc:2454] T c2110d4ada2d45c2b07259374465a8df P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:26.016108 10958 tablet_service.cc:1467] Processing CreateTablet for tablet c9fccada64d1411eb1f75480c8001c49 (DEFAULT_TABLE table=default.Table_757681cc28db4f3695a9df63d0260883 [id=3634ee6acb3b44258b099e7885f6e47d]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 09:57:26.019573 10775 leader_election.cc:341] T c2110d4ada2d45c2b07259374465a8df P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:26.021273 10955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9c259867cf44c4c90e50864475a4970. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:26.024356 10963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17bd30416fe14c0e9ab2e0c0bf5f2fb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:26.024493 10772 leader_election.cc:302] T c2110d4ada2d45c2b07259374465a8df P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21
I20260925 09:57:26.026423 11954 raft_consensus.cc:2781] T c2110d4ada2d45c2b07259374465a8df P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:26.028149 11954 raft_consensus.cc:686] T c2110d4ada2d45c2b07259374465a8df P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Becoming Leader. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Running, Role: LEADER
I20260925 09:57:26.028923 10958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9fccada64d1411eb1f75480c8001c49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:26.031418 11954 consensus_queue.cc:237] T c2110d4ada2d45c2b07259374465a8df P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:26.033071 10806 tablet_service.cc:1467] Processing CreateTablet for tablet 3d84c717b6ac462eb331e25d80c918e4 (DEFAULT_TABLE table=default.Table_e29783826d5c4556865f6b2b079a00bc [id=ee74fddfbd6347ecb49ecb8aa4338e5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:26.035141 10806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d84c717b6ac462eb331e25d80c918e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:26.038718 10957 tablet_service.cc:1467] Processing CreateTablet for tablet 54157e776efa4c6c8e2bd4c58321fcda (DEFAULT_TABLE table=default.Table_175fa05d29fa47b8a45cd68e94d78800 [id=edbacd029b4442a29e3e9ca78cc0b745]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:26.040891 10957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54157e776efa4c6c8e2bd4c58321fcda. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:26.048342 10380 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 5608c5f9b65b4da7b2fbb56417c92d88 (table default.Table_1b4a040b8d60467a869df4b51c09cf46 [id=e46d114e5d484b66bb15eb0416b5ce0d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 5608c5f9b65b4da7b2fbb56417c92d88 consensus state has no leader
I20260925 09:57:25.999203 11084 raft_consensus.cc:348] T 10657a81058241fc988d37e5ea7ff8cf P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:26.055701 11084 raft_consensus.cc:374] T 10657a81058241fc988d37e5ea7ff8cf P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:26.055998 11084 raft_consensus.cc:729] T 10657a81058241fc988d37e5ea7ff8cf P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:26.056664 11084 consensus_queue.cc:260] T 10657a81058241fc988d37e5ea7ff8cf P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:26.059579 10661 tablet_service.cc:1467] Processing CreateTablet for tablet f9c259867cf44c4c90e50864475a4970 (DEFAULT_TABLE table=default.Table_b1ed2ff673484b57bc734908b3c0ddf3 [id=5eec7096e6534be19f9a4a55ab0438b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:26.061669 10655 tablet_service.cc:1467] Processing CreateTablet for tablet 17bd30416fe14c0e9ab2e0c0bf5f2fb5 (DEFAULT_TABLE table=default.Table_4688c2ec003a44218fedb9f17dc0f5e3 [id=0e904cb2f51c411aa5fbdde8907a695d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:26.063877 10819 tablet_service.cc:1467] Processing CreateTablet for tablet abfc8f390d434c47a8c4990f8406c270 (DEFAULT_TABLE table=default.Table_aad29b2508b744aaba08bf51b98cd527 [id=65b6e9ec3f114cea85e70f1509e3d1b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:26.064844 10819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abfc8f390d434c47a8c4990f8406c270. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:26.064600 10663 tablet_service.cc:1467] Processing CreateTablet for tablet c9fccada64d1411eb1f75480c8001c49 (DEFAULT_TABLE table=default.Table_757681cc28db4f3695a9df63d0260883 [id=3634ee6acb3b44258b099e7885f6e47d]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 09:57:25.981603 10388 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c2110d4ada2d45c2b07259374465a8df consensus state has no leader
W20260925 09:57:25.982362 10386 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5608c5f9b65b4da7b2fbb56417c92d88 consensus state has no leader
I20260925 09:57:26.067718 11709 raft_consensus.cc:2229] T 67ce2e3325214aa6bdc3347e9e8f82b0 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:26.067873 10666 tablet_service.cc:1467] Processing CreateTablet for tablet 54157e776efa4c6c8e2bd4c58321fcda (DEFAULT_TABLE table=default.Table_175fa05d29fa47b8a45cd68e94d78800 [id=edbacd029b4442a29e3e9ca78cc0b745]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:26.068529 11709 raft_consensus.cc:2258] T 67ce2e3325214aa6bdc3347e9e8f82b0 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:26.069121 10386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_1f25b6a5838b40758e7645d5d47ad04e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:26.069658 10658 tablet_service.cc:1467] Processing CreateTablet for tablet 3d84c717b6ac462eb331e25d80c918e4 (DEFAULT_TABLE table=default.Table_e29783826d5c4556865f6b2b079a00bc [id=ee74fddfbd6347ecb49ecb8aa4338e5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:26.070236 11709 ts_tablet_manager.cc:1905] T 67ce2e3325214aa6bdc3347e9e8f82b0 P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 09:57:26.070497 10386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1f25b6a5838b40758e7645d5d47ad04e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:26.071906 10389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_7af6fc9b30f54185a51b24ba24892166"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:26.072767 10661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9c259867cf44c4c90e50864475a4970. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:26.073405 10389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7af6fc9b30f54185a51b24ba24892166 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:26.075068 10667 tablet_service.cc:1514] Processing DeleteTablet for tablet 417058493c6547c69e72347cef373292 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:25 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:26.076684 10390 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_a749aff6c09e4425afc67db6a98196bd" } modify_external_catalogs: true
I20260925 09:57:26.079710 10388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_94b396db46ba43d88015187a199247fc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:26.081022 10388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_94b396db46ba43d88015187a199247fc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:26.082166 10666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54157e776efa4c6c8e2bd4c58321fcda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:26.077277 10390 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_a749aff6c09e4425afc67db6a98196bd" } modify_external_catalogs: true
I20260925 09:57:26.092101 10658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d84c717b6ac462eb331e25d80c918e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:26.096135 10655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17bd30416fe14c0e9ab2e0c0bf5f2fb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:26.100515 10667 tablet_service.cc:1467] Processing CreateTablet for tablet abfc8f390d434c47a8c4990f8406c270 (DEFAULT_TABLE table=default.Table_aad29b2508b744aaba08bf51b98cd527 [id=65b6e9ec3f114cea85e70f1509e3d1b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:26.101442 10667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abfc8f390d434c47a8c4990f8406c270. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:26.111075 10619 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 157129us
W20260925 09:57:26.113552 11960 log.cc:924] Time spent T 59f0c6489a33433899bb6e62088121b6 P be97ab5d75934a648d4ab76341193ec0: Append to log took a long time: real 0.122s user 0.000s sys 0.001s
I20260925 09:57:26.109915 11830 raft_consensus.cc:482] T aba253f22afa4200926f31a1b43e3970 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260925 09:57:26.119062 10919 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 60183.2us
I20260925 09:57:26.122020 10664 tablet_service.cc:1467] Processing CreateTablet for tablet b39f9e7102ff4a9c96a2cc5d2676f03d (DEFAULT_TABLE table=default.Table_3f0e11249b90447595581ed433d20c5b [id=d0cfc580b2c64772b90d93e44f1fdb85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:26.116231 10663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9fccada64d1411eb1f75480c8001c49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:26.116432 10387 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 93e1b5837aff48d7907890c40dfcd0d0 (table default.Table_f8b067e4e4cf498bbf44d7bf24c29363 [id=23d14ed1d2d647fc8b3b5ef40c7dc0bb]). Expected version 1 got 0
I20260925 09:57:26.131906 10664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b39f9e7102ff4a9c96a2cc5d2676f03d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:26.138181 10815 tablet_service.cc:1514] Processing DeleteTablet for tablet 417058493c6547c69e72347cef373292 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:25 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:26.139815 10815 tablet_service.cc:1467] Processing CreateTablet for tablet b39f9e7102ff4a9c96a2cc5d2676f03d (DEFAULT_TABLE table=default.Table_3f0e11249b90447595581ed433d20c5b [id=d0cfc580b2c64772b90d93e44f1fdb85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:26.140970 10815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b39f9e7102ff4a9c96a2cc5d2676f03d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:26.144042 11579 ts_tablet_manager.cc:1905] T cf35f39bcd6a41128acb2a4936fcc316 P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:26.140803 10805 tablet_service.cc:1467] Processing CreateTablet for tablet 8a3149fb2325435699af36338e77d042 (DEFAULT_TABLE table=default.Table_4353b0aceb674088b1805b3da35d0c68 [id=8744c891782841239ff886c348c05b7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:26.145692 10805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a3149fb2325435699af36338e77d042. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:26.124915 11830 raft_consensus.cc:504] T aba253f22afa4200926f31a1b43e3970 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:26.147979 10621 consensus_peers.cc:487] T d9e454f82a4e4c429b58f32926319b7c P b8850818e2ee42a082802d0095c08d21 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:26.150592 11830 leader_election.cc:288] T aba253f22afa4200926f31a1b43e3970 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:26.150913 10387 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 3d652a76d96f4dbeafde15f3c16534a6 (table default.Table_be0f658aa41f4188bc16db37e96f4ad1 [id=73083c1560cb4db5a12964387cb49b73]). Expected version 1 got 0
I20260925 09:57:26.153177 10834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aba253f22afa4200926f31a1b43e3970" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:26.154351 11889 raft_consensus.cc:482] T 60fdc299865446c89d3e5374d573029a P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260925 09:57:26.155113 10621 leader_election.cc:341] T aba253f22afa4200926f31a1b43e3970 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:26.155650 11889 raft_consensus.cc:504] T 60fdc299865446c89d3e5374d573029a P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:26.166270 11727 ts_tablet_manager.cc:1918] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:26.166787 11727 log.cc:1196] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/689ab2b6cf1640129f6db8b113cf2fc6
I20260925 09:57:26.171509 10387 catalog_manager.cc:5495] T a21909e2e5794f57971f4e42795235c7 P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:26.177116 10387 catalog_manager.cc:5495] T 7360a869f3be45a884a0379f4ef78e85 P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:26.175503 11901 raft_consensus.cc:482] T 6540aba5854a44419addc5d454e473b7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:26.188654 11901 raft_consensus.cc:504] T 6540aba5854a44419addc5d454e473b7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:26.190162 10390 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet cf775c9d9b114ddcb97dc6c6eca8e0fc
I20260925 09:57:26.195559 10659 tablet_service.cc:1514] Processing DeleteTablet for tablet cf775c9d9b114ddcb97dc6c6eca8e0fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:26 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:26.196317 11901 leader_election.cc:288] T 6540aba5854a44419addc5d454e473b7 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:26.197494 10683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6540aba5854a44419addc5d454e473b7" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:26.199206 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aba253f22afa4200926f31a1b43e3970" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:26.200191 10976 raft_consensus.cc:2454] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:26.202952 10956 tablet_service.cc:1467] Processing CreateTablet for tablet 3d84c717b6ac462eb331e25d80c918e4 (DEFAULT_TABLE table=default.Table_e29783826d5c4556865f6b2b079a00bc [id=ee74fddfbd6347ecb49ecb8aa4338e5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:26.204409 10961 tablet_service.cc:1514] Processing DeleteTablet for tablet 417058493c6547c69e72347cef373292 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:25 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:26.204398 10387 catalog_manager.cc:5495] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 2, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 2 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:26.205727 10387 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 26fae3fbfe374f348a1729c21ec751b9 (table default.Table_0011387b45014de4b24b466f4cb907f7 [id=e3f7aa5e988c4b6c98be5cd3d465245d]). Expected version 1 got 0
I20260925 09:57:26.205816 11889 leader_election.cc:288] T 60fdc299865446c89d3e5374d573029a P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
W20260925 09:57:26.206151 10919 leader_election.cc:341] T 6540aba5854a44419addc5d454e473b7 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:26.207027 11959 tablet.cc:1722] T 93e1b5837aff48d7907890c40dfcd0d0 P be97ab5d75934a648d4ab76341193ec0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:26.207129 11942 tablet.cc:1722] T 59f0c6489a33433899bb6e62088121b6 P b8850818e2ee42a082802d0095c08d21: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260925 09:57:26.211586 10384 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.3:41469 (request call id 23) took 2237 ms (client timeout 1000 ms). Trace:
I20260925 09:57:26.215709 11727 ts_tablet_manager.cc:1939] T 689ab2b6cf1640129f6db8b113cf2fc6 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:26.217525 10683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60fdc299865446c89d3e5374d573029a" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:26.224380 10775 leader_election.cc:341] T 60fdc299865446c89d3e5374d573029a P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:26.226496 11950 tablet.cc:1722] T 59f0c6489a33433899bb6e62088121b6 P c43461c4ee854ae1ac52b873ccada318: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:26.227443 11709 ts_tablet_manager.cc:1918] T 67ce2e3325214aa6bdc3347e9e8f82b0 P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:26.234822 10953 tablet_service.cc:1467] Processing CreateTablet for tablet abfc8f390d434c47a8c4990f8406c270 (DEFAULT_TABLE table=default.Table_aad29b2508b744aaba08bf51b98cd527 [id=65b6e9ec3f114cea85e70f1509e3d1b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:26.241441 11084 ts_tablet_manager.cc:1428] T 10657a81058241fc988d37e5ea7ff8cf P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.276s user 0.008s sys 0.000s
W20260925 09:57:26.249228 10384 rpcz_store.cc:269] 0925 09:57:23.973831 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:24.166917 (+193086us) service_pool.cc:227] Handling call
0925 09:57:24.288220 (+121303us) spinlock_profiling.cc:243] Waited 62.5 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2ad9c3e 00007fabf0610fb4 00007fabf0610f42 00007fabf0610e71 00007fabf0610dec 00007fabf06100a2 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:26.211513 (+1923293us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":20957,"reported_tablets":58,"spinlock_wait_cycles":143846144}
I20260925 09:57:26.250605 11709 log.cc:1196] T 67ce2e3325214aa6bdc3347e9e8f82b0 P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/67ce2e3325214aa6bdc3347e9e8f82b0
I20260925 09:57:26.253980 10807 tablet_service.cc:1514] Processing DeleteTablet for tablet cf775c9d9b114ddcb97dc6c6eca8e0fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:26 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:26.253928 10961 tablet_service.cc:1467] Processing CreateTablet for tablet b39f9e7102ff4a9c96a2cc5d2676f03d (DEFAULT_TABLE table=default.Table_3f0e11249b90447595581ed433d20c5b [id=d0cfc580b2c64772b90d93e44f1fdb85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:26.252893 11709 ts_tablet_manager.cc:1939] T 67ce2e3325214aa6bdc3347e9e8f82b0 P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:26.257174 11709 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 225) took 1024 ms. Trace:
I20260925 09:57:26.261582 11709 rpcz_store.cc:276] 0925 09:57:25.233121 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:25.233344 (+ 223us) service_pool.cc:227] Handling call
0925 09:57:25.821837 (+588493us) ts_tablet_manager.cc:1145] Deleting tablet 67ce2e3325214aa6bdc3347e9e8f82b0
0925 09:57:25.821881 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:26.172229 (+350348us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:26.227210 (+ 54981us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:26.257109 (+ 29899us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":175948,"tablet-delete.queue_time_us":587985}
I20260925 09:57:26.267971 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 67ce2e3325214aa6bdc3347e9e8f82b0 (table default.Table_5157be3699be4fa4ac153a69f6010ecc [id=9cd1024c9d784df9af5f8239e39d1be7]) successfully deleted
I20260925 09:57:26.269671 11709 tablet_replica.cc:330] T 28d60531970a4ab58ce5ef99cb3a96bf P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:26.272208 10960 tablet_service.cc:1514] Processing DeleteTablet for tablet 545ac025f2cb469da847979011a0b3ce with delete_type TABLET_DATA_DELETED (replaced by tablet ab6bdba5ca5d41ae921a90c1f77006c5) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:26.276319 11709 raft_consensus.cc:2229] T 28d60531970a4ab58ce5ef99cb3a96bf P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:26.280858 11709 raft_consensus.cc:2258] T 28d60531970a4ab58ce5ef99cb3a96bf P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:26.284686 10657 tablet_service.cc:1467] Processing CreateTablet for tablet b3d9a3cf5bda4cf6bc1fa3fae1cb9288 (DEFAULT_TABLE table=default.Table_580a41bad60e4da0aa7fe41a67dc8fe8 [id=4cefdaddfd6143468df4502cd35afac4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:26.295710 11709 ts_tablet_manager.cc:1905] T 28d60531970a4ab58ce5ef99cb3a96bf P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:26.298398 10657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3d9a3cf5bda4cf6bc1fa3fae1cb9288. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:26.313795 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6540aba5854a44419addc5d454e473b7" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:26.315613 11082 tablet_bootstrap.cc:492] T 2d087d3e35814c90a03eccd90eeea86c P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:26.316015 11082 ts_tablet_manager.cc:1397] T 2d087d3e35814c90a03eccd90eeea86c P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.976s user 0.014s sys 0.005s
I20260925 09:57:26.318542 10664 tablet_service.cc:1467] Processing CreateTablet for tablet 8a3149fb2325435699af36338e77d042 (DEFAULT_TABLE table=default.Table_4353b0aceb674088b1805b3da35d0c68 [id=8744c891782841239ff886c348c05b7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:26.322057 10960 tablet_service.cc:1514] Processing DeleteTablet for tablet cf775c9d9b114ddcb97dc6c6eca8e0fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:26 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:26.326501 10960 tablet_service.cc:1467] Processing CreateTablet for tablet 8a3149fb2325435699af36338e77d042 (DEFAULT_TABLE table=default.Table_4353b0aceb674088b1805b3da35d0c68 [id=8744c891782841239ff886c348c05b7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:26.340775 10619 leader_election.cc:302] T aba253f22afa4200926f31a1b43e3970 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:26.342182 11830 raft_consensus.cc:2781] T aba253f22afa4200926f31a1b43e3970 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:26.342803 11830 raft_consensus.cc:482] T aba253f22afa4200926f31a1b43e3970 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:26.343595 11830 raft_consensus.cc:3037] T aba253f22afa4200926f31a1b43e3970 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:57:26.348019 10621 consensus_peers.cc:487] T 9e697e7146e34841bd64660b45ebf231 P b8850818e2ee42a082802d0095c08d21 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:26.351223 10956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d84c717b6ac462eb331e25d80c918e4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:26.358706 10918 leader_election.cc:341] T 6540aba5854a44419addc5d454e473b7 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:26.359292 10918 leader_election.cc:302] T 6540aba5854a44419addc5d454e473b7 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:26.360936 11892 raft_consensus.cc:2726] T 6540aba5854a44419addc5d454e473b7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:26.365011 10386 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_1c14b5a0f00f43c38e3c5e0abea90e0b" } new_table_name: "default.Table_25edfe533c64421da7700ac5d60265c3" modify_external_catalogs: true
I20260925 09:57:26.368412 10384 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_908fa06a45334bf7a6700063843028b1" } modify_external_catalogs: true
I20260925 09:57:26.369092 10384 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_908fa06a45334bf7a6700063843028b1" } modify_external_catalogs: true
I20260925 09:57:26.369925 10390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_94f22b074c71401b9899866c584a0190"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:26.372236 10388 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_d586d012b3534ee79591128c9aac74ac" } new_table_name: "default.Table_0d7d27d05491401089db5f65b43e86f7" modify_external_catalogs: true
I20260925 09:57:26.372534 10953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abfc8f390d434c47a8c4990f8406c270. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:26.373270 10390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_94f22b074c71401b9899866c584a0190 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:26.376374 11084 tablet_bootstrap.cc:492] T defaef4d6b754ca5a2dfc21b7a596a5b P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:26.378926 11815 consensus_queue.cc:1035] T 55d14d4d8abe4ec9ac9aeef0d027e804 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:26.380275 10961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b39f9e7102ff4a9c96a2cc5d2676f03d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:26.382145 10960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a3149fb2325435699af36338e77d042. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:26.414256 10389 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_c720bf4c23b44a94b73ef67ad45560ce" } modify_external_catalogs: true
I20260925 09:57:26.414812 10389 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_c720bf4c23b44a94b73ef67ad45560ce" } modify_external_catalogs: true
I20260925 09:57:26.407315 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 190) took 3771 ms. Trace:
I20260925 09:57:26.431929 11727 rpcz_store.cc:276] 0925 09:57:22.635884 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:22.636009 (+ 125us) service_pool.cc:227] Handling call
0925 09:57:25.482959 (+2846950us) ts_tablet_manager.cc:1145] Deleting tablet 689ab2b6cf1640129f6db8b113cf2fc6
0925 09:57:25.483018 (+ 59us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:25.962834 (+479816us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:26.165966 (+203132us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:26.407211 (+241245us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":150811,"tablet-delete.queue_time_us":2846354}
I20260925 09:57:26.433679 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 191) took 3727 ms. Trace:
I20260925 09:57:26.434099 11727 rpcz_store.cc:276] 0925 09:57:22.706766 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:22.706883 (+ 117us) service_pool.cc:227] Handling call
0925 09:57:26.433349 (+3726466us) ts_tablet_manager.cc:1145] Deleting tablet 4be3f6e81ddf418fabc7b694a2e62f35
0925 09:57:26.433420 (+ 71us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:26.433625 (+ 205us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3725879}
I20260925 09:57:26.435456 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 192) took 3722 ms. Trace:
I20260925 09:57:26.459671 10664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a3149fb2325435699af36338e77d042. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:26.471154 11082 raft_consensus.cc:348] T 2d087d3e35814c90a03eccd90eeea86c P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:26.472074 11082 raft_consensus.cc:374] T 2d087d3e35814c90a03eccd90eeea86c P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:26.472409 11082 raft_consensus.cc:729] T 2d087d3e35814c90a03eccd90eeea86c P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:26.476426 10816 tablet_service.cc:1467] Processing CreateTablet for tablet b3d9a3cf5bda4cf6bc1fa3fae1cb9288 (DEFAULT_TABLE table=default.Table_580a41bad60e4da0aa7fe41a67dc8fe8 [id=4cefdaddfd6143468df4502cd35afac4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:26.492319 10393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_04c588671331474fac69a076bb87ba84"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:26.493911 10393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_04c588671331474fac69a076bb87ba84 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:26.496342 10816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3d9a3cf5bda4cf6bc1fa3fae1cb9288. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:26.497555 10772 leader_election.cc:341] T 60fdc299865446c89d3e5374d573029a P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 60fdc299865446c89d3e5374d573029a
I20260925 09:57:26.498003 10772 leader_election.cc:302] T 60fdc299865446c89d3e5374d573029a P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:26.499040 11780 raft_consensus.cc:2726] T 60fdc299865446c89d3e5374d573029a P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:26.504240 11727 rpcz_store.cc:276] 0925 09:57:22.712615 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:22.712740 (+ 125us) service_pool.cc:227] Handling call
0925 09:57:26.435075 (+3722335us) ts_tablet_manager.cc:1145] Deleting tablet 752105c1514f4c1db2a11b95b413b4f6
0925 09:57:26.435155 (+ 80us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:26.435383 (+ 228us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3721853}
I20260925 09:57:26.504311 11082 consensus_queue.cc:260] T 2d087d3e35814c90a03eccd90eeea86c P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:26.425521 11084 tablet_bootstrap.cc:654] T defaef4d6b754ca5a2dfc21b7a596a5b P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:26.489327 11959 tablet.cc:1722] T 59f0c6489a33433899bb6e62088121b6 P be97ab5d75934a648d4ab76341193ec0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:26.514392 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 689ab2b6cf1640129f6db8b113cf2fc6 (table default.Table_077f38b3329348828ef80c0f4e0c972f [id=51887beca2a54a4bb1e8f4262bc4fb4a]) successfully deleted
I20260925 09:57:26.522123 11709 ts_tablet_manager.cc:1918] T 28d60531970a4ab58ce5ef99cb3a96bf P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:57:26.522521 11709 log.cc:1196] T 28d60531970a4ab58ce5ef99cb3a96bf P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/28d60531970a4ab58ce5ef99cb3a96bf
I20260925 09:57:26.523232 11709 ts_tablet_manager.cc:1939] T 28d60531970a4ab58ce5ef99cb3a96bf P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:26.537738 11709 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 224) took 1304 ms. Trace:
I20260925 09:57:26.538049 11709 rpcz_store.cc:276] 0925 09:57:25.232846 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:25.253429 (+ 20583us) service_pool.cc:227] Handling call
0925 09:57:26.269478 (+1016049us) ts_tablet_manager.cc:1145] Deleting tablet 28d60531970a4ab58ce5ef99cb3a96bf
0925 09:57:26.269534 (+ 56us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:26.345774 (+ 76240us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:26.475193 (+129419us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:26.537685 (+ 62492us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1014959}
I20260925 09:57:26.539273 11709 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 227) took 1206 ms. Trace:
I20260925 09:57:26.539600 11709 rpcz_store.cc:276] 0925 09:57:25.332530 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:25.332686 (+ 156us) service_pool.cc:227] Handling call
0925 09:57:26.539004 (+1206318us) ts_tablet_manager.cc:1145] Deleting tablet 689ab2b6cf1640129f6db8b113cf2fc6
0925 09:57:26.539056 (+ 52us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:26.539223 (+ 167us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1205185}
I20260925 09:57:26.540632 11709 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 226) took 1208 ms. Trace:
I20260925 09:57:26.540934 11709 rpcz_store.cc:276] 0925 09:57:25.332177 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:25.335480 (+ 3303us) service_pool.cc:227] Handling call
0925 09:57:26.540366 (+1204886us) ts_tablet_manager.cc:1145] Deleting tablet cf35f39bcd6a41128acb2a4936fcc316
0925 09:57:26.540403 (+ 37us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:26.540587 (+ 184us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1204041}
I20260925 09:57:26.551651 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 193) took 3832 ms. Trace:
I20260925 09:57:26.552035 11727 rpcz_store.cc:276] 0925 09:57:22.719408 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:22.719526 (+ 118us) service_pool.cc:227] Handling call
0925 09:57:26.519645 (+3800119us) ts_tablet_manager.cc:1145] Deleting tablet c07c6205d1284a6fbb369c010a226da7
0925 09:57:26.551372 (+ 31727us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:26.551592 (+ 220us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3799519}
I20260925 09:57:26.554623 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 195) took 3834 ms. Trace:
I20260925 09:57:26.554951 11727 rpcz_store.cc:276] 0925 09:57:22.720027 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:22.721120 (+ 1093us) service_pool.cc:227] Handling call
0925 09:57:26.554292 (+3833172us) ts_tablet_manager.cc:1145] Deleting tablet c16e096038084b2b9148390b6e773294
0925 09:57:26.554360 (+ 68us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:26.554572 (+ 212us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3832635}
I20260925 09:57:26.555974 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 28d60531970a4ab58ce5ef99cb3a96bf (table default.Table_ce09d3a8d8e84973894f33b18bbfdb22 [id=6851500c7dba44f1bf353bb79bafd97a]) successfully deleted
I20260925 09:57:26.556170 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 197) took 3817 ms. Trace:
W20260925 09:57:26.556651 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 689ab2b6cf1640129f6db8b113cf2fc6 because the tablet was not found. No further retry: Not found: Tablet not found: 689ab2b6cf1640129f6db8b113cf2fc6
I20260925 09:57:26.556663 11727 rpcz_store.cc:276] 0925 09:57:22.739287 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:22.744814 (+ 5527us) service_pool.cc:227] Handling call
0925 09:57:26.555855 (+3811041us) ts_tablet_manager.cc:1145] Deleting tablet 04fafb9dcf274806b11b2598e73853e9
0925 09:57:26.555911 (+ 56us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:26.556118 (+ 207us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3810446}
W20260925 09:57:26.557210 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet cf35f39bcd6a41128acb2a4936fcc316 because the tablet was not found. No further retry: Not found: Tablet not found: cf35f39bcd6a41128acb2a4936fcc316
I20260925 09:57:26.557852 11727 tablet_replica.cc:330] T a44d9f2f7e3e48aeb0f31fa29ae1988a P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:26.558916 11727 raft_consensus.cc:2229] T a44d9f2f7e3e48aeb0f31fa29ae1988a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260925 09:57:26.559227 10386 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f0bf545f70ae4cd6a1ad2540c7158ecc has no consensus state
W20260925 09:57:26.560391 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 4be3f6e81ddf418fabc7b694a2e62f35 because the tablet was not found. No further retry: Not found: Tablet not found: 4be3f6e81ddf418fabc7b694a2e62f35
I20260925 09:57:26.569259 11082 ts_tablet_manager.cc:1428] T 2d087d3e35814c90a03eccd90eeea86c P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.253s user 0.007s sys 0.000s
I20260925 09:57:26.570309 11082 tablet_bootstrap.cc:492] T 3b9cdad4f9144daea800553abd378e0b P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:26.573247 11830 raft_consensus.cc:504] T aba253f22afa4200926f31a1b43e3970 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:26.575554 11830 leader_election.cc:288] T aba253f22afa4200926f31a1b43e3970 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:26.585441 10391 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 59f0c6489a33433899bb6e62088121b6 (table default.Table_2d40588792754851bf107d378c5950dd [id=64032d286bf94c4cbdddde18adfb739e]). Expected version 1 got 0
I20260925 09:57:26.586673 10391 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 93e1b5837aff48d7907890c40dfcd0d0 (table default.Table_f8b067e4e4cf498bbf44d7bf24c29363 [id=23d14ed1d2d647fc8b3b5ef40c7dc0bb]). Expected version 1 got 0
I20260925 09:57:26.593504 10391 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 26fae3fbfe374f348a1729c21ec751b9 (table default.Table_0011387b45014de4b24b466f4cb907f7 [id=e3f7aa5e988c4b6c98be5cd3d465245d]). Expected version 1 got 0
I20260925 09:57:26.598109 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aba253f22afa4200926f31a1b43e3970" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
W20260925 09:57:26.600291 10621 leader_election.cc:341] T aba253f22afa4200926f31a1b43e3970 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:26.599349 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 752105c1514f4c1db2a11b95b413b4f6 because the tablet was not found. No further retry: Not found: Tablet not found: 752105c1514f4c1db2a11b95b413b4f6
I20260925 09:57:26.587278 10955 tablet_service.cc:1467] Processing CreateTablet for tablet b3d9a3cf5bda4cf6bc1fa3fae1cb9288 (DEFAULT_TABLE table=default.Table_580a41bad60e4da0aa7fe41a67dc8fe8 [id=4cefdaddfd6143468df4502cd35afac4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:26.601871 10955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3d9a3cf5bda4cf6bc1fa3fae1cb9288. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:26.601828 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet c07c6205d1284a6fbb369c010a226da7 because the tablet was not found. No further retry: Not found: Tablet not found: c07c6205d1284a6fbb369c010a226da7
I20260925 09:57:26.603338 10391 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 3d652a76d96f4dbeafde15f3c16534a6 (table default.Table_be0f658aa41f4188bc16db37e96f4ad1 [id=73083c1560cb4db5a12964387cb49b73]). Expected version 1 got 0
I20260925 09:57:26.581913 11940 consensus_queue.cc:1035] T 9e697e7146e34841bd64660b45ebf231 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:26.609418 11919 consensus_queue.cc:1035] T 55d14d4d8abe4ec9ac9aeef0d027e804 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:26.614688 11579 ts_tablet_manager.cc:1918] T cf35f39bcd6a41128acb2a4936fcc316 P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:26.616305 11579 log.cc:1196] T cf35f39bcd6a41128acb2a4936fcc316 P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/cf35f39bcd6a41128acb2a4936fcc316
I20260925 09:57:26.617069 11579 ts_tablet_manager.cc:1939] T cf35f39bcd6a41128acb2a4936fcc316 P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
W20260925 09:57:26.619599 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet c16e096038084b2b9148390b6e773294 because the tablet was not found. No further retry: Not found: Tablet not found: c16e096038084b2b9148390b6e773294
W20260925 09:57:26.620154 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 04fafb9dcf274806b11b2598e73853e9 because the tablet was not found. No further retry: Not found: Tablet not found: 04fafb9dcf274806b11b2598e73853e9
I20260925 09:57:26.621731 10983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aba253f22afa4200926f31a1b43e3970" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:26.622255 10983 raft_consensus.cc:3037] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:26.628055 10983 raft_consensus.cc:2454] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:26.633112 10385 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 3d652a76d96f4dbeafde15f3c16534a6 (table default.Table_be0f658aa41f4188bc16db37e96f4ad1 [id=73083c1560cb4db5a12964387cb49b73]). Expected version 1 got 0
I20260925 09:57:26.633953 10385 catalog_manager.cc:5495] T 55d14d4d8abe4ec9ac9aeef0d027e804 P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:26.638988 11084 tablet_bootstrap.cc:492] T defaef4d6b754ca5a2dfc21b7a596a5b P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:26.639250 10384 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 3039ec3febf848dba7c4e6fd38bb8053
I20260925 09:57:26.639456 11084 ts_tablet_manager.cc:1397] T defaef4d6b754ca5a2dfc21b7a596a5b P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.263s user 0.008s sys 0.008s
I20260925 09:57:26.641610 10389 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 9a73a1f242e7430cbf52229c07b4dc82
I20260925 09:57:26.641547 11084 raft_consensus.cc:348] T defaef4d6b754ca5a2dfc21b7a596a5b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:26.642161 11084 raft_consensus.cc:374] T defaef4d6b754ca5a2dfc21b7a596a5b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:26.642402 11084 raft_consensus.cc:729] T defaef4d6b754ca5a2dfc21b7a596a5b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:26.643018 10389 master_service.cc:946] ReplaceTablet: received request to replace tablet 69028aa7908c45b896da4ba8deef07b1 from {username='slave'} at 127.0.0.1:35140
I20260925 09:57:26.642989 11084 consensus_queue.cc:260] T defaef4d6b754ca5a2dfc21b7a596a5b P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:26.646875 10666 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a73a1f242e7430cbf52229c07b4dc82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:26 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:26.647897 11084 ts_tablet_manager.cc:1428] T defaef4d6b754ca5a2dfc21b7a596a5b P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20260925 09:57:26.658099 10384 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_ff6257b50da64b15b5f006b7f178ca4a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:26.658510 10390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_63486992aef8441d8596bf9a0e3b98a9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:26.660079 10390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_63486992aef8441d8596bf9a0e3b98a9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 09:57:26.660894 10384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ff6257b50da64b15b5f006b7f178ca4a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:26.669283 10385 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 26fae3fbfe374f348a1729c21ec751b9 (table default.Table_0011387b45014de4b24b466f4cb907f7 [id=e3f7aa5e988c4b6c98be5cd3d465245d]). Expected version 1 got 0
I20260925 09:57:26.627660 11085 tablet_bootstrap.cc:654] T 9e697e7146e34841bd64660b45ebf231 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:26.656183 10815 tablet_service.cc:1514] Processing DeleteTablet for tablet 3039ec3febf848dba7c4e6fd38bb8053 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:26 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:26.671325 10385 catalog_manager.cc:5495] T 9e697e7146e34841bd64660b45ebf231 P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:26.674214 10392 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: default.Table_6eff7b87b3b74ca4bab719140667a87b [id=7889081642604700bbc6d253ab105260] alter complete (version 2)
I20260925 09:57:26.657467 10816 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a73a1f242e7430cbf52229c07b4dc82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:26 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:26.659423 11727 raft_consensus.cc:2258] T a44d9f2f7e3e48aeb0f31fa29ae1988a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:26.685814 11727 ts_tablet_manager.cc:1905] T a44d9f2f7e3e48aeb0f31fa29ae1988a P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:26.671573 10953 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a73a1f242e7430cbf52229c07b4dc82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:26 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:26.672087 11901 raft_consensus.cc:482] T 10657a81058241fc988d37e5ea7ff8cf P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:26.687624 11901 raft_consensus.cc:504] T 10657a81058241fc988d37e5ea7ff8cf P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:26.689333 11901 leader_election.cc:288] T 10657a81058241fc988d37e5ea7ff8cf P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:26.690937 10956 tablet_service.cc:1514] Processing DeleteTablet for tablet 3039ec3febf848dba7c4e6fd38bb8053 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:26 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:26.691171 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10657a81058241fc988d37e5ea7ff8cf" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:26.691587 10387 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.1:56563 (request call id 34) took 1412 ms (client timeout 1000 ms). Trace:
W20260925 09:57:26.691926 10387 rpcz_store.cc:269] 0925 09:57:25.279190 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:25.409615 (+130425us) service_pool.cc:227] Handling call
0925 09:57:25.503272 (+ 93657us) spinlock_profiling.cc:243] Waited 76.5 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2ad9c3e 00007fabf0610fb4 00007fabf0610f42 00007fabf0610e71 00007fabf0610dec 00007fabf06100a2 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:26.691511 (+1188239us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:26.345121 (+ 0us) spinlock_profiling.cc:243] Waited 83.1 ms on lock 0x7b5400011728. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf2fa7e91 00007fabf2fa6afc 00007fabf0f7c7cc 00007fabf0f68603 00007fabf0f682b8 00007fabf0f6cba1 00007fabf0f6cb59 00007fabf0f6cae9 00007fabf0f6cab1 00007fabf0f6bdad 0000000000580784
0925 09:57:26.362133 (+ 17012us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:26.362584 (+ 451us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:26.362608 (+ 24us) write_op.cc:435] Acquired schema lock
0925 09:57:26.362620 (+ 12us) tablet.cc:660] Decoding operations
0925 09:57:26.374567 (+ 11947us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:26.382869 (+ 8302us) spinlock_profiling.cc:243] Waited 6.22 ms on lock 0x7b6000070278. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0ea0361 00007fabf0fdb8f3 00007fabf0fdb663 00007fabf0f7f9c6 00007fabf0e7f4f5 00007fabf0f7cdab 00007fabf0f68603 00007fabf0f682b8 00007fabf0f6cba1 00007fabf0f6cb59 00007fabf0f6cae9
0925 09:57:26.382911 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:26.382937 (+ 26us) tablet.cc:683] Acquiring locks for 3 operations
0925 09:57:26.383232 (+ 295us) tablet.cc:699] Row locks acquired
0925 09:57:26.383248 (+ 16us) write_op.cc:260] PREPARE: finished
0925 09:57:26.383360 (+ 112us) write_op.cc:270] Start()
0925 09:57:26.392890 (+ 9530us) write_op.cc:276] Timestamp: P: 1790330246383290 usec, L: 0
0925 09:57:26.392912 (+ 22us) op_driver.cc:331] REPLICATION: starting
0925 09:57:26.393502 (+ 590us) log.cc:842] Serialized 1266 byte log entry
0925 09:57:26.554717 (+161215us) op_driver.cc:444] REPLICATION: finished
0925 09:57:26.554974 (+ 257us) write_op.cc:301] APPLY: starting
0925 09:57:26.555034 (+ 60us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:26.555163 (+ 129us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:26.555202 (+ 39us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:26.563921 (+ 8719us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:26.564069 (+ 148us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0925 09:57:26.564112 (+ 43us) write_op.cc:312] APPLY: finished
0925 09:57:26.564292 (+ 180us) log.cc:842] Serialized 35 byte log entry
0925 09:57:26.564495 (+ 203us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:26.564621 (+ 126us) write_op.cc:454] Released schema lock
0925 09:57:26.564675 (+ 54us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:26.564843 (+ 168us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:26.564867 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1050,"reported_tablets":20,"spinlock_wait_cycles":175841024,"child_traces":[["op",{"apply.queue_time_us":88,"apply.run_cpu_time_us":2029,"apply.run_wall_time_us":10089,"num_ops":3,"prepare.queue_time_us":745,"prepare.run_cpu_time_us":6540,"prepare.run_wall_time_us":133469,"replication_time_us":161719,"spinlock_wait_cycles":205330560}]]}
I20260925 09:57:26.692170 11084 tablet_bootstrap.cc:492] T a436fe287a0d48a1821191adcd031d74 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:26.692507 11901 raft_consensus.cc:482] T e684798155d445bfb325a1b7c5c48753 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:26.696889 11901 raft_consensus.cc:504] T e684798155d445bfb325a1b7c5c48753 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:26.699239 11901 leader_election.cc:288] T e684798155d445bfb325a1b7c5c48753 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
W20260925 09:57:26.692993 10918 leader_election.cc:341] T 10657a81058241fc988d37e5ea7ff8cf P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:26.700805 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e684798155d445bfb325a1b7c5c48753" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:26.702158 10918 leader_election.cc:341] T e684798155d445bfb325a1b7c5c48753 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:26.711803 10393 master_service.cc:946] ReplaceTablet: received request to replace tablet a81530200cf14717bc7cbf67bb73f4e5 from {username='slave'} at 127.0.0.1:35140
I20260925 09:57:26.712337 10619 leader_election.cc:302] T aba253f22afa4200926f31a1b43e3970 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:26.726848 11727 ts_tablet_manager.cc:1918] T a44d9f2f7e3e48aeb0f31fa29ae1988a P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:26.727190 11727 log.cc:1196] T a44d9f2f7e3e48aeb0f31fa29ae1988a P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/a44d9f2f7e3e48aeb0f31fa29ae1988a
I20260925 09:57:26.734369 11727 ts_tablet_manager.cc:1939] T a44d9f2f7e3e48aeb0f31fa29ae1988a P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:26.741495 11709 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 228) took 1407 ms. Trace:
I20260925 09:57:26.744052 11579 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 228) took 1273 ms. Trace:
I20260925 09:57:26.744302 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 200) took 3992 ms. Trace:
I20260925 09:57:26.744385 11579 rpcz_store.cc:276] 0925 09:57:25.471062 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:25.471544 (+ 482us) service_pool.cc:227] Handling call
0925 09:57:25.884742 (+413198us) ts_tablet_manager.cc:1145] Deleting tablet cf35f39bcd6a41128acb2a4936fcc316
0925 09:57:25.884775 (+ 33us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:26.141741 (+256966us) spinlock_profiling.cc:243] Waited 62.6 ms on lock 0x7b2800098fa8. stack: 00007f0270186033 00007f0270185f30 00007f026fd07534 00000000004db2b4 00000000004db249 00007f0278ffe8b1 00007f02790121ea 00007f027747e49f 00007f027752b3fb 00007f0278138cfa 00007f0278168e10 00007f027814fe0d 00007f027814fdb9 00007f027814fd49 00007f027814fd11 00007f027814ef3d
0925 09:57:26.334316 (+192575us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:26.614367 (+280051us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:26.743990 (+129623us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":33796,"spinlock_wait_cycles":143955840,"tablet-delete.queue_time_us":412677}
I20260925 09:57:26.744733 11727 rpcz_store.cc:276] 0925 09:57:22.752343 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:22.752532 (+ 189us) service_pool.cc:227] Handling call
0925 09:57:26.557675 (+3805143us) ts_tablet_manager.cc:1145] Deleting tablet a44d9f2f7e3e48aeb0f31fa29ae1988a
0925 09:57:26.557735 (+ 60us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:26.710706 (+152971us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:26.726647 (+ 15941us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:26.744203 (+ 17556us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3804556}
I20260925 09:57:26.746281 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet a44d9f2f7e3e48aeb0f31fa29ae1988a (table default.Table_eb065d5979c64931abf91d82b8623a0a [id=5b74f0855d2046e3a9463f732ae19e62]) successfully deleted
I20260925 09:57:26.746397 11579 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 229) took 1275 ms. Trace:
I20260925 09:57:26.746769 11579 rpcz_store.cc:276] 0925 09:57:25.471448 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:25.472177 (+ 729us) service_pool.cc:227] Handling call
0925 09:57:26.746136 (+1273959us) ts_tablet_manager.cc:1145] Deleting tablet 67ce2e3325214aa6bdc3347e9e8f82b0
0925 09:57:26.746184 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:26.746348 (+ 164us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1273373}
I20260925 09:57:26.747620 11579 tablet_replica.cc:330] T 3be0eccc65564bfba3d6e5c398c61fb6 P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:26.749018 11579 raft_consensus.cc:2229] T 3be0eccc65564bfba3d6e5c398c61fb6 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:26.749680 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet cf35f39bcd6a41128acb2a4936fcc316 (table default.Table_fea25102a7f84d58927645a0feae51e3 [id=61a4c50b2c6e43c1a471f63b59d218cb]) successfully deleted
W20260925 09:57:26.750399 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 67ce2e3325214aa6bdc3347e9e8f82b0 because the tablet was not found. No further retry: Not found: Tablet not found: 67ce2e3325214aa6bdc3347e9e8f82b0
I20260925 09:57:26.754050 11727 tablet_replica.cc:330] T 977a59175e5b43eeb6f23b938afd00a3 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:26.754405 11579 raft_consensus.cc:2258] T 3be0eccc65564bfba3d6e5c398c61fb6 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:26.758699 11579 ts_tablet_manager.cc:1905] T 3be0eccc65564bfba3d6e5c398c61fb6 P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:26.762598 10816 tablet_service.cc:1514] Processing DeleteTablet for tablet cf35f39bcd6a41128acb2a4936fcc316 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:16 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:26.771268 11084 tablet_bootstrap.cc:654] T a436fe287a0d48a1821191adcd031d74 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:26.783661 11709 rpcz_store.cc:276] 0925 09:57:25.334544 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:25.334767 (+ 223us) service_pool.cc:227] Handling call
0925 09:57:26.541766 (+1206999us) ts_tablet_manager.cc:1145] Deleting tablet 3706253e53d841aeaa43889ead13af32
0925 09:57:26.541804 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:26.741426 (+199622us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1205091}
I20260925 09:57:26.790020 10387 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_580a41bad60e4da0aa7fe41a67dc8fe8" } new_table_name: "default.Table_cc9f467e078c448f87392bf324b040a8" modify_external_catalogs: true
I20260925 09:57:26.794672 10389 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 69028aa7908c45b896da4ba8deef07b1
I20260925 09:57:26.797251 10389 catalog_manager.cc:6565] ReplaceTablet: tablet 69028aa7908c45b896da4ba8deef07b1 deleted and replaced by tablet 2015c10f503e4751ad9ff77404fca50b
I20260925 09:57:26.797739 10955 tablet_service.cc:1514] Processing DeleteTablet for tablet 69028aa7908c45b896da4ba8deef07b1 with delete_type TABLET_DATA_DELETED (replaced by tablet 2015c10f503e4751ad9ff77404fca50b) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:26.804379 10816 tablet_service.cc:1514] Processing DeleteTablet for tablet 69028aa7908c45b896da4ba8deef07b1 with delete_type TABLET_DATA_DELETED (replaced by tablet 2015c10f503e4751ad9ff77404fca50b) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:26.806427 10816 tablet_service.cc:1514] Processing DeleteTablet for tablet ec1b78e61e8d46cb8b87f60786059658 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:21 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:26.808149 10816 tablet_service.cc:1514] Processing DeleteTablet for tablet 28d60531970a4ab58ce5ef99cb3a96bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:24 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:26.810601 10816 tablet_service.cc:1514] Processing DeleteTablet for tablet 67ce2e3325214aa6bdc3347e9e8f82b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:23 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:26.812546 10955 tablet_service.cc:1514] Processing DeleteTablet for tablet 689ab2b6cf1640129f6db8b113cf2fc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:20 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:26.813676 11727 raft_consensus.cc:2229] T 977a59175e5b43eeb6f23b938afd00a3 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:26.814159 11727 raft_consensus.cc:2258] T 977a59175e5b43eeb6f23b938afd00a3 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:26.816305 11727 ts_tablet_manager.cc:1905] T 977a59175e5b43eeb6f23b938afd00a3 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:26.819710 10955 tablet_service.cc:1514] Processing DeleteTablet for tablet a44d9f2f7e3e48aeb0f31fa29ae1988a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:18 UTC) from {username='slave'} at 127.0.0.1:53560
W20260925 09:57:26.820329 10391 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.2:40761 (request call id 32) took 1377 ms (client timeout 1000 ms). Trace:
W20260925 09:57:26.820677 10391 rpcz_store.cc:269] 0925 09:57:25.442891 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:25.518210 (+ 75319us) service_pool.cc:227] Handling call
0925 09:57:26.820258 (+1302048us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":3561,"reported_tablets":31}
W20260925 09:57:26.825482 10392 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.3:41469 (request call id 24) took 1881 ms (client timeout 1000 ms). Trace:
I20260925 09:57:26.826946 11084 tablet_bootstrap.cc:492] T a436fe287a0d48a1821191adcd031d74 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:26.827499 11084 ts_tablet_manager.cc:1397] T a436fe287a0d48a1821191adcd031d74 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.137s user 0.011s sys 0.000s
W20260925 09:57:26.825830 10392 rpcz_store.cc:269] 0925 09:57:24.944494 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:25.206451 (+261957us) service_pool.cc:227] Handling call
0925 09:57:26.825413 (+1618962us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:26.182817 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:26.183236 (+ 419us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:26.183263 (+ 27us) write_op.cc:435] Acquired schema lock
0925 09:57:26.183278 (+ 15us) tablet.cc:660] Decoding operations
0925 09:57:26.183543 (+ 265us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:26.183575 (+ 32us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:26.183595 (+ 20us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:26.183713 (+ 118us) tablet.cc:699] Row locks acquired
0925 09:57:26.183725 (+ 12us) write_op.cc:260] PREPARE: finished
0925 09:57:26.183819 (+ 94us) write_op.cc:270] Start()
0925 09:57:26.183887 (+ 68us) write_op.cc:276] Timestamp: P: 1790330246183768 usec, L: 0
0925 09:57:26.183902 (+ 15us) op_driver.cc:331] REPLICATION: starting
0925 09:57:26.253900 (+ 69998us) spinlock_profiling.cc:243] Waited 26.1 ms on lock 0x7b5400012320. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf18330c4 00007fabf0a9af1b 00007fabf0a9ac6a 00007fabf0b06cd5 00007fabf0b07179 00007fabf0f68ade 00007fabf0f682b8 00007fabf0f6cba1 00007fabf0f6cb59 00007fabf0f6cae9 00007fabf0f6cab1
0925 09:57:26.254325 (+ 425us) log.cc:842] Serialized 356 byte log entry
0925 09:57:26.524738 (+270413us) op_driver.cc:444] REPLICATION: finished
0925 09:57:26.525243 (+ 505us) write_op.cc:301] APPLY: starting
0925 09:57:26.525362 (+ 119us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:26.525532 (+ 170us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:26.525588 (+ 56us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:26.526116 (+ 528us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:26.526497 (+ 381us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:26.526594 (+ 97us) write_op.cc:312] APPLY: finished
0925 09:57:26.526890 (+ 296us) log.cc:842] Serialized 23 byte log entry
0925 09:57:26.527220 (+ 330us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:26.527369 (+ 149us) write_op.cc:454] Released schema lock
0925 09:57:26.531522 (+ 4153us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:26.531927 (+ 405us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:26.531981 (+ 54us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":51405,"reported_tablets":61,"child_traces":[["op",{"apply.queue_time_us":254,"apply.run_cpu_time_us":3035,"apply.run_wall_time_us":7035,"num_ops":1,"prepare.queue_time_us":63513,"prepare.run_cpu_time_us":3807,"prepare.run_wall_time_us":71802,"raft.queue_time_us":439,"raft.run_cpu_time_us":547,"raft.run_wall_time_us":555,"replication_time_us":340765,"spinlock_wait_cycles":60097280}]]}
I20260925 09:57:26.830734 11084 raft_consensus.cc:348] T a436fe287a0d48a1821191adcd031d74 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:26.842813 11901 raft_consensus.cc:482] T 417058493c6547c69e72347cef373292 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:26.843420 11901 raft_consensus.cc:504] T 417058493c6547c69e72347cef373292 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:26.850419 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "417058493c6547c69e72347cef373292" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:26.851679 10918 leader_election.cc:341] T 417058493c6547c69e72347cef373292 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:26.852550 11901 leader_election.cc:288] T 417058493c6547c69e72347cef373292 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:26.857883 11085 tablet_bootstrap.cc:492] T 9e697e7146e34841bd64660b45ebf231 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:26.858279 11085 ts_tablet_manager.cc:1397] T 9e697e7146e34841bd64660b45ebf231 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.863s user 0.020s sys 0.001s
I20260925 09:57:26.859081 10389 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_2d40588792754851bf107d378c5950dd" } new_table_name: "default.Table_01f37e9d4f5b4dc5a66a24eac603810d" modify_external_catalogs: true
I20260925 09:57:26.860476 11085 raft_consensus.cc:348] T 9e697e7146e34841bd64660b45ebf231 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:26.861104 11085 raft_consensus.cc:374] T 9e697e7146e34841bd64660b45ebf231 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:26.861338 11085 raft_consensus.cc:729] T 9e697e7146e34841bd64660b45ebf231 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:26.863564 10391 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_8b45818055ec40e386d43e2641ec6ed0" } modify_external_catalogs: true
I20260925 09:57:26.864039 10391 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_8b45818055ec40e386d43e2641ec6ed0" } modify_external_catalogs: true
I20260925 09:57:26.869021 10392 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_48f368d811494119be0dc534e2fdcfbd" } modify_external_catalogs: true
I20260925 09:57:26.869701 10392 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_48f368d811494119be0dc534e2fdcfbd" } modify_external_catalogs: true
I20260925 09:57:26.870345 11901 raft_consensus.cc:482] T 2358acdaa5104e3eacbab8bf182b8f30 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:26.867866 10384 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_a5e309b8633e4779a51120c48f91df43"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:26.870774 11901 raft_consensus.cc:504] T 2358acdaa5104e3eacbab8bf182b8f30 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:26.873176 11901 leader_election.cc:288] T 2358acdaa5104e3eacbab8bf182b8f30 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:26.875471 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2358acdaa5104e3eacbab8bf182b8f30" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:26.876876 10918 leader_election.cc:341] T 2358acdaa5104e3eacbab8bf182b8f30 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:26.872546 10384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a5e309b8633e4779a51120c48f91df43 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 09:57:26.876840 11980 log.cc:924] Time spent T ef22f64036684dce9d7c69c37722f468 P be97ab5d75934a648d4ab76341193ec0: Append to log took a long time: real 0.066s user 0.000s sys 0.000s
I20260925 09:57:26.861883 11085 consensus_queue.cc:260] T 9e697e7146e34841bd64660b45ebf231 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:26.794502 10666 tablet_service.cc:1514] Processing DeleteTablet for tablet 359090244c4748e8a2d45e3c77e89498 with delete_type TABLET_DATA_DELETED (replaced by tablet b39f9e7102ff4a9c96a2cc5d2676f03d) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:26.790552 10657 tablet_service.cc:1514] Processing DeleteTablet for tablet 3039ec3febf848dba7c4e6fd38bb8053 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:26 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:26.832717 11919 raft_consensus.cc:2781] T aba253f22afa4200926f31a1b43e3970 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:26.886224 11709 tablet_replica.cc:330] T 3be0eccc65564bfba3d6e5c398c61fb6 P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:26.849798 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e684798155d445bfb325a1b7c5c48753" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:26.852610 10667 tablet_service.cc:1514] Processing DeleteTablet for tablet d9e454f82a4e4c429b58f32926319b7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:22 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:26.853116 10658 tablet_service.cc:1514] Processing DeleteTablet for tablet 69028aa7908c45b896da4ba8deef07b1 with delete_type TABLET_DATA_DELETED (replaced by tablet 2015c10f503e4751ad9ff77404fca50b) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:26.887245 11950 tablet.cc:1722] T ef22f64036684dce9d7c69c37722f468 P c43461c4ee854ae1ac52b873ccada318: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:26.851996 10655 tablet_service.cc:1514] Processing DeleteTablet for tablet 67ce2e3325214aa6bdc3347e9e8f82b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:23 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:26.889525 11959 tablet.cc:1722] T ef22f64036684dce9d7c69c37722f468 P be97ab5d75934a648d4ab76341193ec0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:26.893769 11085 ts_tablet_manager.cc:1428] T 9e697e7146e34841bd64660b45ebf231 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.035s user 0.005s sys 0.000s
I20260925 09:57:26.894606 11085 tablet_bootstrap.cc:492] T df006163f72743b594baf1a3efd1c771 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:26.895443 11709 raft_consensus.cc:2229] T 3be0eccc65564bfba3d6e5c398c61fb6 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:26.848531 10683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10657a81058241fc988d37e5ea7ff8cf" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:26.829428 11942 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:37876 (request call id 222) took 1621 ms. Trace:
I20260925 09:57:26.900806 11942 rpcz_store.cc:276] 0925 09:57:25.203427 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:25.213456 (+ 10029us) service_pool.cc:227] Handling call
0925 09:57:26.824752 (+1611296us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:25.467367 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0925 09:57:25.656580 (+189213us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0925 09:57:25.657760 (+ 1180us) alter_schema_op.cc:65] Acquired schema lock
0925 09:57:25.676329 (+ 18569us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0925 09:57:25.676509 (+ 180us) alter_schema_op.cc:126] START. Timestamp: 1790330245676532
0925 09:57:25.676538 (+ 29us) op_driver.cc:331] REPLICATION: starting
0925 09:57:25.677262 (+ 724us) log.cc:842] Serialized 173 byte log entry
0925 09:57:25.920103 (+242841us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b54000cd3a0. stack: 00007fb965334033 00007fb965333f30 00007fb964eb5534 00000000004db2b4 00000000004db249 00007fb96c240af0 00007fb96c24b028 00007fb96c24ac6a 00007fb96c2b6cd5 00007fb96c2b7179 00007fb96c718ade 00007fb96c7182b8 00007fb96c71cba1 00007fb96c71cb59 00007fb96c71cae9 00007fb96c71cab1
0925 09:57:26.192072 (+271969us) op_driver.cc:444] REPLICATION: finished
0925 09:57:26.207046 (+ 14974us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0925 09:57:26.229685 (+ 22639us) log.cc:842] Serialized 5 byte log entry
0925 09:57:26.611447 (+381762us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:26.794150 (+182703us) log.cc:842] Serialized 14 byte log entry
0925 09:57:26.816292 (+ 22142us) alter_schema_op.cc:71] Released schema lock
0925 09:57:26.817368 (+ 1076us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":14739,"prepare.queue_time_us":252967,"prepare.run_cpu_time_us":6365,"prepare.run_wall_time_us":479134,"raft.queue_time_us":46972,"raft.run_cpu_time_us":2452,"raft.run_wall_time_us":2452,"replication_time_us":515429,"spinlock_wait_cycles":254956928,"thread_start_us":344447,"threads_started":2,"wal-append.queue_time_us":119474}]]}
I20260925 09:57:26.907522 11919 raft_consensus.cc:686] T aba253f22afa4200926f31a1b43e3970 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:26.908383 11919 consensus_queue.cc:237] T aba253f22afa4200926f31a1b43e3970 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:26.917135 11709 raft_consensus.cc:2258] T 3be0eccc65564bfba3d6e5c398c61fb6 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:26.924865 11709 ts_tablet_manager.cc:1905] T 3be0eccc65564bfba3d6e5c398c61fb6 P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 09:57:26.925276 10619 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 213041us
I20260925 09:57:26.831800 11084 raft_consensus.cc:374] T a436fe287a0d48a1821191adcd031d74 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:26.926852 11084 raft_consensus.cc:729] T a436fe287a0d48a1821191adcd031d74 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:26.927614 11084 consensus_queue.cc:260] T a436fe287a0d48a1821191adcd031d74 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:26.851434 10678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "417058493c6547c69e72347cef373292" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:26.885987 11082 tablet_bootstrap.cc:654] T 3b9cdad4f9144daea800553abd378e0b P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:26.932289 10390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_6248142ab04c4496a3db73e70549b09b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:26.933744 10390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6248142ab04c4496a3db73e70549b09b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:26.944751 11579 ts_tablet_manager.cc:1918] T 3be0eccc65564bfba3d6e5c398c61fb6 P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:26.945107 11579 log.cc:1196] T 3be0eccc65564bfba3d6e5c398c61fb6 P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/3be0eccc65564bfba3d6e5c398c61fb6
W20260925 09:57:26.949035 10919 leader_election.cc:341] T e684798155d445bfb325a1b7c5c48753 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:26.949133 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 3706253e53d841aeaa43889ead13af32 because the tablet was not found. No further retry: Not found: Tablet not found: 3706253e53d841aeaa43889ead13af32
I20260925 09:57:26.949410 10919 leader_election.cc:302] T e684798155d445bfb325a1b7c5c48753 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:26.950408 11901 raft_consensus.cc:2726] T e684798155d445bfb325a1b7c5c48753 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:26.951333 10919 leader_election.cc:341] T 10657a81058241fc988d37e5ea7ff8cf P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:26.951687 10919 leader_election.cc:302] T 10657a81058241fc988d37e5ea7ff8cf P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:26.952615 11901 raft_consensus.cc:2726] T 10657a81058241fc988d37e5ea7ff8cf P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:26.954588 11942 tablet.cc:1722] T ef22f64036684dce9d7c69c37722f468 P b8850818e2ee42a082802d0095c08d21: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:26.955864 10688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2358acdaa5104e3eacbab8bf182b8f30" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:26.956292 10919 leader_election.cc:341] T 417058493c6547c69e72347cef373292 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:26.957620 10919 leader_election.cc:302] T 417058493c6547c69e72347cef373292 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
W20260925 09:57:26.960425 10919 leader_election.cc:341] T 2358acdaa5104e3eacbab8bf182b8f30 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:26.960902 10919 leader_election.cc:302] T 2358acdaa5104e3eacbab8bf182b8f30 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:26.961638 11892 raft_consensus.cc:2726] T 417058493c6547c69e72347cef373292 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:26.962230 11892 raft_consensus.cc:2726] T 2358acdaa5104e3eacbab8bf182b8f30 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:26.966264 11084 ts_tablet_manager.cc:1428] T a436fe287a0d48a1821191adcd031d74 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.138s user 0.006s sys 0.000s
W20260925 09:57:26.983783 10387 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b3d9a3cf5bda4cf6bc1fa3fae1cb9288 consensus state has no leader
I20260925 09:57:26.988507 11901 raft_consensus.cc:482] T defaef4d6b754ca5a2dfc21b7a596a5b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:26.989256 11727 ts_tablet_manager.cc:1918] T 977a59175e5b43eeb6f23b938afd00a3 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:26.989118 11901 raft_consensus.cc:504] T defaef4d6b754ca5a2dfc21b7a596a5b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:26.989638 11727 log.cc:1196] T 977a59175e5b43eeb6f23b938afd00a3 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/977a59175e5b43eeb6f23b938afd00a3
I20260925 09:57:26.990285 11727 ts_tablet_manager.cc:1939] T 977a59175e5b43eeb6f23b938afd00a3 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:26.990854 11901 leader_election.cc:288] T defaef4d6b754ca5a2dfc21b7a596a5b P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:26.992004 11901 raft_consensus.cc:482] T 3706253e53d841aeaa43889ead13af32 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b8850818e2ee42a082802d0095c08d21)
I20260925 09:57:26.992518 11901 raft_consensus.cc:504] T 3706253e53d841aeaa43889ead13af32 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:26.994717 11901 leader_election.cc:288] T 3706253e53d841aeaa43889ead13af32 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
W20260925 09:57:26.997063 10919 leader_election.cc:341] T 3706253e53d841aeaa43889ead13af32 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Not found: Tablet not found: 3706253e53d841aeaa43889ead13af32
I20260925 09:57:26.955487 11954 consensus_queue.cc:1035] T c2110d4ada2d45c2b07259374465a8df P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.043s
W20260925 09:57:26.998660 10918 leader_election.cc:341] T 3706253e53d841aeaa43889ead13af32 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Not found: Tablet not found: 3706253e53d841aeaa43889ead13af32
I20260925 09:57:26.993134 10834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "defaef4d6b754ca5a2dfc21b7a596a5b" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:26.945781 11579 ts_tablet_manager.cc:1939] T 3be0eccc65564bfba3d6e5c398c61fb6 P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
W20260925 09:57:27.001048 10775 consensus_peers.cc:487] T c2110d4ada2d45c2b07259374465a8df P c43461c4ee854ae1ac52b873ccada318 -> Peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Couldn't send request to peer b8850818e2ee42a082802d0095c08d21. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:27.001221 10918 leader_election.cc:302] T 3706253e53d841aeaa43889ead13af32 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
W20260925 09:57:27.002146 10918 leader_election.cc:341] T defaef4d6b754ca5a2dfc21b7a596a5b P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:27.006254 11892 raft_consensus.cc:2726] T 3706253e53d841aeaa43889ead13af32 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 09:57:27.003167 10683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "defaef4d6b754ca5a2dfc21b7a596a5b" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:27.008606 10919 leader_election.cc:341] T defaef4d6b754ca5a2dfc21b7a596a5b P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:27.008980 10919 leader_election.cc:302] T defaef4d6b754ca5a2dfc21b7a596a5b P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:27.009896 11892 raft_consensus.cc:2726] T defaef4d6b754ca5a2dfc21b7a596a5b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:26.996760 11085 tablet_bootstrap.cc:654] T df006163f72743b594baf1a3efd1c771 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:27.015172 11579 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 230) took 1458 ms. Trace:
I20260925 09:57:27.017165 11082 tablet_bootstrap.cc:492] T 3b9cdad4f9144daea800553abd378e0b P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:27.017599 11082 ts_tablet_manager.cc:1397] T 3b9cdad4f9144daea800553abd378e0b P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.447s user 0.009s sys 0.004s
I20260925 09:57:27.019924 11082 raft_consensus.cc:348] T 3b9cdad4f9144daea800553abd378e0b P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.032392 11082 raft_consensus.cc:374] T 3b9cdad4f9144daea800553abd378e0b P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:27.032724 11082 raft_consensus.cc:729] T 3b9cdad4f9144daea800553abd378e0b P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:27.033383 11082 consensus_queue.cc:260] T 3b9cdad4f9144daea800553abd378e0b P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.022168 11579 rpcz_store.cc:276] 0925 09:57:25.556382 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:25.558576 (+ 2194us) service_pool.cc:227] Handling call
0925 09:57:26.747499 (+1188923us) ts_tablet_manager.cc:1145] Deleting tablet 3be0eccc65564bfba3d6e5c398c61fb6
0925 09:57:26.747532 (+ 33us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:26.774305 (+ 26773us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:26.944515 (+170210us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:27.015116 (+ 70601us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1187849}
I20260925 09:57:27.036702 11084 tablet_bootstrap.cc:492] T 96a80382bb7d4395a14ad9ae972f4a25 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:27.037473 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 3be0eccc65564bfba3d6e5c398c61fb6 (table default.Table_0af2360d761342cf9f057be5f265aa41 [id=99ba81c4d0dd497398e048a05759419a]) successfully deleted
I20260925 09:57:27.038605 11579 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 233) took 1310 ms. Trace:
I20260925 09:57:27.039038 11082 ts_tablet_manager.cc:1428] T 3b9cdad4f9144daea800553abd378e0b P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.021s user 0.006s sys 0.001s
I20260925 09:57:27.042831 11084 tablet_bootstrap.cc:654] T 96a80382bb7d4395a14ad9ae972f4a25 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:27.038959 11579 rpcz_store.cc:276] 0925 09:57:25.728029 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:25.728725 (+ 696us) service_pool.cc:227] Handling call
0925 09:57:27.038249 (+1309524us) ts_tablet_manager.cc:1145] Deleting tablet 4e3120347c8e4a3eab57d6562ae40e8d
0925 09:57:27.038316 (+ 67us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:27.038554 (+ 238us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1308998}
W20260925 09:57:27.043926 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 4e3120347c8e4a3eab57d6562ae40e8d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4e3120347c8e4a3eab57d6562ae40e8d already in progress: creating tablet
I20260925 09:57:27.045675 11082 tablet_bootstrap.cc:492] T df006163f72743b594baf1a3efd1c771 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:27.049551 11579 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 234) took 1308 ms. Trace:
I20260925 09:57:27.050802 11579 rpcz_store.cc:276] 0925 09:57:25.739744 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:25.739870 (+ 126us) service_pool.cc:227] Handling call
0925 09:57:27.044671 (+1304801us) ts_tablet_manager.cc:1145] Deleting tablet 10657a81058241fc988d37e5ea7ff8cf
0925 09:57:27.044709 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:27.048453 (+ 3744us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1304146}
W20260925 09:57:27.056430 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 10657a81058241fc988d37e5ea7ff8cf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 10657a81058241fc988d37e5ea7ff8cf already in progress: creating tablet
I20260925 09:57:27.055574 11085 tablet_bootstrap.cc:492] T df006163f72743b594baf1a3efd1c771 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:27.057418 11085 ts_tablet_manager.cc:1397] T df006163f72743b594baf1a3efd1c771 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.163s user 0.009s sys 0.000s
I20260925 09:57:27.058262 11082 tablet_bootstrap.cc:654] T df006163f72743b594baf1a3efd1c771 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:27.061160 10378 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet b3d9a3cf5bda4cf6bc1fa3fae1cb9288 (table default.Table_cc9f467e078c448f87392bf324b040a8 [id=4cefdaddfd6143468df4502cd35afac4], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet b3d9a3cf5bda4cf6bc1fa3fae1cb9288 consensus state has no leader
I20260925 09:57:27.070135 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 201) took 4317 ms. Trace:
I20260925 09:57:27.070116 10387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_13782377553b4270abed8a7beae0a959"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:27.070472 11727 rpcz_store.cc:276] 0925 09:57:22.752663 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:22.753304 (+ 641us) service_pool.cc:227] Handling call
0925 09:57:26.747322 (+3994018us) ts_tablet_manager.cc:1145] Deleting tablet 977a59175e5b43eeb6f23b938afd00a3
0925 09:57:26.753885 (+ 6563us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:26.907164 (+153279us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:26.989008 (+ 81844us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:27.070061 (+ 81053us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":49160,"tablet-delete.queue_time_us":3993347}
I20260925 09:57:27.071591 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 977a59175e5b43eeb6f23b938afd00a3 (table default.Table_ff4e7021d36b43f8b51f19960e707ac8 [id=f17a11d223204599afb24e2e9d1d9d9d]) successfully deleted
W20260925 09:57:27.071648 10387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_13782377553b4270abed8a7beae0a959 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:27.071949 11727 tablet_replica.cc:330] T 7a31b999f0fd447eacff13de71aa3fff P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
W20260925 09:57:27.074235 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 417058493c6547c69e72347cef373292 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 417058493c6547c69e72347cef373292 already in progress: creating tablet
W20260925 09:57:27.078117 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet cf775c9d9b114ddcb97dc6c6eca8e0fc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cf775c9d9b114ddcb97dc6c6eca8e0fc already in progress: creating tablet
I20260925 09:57:27.081985 11084 tablet_bootstrap.cc:492] T 96a80382bb7d4395a14ad9ae972f4a25 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:27.082407 11084 ts_tablet_manager.cc:1397] T 96a80382bb7d4395a14ad9ae972f4a25 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.046s user 0.009s sys 0.000s
I20260925 09:57:27.074952 11085 raft_consensus.cc:348] T df006163f72743b594baf1a3efd1c771 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.085887 11085 raft_consensus.cc:374] T df006163f72743b594baf1a3efd1c771 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:27.086138 11085 raft_consensus.cc:729] T df006163f72743b594baf1a3efd1c771 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
W20260925 09:57:27.087582 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 3039ec3febf848dba7c4e6fd38bb8053 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3039ec3febf848dba7c4e6fd38bb8053 already in progress: creating tablet
I20260925 09:57:27.088024 11727 raft_consensus.cc:2229] T 7a31b999f0fd447eacff13de71aa3fff P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:27.088483 11727 raft_consensus.cc:2258] T 7a31b999f0fd447eacff13de71aa3fff P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:27.090256 11727 ts_tablet_manager.cc:1905] T 7a31b999f0fd447eacff13de71aa3fff P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:27.094167 11084 raft_consensus.cc:348] T 96a80382bb7d4395a14ad9ae972f4a25 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.094893 11084 raft_consensus.cc:374] T 96a80382bb7d4395a14ad9ae972f4a25 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:27.095146 11084 raft_consensus.cc:729] T 96a80382bb7d4395a14ad9ae972f4a25 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:27.095923 10393 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet a81530200cf14717bc7cbf67bb73f4e5
I20260925 09:57:27.097150 10393 catalog_manager.cc:6565] ReplaceTablet: tablet a81530200cf14717bc7cbf67bb73f4e5 deleted and replaced by tablet e9faf70be8ac43c79305980d685a28eb
I20260925 09:57:27.098784 10955 tablet_service.cc:1514] Processing DeleteTablet for tablet a81530200cf14717bc7cbf67bb73f4e5 with delete_type TABLET_DATA_DELETED (replaced by tablet e9faf70be8ac43c79305980d685a28eb) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:27.100809 10816 tablet_service.cc:1514] Processing DeleteTablet for tablet a81530200cf14717bc7cbf67bb73f4e5 with delete_type TABLET_DATA_DELETED (replaced by tablet e9faf70be8ac43c79305980d685a28eb) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:27.102560 10663 tablet_service.cc:1514] Processing DeleteTablet for tablet a81530200cf14717bc7cbf67bb73f4e5 with delete_type TABLET_DATA_DELETED (replaced by tablet e9faf70be8ac43c79305980d685a28eb) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:27.103888 10393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_40181c2d583b42b89dbb7e1f589f5317"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:27.101004 11084 consensus_queue.cc:260] T 96a80382bb7d4395a14ad9ae972f4a25 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:27.106015 10393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_40181c2d583b42b89dbb7e1f589f5317 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 09:57:27.113131 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 9a73a1f242e7430cbf52229c07b4dc82 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9a73a1f242e7430cbf52229c07b4dc82 already in progress: creating tablet
W20260925 09:57:27.119102 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet cf35f39bcd6a41128acb2a4936fcc316 because the tablet was not found. No further retry: Not found: Tablet not found: cf35f39bcd6a41128acb2a4936fcc316
W20260925 09:57:27.122915 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 69028aa7908c45b896da4ba8deef07b1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 69028aa7908c45b896da4ba8deef07b1 already in progress: creating tablet
I20260925 09:57:27.116497 11085 consensus_queue.cc:260] T df006163f72743b594baf1a3efd1c771 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.123816 11961 raft_consensus.cc:482] T df006163f72743b594baf1a3efd1c771 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:27.124215 11961 raft_consensus.cc:504] T df006163f72743b594baf1a3efd1c771 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.127913 11961 leader_election.cc:288] T df006163f72743b594baf1a3efd1c771 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:27.129120 10981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df006163f72743b594baf1a3efd1c771" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:27.129878 10981 raft_consensus.cc:2454] T df006163f72743b594baf1a3efd1c771 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
I20260925 09:57:27.131251 10772 leader_election.cc:302] T df006163f72743b594baf1a3efd1c771 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:27.132301 11961 raft_consensus.cc:2781] T df006163f72743b594baf1a3efd1c771 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:27.132773 11961 raft_consensus.cc:482] T df006163f72743b594baf1a3efd1c771 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:27.133185 11961 raft_consensus.cc:3037] T df006163f72743b594baf1a3efd1c771 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:27.134043 11579 tablet_replica.cc:330] T ec1b78e61e8d46cb8b87f60786059658 P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:27.136111 11579 raft_consensus.cc:2229] T ec1b78e61e8d46cb8b87f60786059658 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:27.142031 10390 master_service.cc:946] ReplaceTablet: received request to replace tablet aba253f22afa4200926f31a1b43e3970 from {username='slave'} at 127.0.0.1:35140
I20260925 09:57:27.145351 11579 raft_consensus.cc:2258] T ec1b78e61e8d46cb8b87f60786059658 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:27.150033 11579 ts_tablet_manager.cc:1905] T ec1b78e61e8d46cb8b87f60786059658 P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:27.119242 11084 ts_tablet_manager.cc:1428] T 96a80382bb7d4395a14ad9ae972f4a25 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.037s user 0.007s sys 0.002s
I20260925 09:57:27.170910 11084 tablet_bootstrap.cc:492] T ab6bdba5ca5d41ae921a90c1f77006c5 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:27.171722 10386 catalog_manager.cc:5495] T c2110d4ada2d45c2b07259374465a8df P c43461c4ee854ae1ac52b873ccada318 reported cstate change: term changed from 0 to 1, leader changed from <none> to c43461c4ee854ae1ac52b873ccada318 (127.10.9.2). New cstate: current_term: 1 leader_uuid: "c43461c4ee854ae1ac52b873ccada318" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:27.173077 10386 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet c2110d4ada2d45c2b07259374465a8df (table default.Table_44ddd064d56640a0a0e3dd8fd38344fd [id=8307778a24ee442b8f7e89c09652309e]). Expected version 1 got 0
I20260925 09:57:27.174652 10386 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 26fae3fbfe374f348a1729c21ec751b9 (table default.Table_0011387b45014de4b24b466f4cb907f7 [id=e3f7aa5e988c4b6c98be5cd3d465245d]). Expected version 1 got 0
I20260925 09:57:27.175596 10386 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet ef22f64036684dce9d7c69c37722f468 (table default.Table_0d7d27d05491401089db5f65b43e86f7 [id=a20216b9b3074e408bee74e39c5819b4]). Expected version 1 got 0
I20260925 09:57:27.176236 10386 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 3d652a76d96f4dbeafde15f3c16534a6 (table default.Table_be0f658aa41f4188bc16db37e96f4ad1 [id=73083c1560cb4db5a12964387cb49b73]). Expected version 1 got 0
I20260925 09:57:27.180605 10683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df006163f72743b594baf1a3efd1c771" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:27.182376 10775 leader_election.cc:341] T df006163f72743b594baf1a3efd1c771 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:27.185627 11919 raft_consensus.cc:482] T 2d087d3e35814c90a03eccd90eeea86c P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:27.186190 11919 raft_consensus.cc:504] T 2d087d3e35814c90a03eccd90eeea86c P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.187191 10384 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_1b4a040b8d60467a869df4b51c09cf46" } new_table_name: "default.Table_f37bcee5db2044f2bea9c353982b30cd" modify_external_catalogs: true
I20260925 09:57:27.189059 11919 leader_election.cc:288] T 2d087d3e35814c90a03eccd90eeea86c P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:27.189939 10981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d087d3e35814c90a03eccd90eeea86c" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:27.190539 10981 raft_consensus.cc:2454] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:27.190444 10834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d087d3e35814c90a03eccd90eeea86c" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:27.191653 10619 leader_election.cc:302] T 2d087d3e35814c90a03eccd90eeea86c P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
W20260925 09:57:27.192504 10621 leader_election.cc:341] T 2d087d3e35814c90a03eccd90eeea86c P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:27.197400 11082 tablet_bootstrap.cc:492] T df006163f72743b594baf1a3efd1c771 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:27.197850 11082 ts_tablet_manager.cc:1397] T df006163f72743b594baf1a3efd1c771 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.152s user 0.009s sys 0.018s
I20260925 09:57:27.200212 11919 raft_consensus.cc:2781] T 2d087d3e35814c90a03eccd90eeea86c P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:27.200536 11919 raft_consensus.cc:482] T 2d087d3e35814c90a03eccd90eeea86c P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:27.200826 11919 raft_consensus.cc:3037] T 2d087d3e35814c90a03eccd90eeea86c P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:27.201402 11082 raft_consensus.cc:348] T df006163f72743b594baf1a3efd1c771 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.201854 11981 raft_consensus.cc:482] T 3b9cdad4f9144daea800553abd378e0b P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:27.202086 11082 raft_consensus.cc:374] T df006163f72743b594baf1a3efd1c771 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:27.202378 11082 raft_consensus.cc:729] T df006163f72743b594baf1a3efd1c771 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:27.202250 11981 raft_consensus.cc:504] T 3b9cdad4f9144daea800553abd378e0b P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.204087 11981 leader_election.cc:288] T 3b9cdad4f9144daea800553abd378e0b P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:27.205660 10981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b9cdad4f9144daea800553abd378e0b" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:27.206208 10981 raft_consensus.cc:2454] T 3b9cdad4f9144daea800553abd378e0b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:27.166923 11961 raft_consensus.cc:504] T df006163f72743b594baf1a3efd1c771 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.207602 10619 leader_election.cc:302] T 3b9cdad4f9144daea800553abd378e0b P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:27.210361 11981 raft_consensus.cc:2781] T 3b9cdad4f9144daea800553abd378e0b P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:27.210119 11082 consensus_queue.cc:260] T df006163f72743b594baf1a3efd1c771 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.210659 11981 raft_consensus.cc:482] T 3b9cdad4f9144daea800553abd378e0b P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:27.212261 11981 raft_consensus.cc:3037] T 3b9cdad4f9144daea800553abd378e0b P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:27.214067 11082 ts_tablet_manager.cc:1428] T df006163f72743b594baf1a3efd1c771 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.016s user 0.005s sys 0.000s
I20260925 09:57:27.214989 11082 tablet_bootstrap.cc:492] T 3ff4cf556eee494d8cd590884466f09f P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:27.216912 11085 ts_tablet_manager.cc:1428] T df006163f72743b594baf1a3efd1c771 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.159s user 0.007s sys 0.001s
I20260925 09:57:27.217895 11085 tablet_bootstrap.cc:492] T aba253f22afa4200926f31a1b43e3970 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:27.218806 10834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b9cdad4f9144daea800553abd378e0b" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:27.222234 10621 leader_election.cc:341] T 3b9cdad4f9144daea800553abd378e0b P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:27.235203 11082 tablet_bootstrap.cc:654] T 3ff4cf556eee494d8cd590884466f09f P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:27.236701 11084 tablet_bootstrap.cc:654] T ab6bdba5ca5d41ae921a90c1f77006c5 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:27.237762 10390 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet aba253f22afa4200926f31a1b43e3970
I20260925 09:57:27.238979 10390 catalog_manager.cc:6565] ReplaceTablet: tablet aba253f22afa4200926f31a1b43e3970 deleted and replaced by tablet 815b4b1bd914433798dcb5deec6ec79c
I20260925 09:57:27.240527 10663 tablet_service.cc:1514] Processing DeleteTablet for tablet aba253f22afa4200926f31a1b43e3970 with delete_type TABLET_DATA_DELETED (replaced by tablet 815b4b1bd914433798dcb5deec6ec79c) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:27.241448 11919 raft_consensus.cc:504] T 2d087d3e35814c90a03eccd90eeea86c P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.242692 10955 tablet_service.cc:1514] Processing DeleteTablet for tablet aba253f22afa4200926f31a1b43e3970 with delete_type TABLET_DATA_DELETED (replaced by tablet 815b4b1bd914433798dcb5deec6ec79c) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:27.244850 11981 raft_consensus.cc:504] T 3b9cdad4f9144daea800553abd378e0b P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.245705 10816 tablet_service.cc:1514] Processing DeleteTablet for tablet aba253f22afa4200926f31a1b43e3970 with delete_type TABLET_DATA_DELETED (replaced by tablet 815b4b1bd914433798dcb5deec6ec79c) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:27.245959 10981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df006163f72743b594baf1a3efd1c771" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:27.246547 10981 raft_consensus.cc:3037] T df006163f72743b594baf1a3efd1c771 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:27.249647 10834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d087d3e35814c90a03eccd90eeea86c" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
W20260925 09:57:27.251780 10621 leader_election.cc:341] T 2d087d3e35814c90a03eccd90eeea86c P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:27.252440 10683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df006163f72743b594baf1a3efd1c771" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:27.252970 10683 raft_consensus.cc:3037] T df006163f72743b594baf1a3efd1c771 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:27.254482 10981 raft_consensus.cc:2454] T df006163f72743b594baf1a3efd1c771 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:27.255532 10772 leader_election.cc:302] T df006163f72743b594baf1a3efd1c771 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:27.256438 11954 raft_consensus.cc:2781] T df006163f72743b594baf1a3efd1c771 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:27.256924 11954 raft_consensus.cc:686] T df006163f72743b594baf1a3efd1c771 P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Becoming Leader. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Running, Role: LEADER
I20260925 09:57:27.257781 11954 consensus_queue.cc:237] T df006163f72743b594baf1a3efd1c771 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.258719 11919 leader_election.cc:288] T 2d087d3e35814c90a03eccd90eeea86c P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:27.259769 10982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d087d3e35814c90a03eccd90eeea86c" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:27.260326 10982 raft_consensus.cc:3037] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:27.263019 11780 raft_consensus.cc:482] T 9e697e7146e34841bd64660b45ebf231 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:27.263514 11780 raft_consensus.cc:504] T 9e697e7146e34841bd64660b45ebf231 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.265573 11780 leader_election.cc:288] T 9e697e7146e34841bd64660b45ebf231 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:27.267211 10983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e697e7146e34841bd64660b45ebf231" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:27.268812 11961 leader_election.cc:288] T df006163f72743b594baf1a3efd1c771 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:27.271759 10982 raft_consensus.cc:2454] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:27.279013 10678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e697e7146e34841bd64660b45ebf231" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:27.282686 10683 raft_consensus.cc:2454] T df006163f72743b594baf1a3efd1c771 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:27.283638 10775 leader_election.cc:302] T 9e697e7146e34841bd64660b45ebf231 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:27.287012 11961 raft_consensus.cc:2726] T 9e697e7146e34841bd64660b45ebf231 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:27.288622 10619 leader_election.cc:302] T 2d087d3e35814c90a03eccd90eeea86c P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:27.289518 11977 raft_consensus.cc:2781] T 2d087d3e35814c90a03eccd90eeea86c P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:27.289917 11977 raft_consensus.cc:686] T 2d087d3e35814c90a03eccd90eeea86c P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:27.290607 11977 consensus_queue.cc:237] T 2d087d3e35814c90a03eccd90eeea86c P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.295015 11085 tablet_bootstrap.cc:654] T aba253f22afa4200926f31a1b43e3970 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:27.304442 10387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_8f6c92e47621419db4bc9e3e2dd5ce12"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:27.305981 10387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8f6c92e47621419db4bc9e3e2dd5ce12 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:27.311182 10390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_7c70dabea5b347c8b156b4b1c72703eb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:27.312613 10390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7c70dabea5b347c8b156b4b1c72703eb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:27.313913 11084 tablet_bootstrap.cc:492] T ab6bdba5ca5d41ae921a90c1f77006c5 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:27.314347 11084 ts_tablet_manager.cc:1397] T ab6bdba5ca5d41ae921a90c1f77006c5 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.144s user 0.008s sys 0.000s
I20260925 09:57:27.316852 11084 raft_consensus.cc:348] T ab6bdba5ca5d41ae921a90c1f77006c5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.317577 11084 raft_consensus.cc:374] T ab6bdba5ca5d41ae921a90c1f77006c5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:27.317844 11084 raft_consensus.cc:729] T ab6bdba5ca5d41ae921a90c1f77006c5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:27.318442 11901 raft_consensus.cc:482] T d5956be0da6441c79723e72f12dfe54a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:27.319007 11901 raft_consensus.cc:504] T d5956be0da6441c79723e72f12dfe54a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.321386 11901 leader_election.cc:288] T d5956be0da6441c79723e72f12dfe54a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:27.324198 10834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5956be0da6441c79723e72f12dfe54a" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:27.318442 11084 consensus_queue.cc:260] T ab6bdba5ca5d41ae921a90c1f77006c5 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:27.325672 10918 leader_election.cc:341] T d5956be0da6441c79723e72f12dfe54a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:27.328415 11084 ts_tablet_manager.cc:1428] T ab6bdba5ca5d41ae921a90c1f77006c5 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.014s user 0.006s sys 0.001s
I20260925 09:57:27.329071 11981 leader_election.cc:288] T 3b9cdad4f9144daea800553abd378e0b P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:27.329258 11084 tablet_bootstrap.cc:492] T 350904aa5957496d90a2aa2e637b591c P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:27.331779 10834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b9cdad4f9144daea800553abd378e0b" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
W20260925 09:57:27.335130 10621 leader_election.cc:341] T 3b9cdad4f9144daea800553abd378e0b P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:27.335777 10981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b9cdad4f9144daea800553abd378e0b" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:27.336426 10981 raft_consensus.cc:3037] T 3b9cdad4f9144daea800553abd378e0b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:27.336136 10683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5956be0da6441c79723e72f12dfe54a" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:27.337520 11579 ts_tablet_manager.cc:1918] T ec1b78e61e8d46cb8b87f60786059658 P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:27.337847 11579 log.cc:1196] T ec1b78e61e8d46cb8b87f60786059658 P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/ec1b78e61e8d46cb8b87f60786059658
W20260925 09:57:27.337785 10919 leader_election.cc:341] T d5956be0da6441c79723e72f12dfe54a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:27.338155 10919 leader_election.cc:302] T d5956be0da6441c79723e72f12dfe54a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:27.338419 11579 ts_tablet_manager.cc:1939] T ec1b78e61e8d46cb8b87f60786059658 P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:27.339444 11901 raft_consensus.cc:2726] T d5956be0da6441c79723e72f12dfe54a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:27.344626 11901 raft_consensus.cc:482] T 2f07dc9548364870923165f4ea524310 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:27.345026 11901 raft_consensus.cc:504] T 2f07dc9548364870923165f4ea524310 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.346851 11901 leader_election.cc:288] T 2f07dc9548364870923165f4ea524310 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:27.348658 11709 ts_tablet_manager.cc:1918] T 3be0eccc65564bfba3d6e5c398c61fb6 P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:27.349054 11709 log.cc:1196] T 3be0eccc65564bfba3d6e5c398c61fb6 P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/3be0eccc65564bfba3d6e5c398c61fb6
I20260925 09:57:27.349146 10834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f07dc9548364870923165f4ea524310" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:27.349936 11709 ts_tablet_manager.cc:1939] T 3be0eccc65564bfba3d6e5c398c61fb6 P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
W20260925 09:57:27.350549 10918 leader_election.cc:341] T 2f07dc9548364870923165f4ea524310 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:27.350632 11709 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 231) took 1764 ms. Trace:
I20260925 09:57:27.364189 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet ec1b78e61e8d46cb8b87f60786059658 (table default.Table_09fe8f366734488c93a79eeeec2c96f9 [id=a7023f3c32884ce8a7ca21fc95afc721]) successfully deleted
W20260925 09:57:27.365378 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 28d60531970a4ab58ce5ef99cb3a96bf because the tablet was not found. No further retry: Not found: Tablet not found: 28d60531970a4ab58ce5ef99cb3a96bf
W20260925 09:57:27.367013 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 67ce2e3325214aa6bdc3347e9e8f82b0 because the tablet was not found. No further retry: Not found: Tablet not found: 67ce2e3325214aa6bdc3347e9e8f82b0
W20260925 09:57:27.369349 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet a81530200cf14717bc7cbf67bb73f4e5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a81530200cf14717bc7cbf67bb73f4e5 already in progress: creating tablet
W20260925 09:57:27.370785 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet aba253f22afa4200926f31a1b43e3970 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aba253f22afa4200926f31a1b43e3970 already in progress: creating tablet
I20260925 09:57:27.372465 11727 ts_tablet_manager.cc:1918] T 7a31b999f0fd447eacff13de71aa3fff P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:27.372854 11727 log.cc:1196] T 7a31b999f0fd447eacff13de71aa3fff P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/7a31b999f0fd447eacff13de71aa3fff
I20260925 09:57:27.373497 11727 ts_tablet_manager.cc:1939] T 7a31b999f0fd447eacff13de71aa3fff P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:27.378973 11084 tablet_bootstrap.cc:654] T 350904aa5957496d90a2aa2e637b591c P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:27.394064 11084 tablet_bootstrap.cc:492] T 350904aa5957496d90a2aa2e637b591c P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:27.394490 11084 ts_tablet_manager.cc:1397] T 350904aa5957496d90a2aa2e637b591c P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.065s user 0.008s sys 0.000s
I20260925 09:57:27.402519 11084 raft_consensus.cc:348] T 350904aa5957496d90a2aa2e637b591c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.406044 10390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_548921f3f0fd49069b0c73f688f4fa36"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:27.411813 11085 tablet_bootstrap.cc:492] T aba253f22afa4200926f31a1b43e3970 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:27.412365 10393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_b5a8166b631b40b998c23247d1382925"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:27.412241 11709 rpcz_store.cc:276] 0925 09:57:25.586706 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:25.586951 (+ 245us) service_pool.cc:227] Handling call
0925 09:57:26.886058 (+1299107us) ts_tablet_manager.cc:1145] Deleting tablet 3be0eccc65564bfba3d6e5c398c61fb6
0925 09:57:26.886102 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:27.162499 (+276397us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:27.348470 (+185971us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:27.350588 (+ 2118us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":20378,"tablet-delete.queue_time_us":1278356}
W20260925 09:57:27.413728 10393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b5a8166b631b40b998c23247d1382925 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:27.413832 11709 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 232) took 1685 ms. Trace:
I20260925 09:57:27.412341 11085 ts_tablet_manager.cc:1397] T aba253f22afa4200926f31a1b43e3970 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.195s user 0.009s sys 0.002s
I20260925 09:57:27.414080 11709 rpcz_store.cc:276] 0925 09:57:25.728707 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:25.728997 (+ 290us) service_pool.cc:227] Handling call
0925 09:57:27.413518 (+1684521us) ts_tablet_manager.cc:1145] Deleting tablet 4e3120347c8e4a3eab57d6562ae40e8d
0925 09:57:27.413558 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:27.413783 (+ 225us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1588791}
I20260925 09:57:27.351356 10683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f07dc9548364870923165f4ea524310" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:27.409302 11082 tablet_bootstrap.cc:492] T 3ff4cf556eee494d8cd590884466f09f P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:27.415192 11082 ts_tablet_manager.cc:1397] T 3ff4cf556eee494d8cd590884466f09f P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.200s user 0.010s sys 0.031s
W20260925 09:57:27.415684 10919 leader_election.cc:341] T 2f07dc9548364870923165f4ea524310 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:27.416083 10919 leader_election.cc:302] T 2f07dc9548364870923165f4ea524310 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:27.416543 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 3be0eccc65564bfba3d6e5c398c61fb6 (table default.Table_0af2360d761342cf9f057be5f265aa41 [id=99ba81c4d0dd497398e048a05759419a]) successfully deleted
I20260925 09:57:27.417076 11901 raft_consensus.cc:2726] T 2f07dc9548364870923165f4ea524310 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:27.417140 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 4e3120347c8e4a3eab57d6562ae40e8d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4e3120347c8e4a3eab57d6562ae40e8d already in progress: creating tablet
I20260925 09:57:27.417526 11082 raft_consensus.cc:348] T 3ff4cf556eee494d8cd590884466f09f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.418154 11082 raft_consensus.cc:374] T 3ff4cf556eee494d8cd590884466f09f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:27.418403 11082 raft_consensus.cc:729] T 3ff4cf556eee494d8cd590884466f09f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:27.403224 11084 raft_consensus.cc:374] T 350904aa5957496d90a2aa2e637b591c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:27.420145 11084 raft_consensus.cc:729] T 350904aa5957496d90a2aa2e637b591c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:27.421478 11085 raft_consensus.cc:348] T aba253f22afa4200926f31a1b43e3970 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.421574 11084 consensus_queue.cc:260] T 350904aa5957496d90a2aa2e637b591c P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.422161 11085 raft_consensus.cc:374] T aba253f22afa4200926f31a1b43e3970 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:27.422401 11085 raft_consensus.cc:729] T aba253f22afa4200926f31a1b43e3970 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:27.426074 11084 ts_tablet_manager.cc:1428] T 350904aa5957496d90a2aa2e637b591c P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.031s user 0.005s sys 0.003s
I20260925 09:57:27.419009 11082 consensus_queue.cc:260] T 3ff4cf556eee494d8cd590884466f09f P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:27.433995 10390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_548921f3f0fd49069b0c73f688f4fa36 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:27.436251 11082 ts_tablet_manager.cc:1428] T 3ff4cf556eee494d8cd590884466f09f P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.021s user 0.005s sys 0.000s
I20260925 09:57:27.440096 11082 tablet_bootstrap.cc:492] T 60fdc299865446c89d3e5374d573029a P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:27.428783 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 216) took 4012 ms. Trace:
I20260925 09:57:27.444177 11727 rpcz_store.cc:276] 0925 09:57:23.416176 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:23.491518 (+ 75342us) service_pool.cc:227] Handling call
0925 09:57:27.071789 (+3580271us) ts_tablet_manager.cc:1145] Deleting tablet 7a31b999f0fd447eacff13de71aa3fff
0925 09:57:27.071843 (+ 54us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:27.184310 (+112467us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:27.372236 (+187926us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:27.428680 (+ 56444us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":15234,"tablet-delete.queue_time_us":3579726}
I20260925 09:57:27.447504 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 7a31b999f0fd447eacff13de71aa3fff (table default.Table_489e2c74ed6249ce9cf54541e0a491cd [id=8d2c4997954141129ebbec9e72658d90]) successfully deleted
I20260925 09:57:27.448571 11727 tablet_replica.cc:330] T 3706253e53d841aeaa43889ead13af32 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:27.449514 11727 raft_consensus.cc:2229] T 3706253e53d841aeaa43889ead13af32 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:27.450052 11727 raft_consensus.cc:2258] T 3706253e53d841aeaa43889ead13af32 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:27.423170 11085 consensus_queue.cc:260] T aba253f22afa4200926f31a1b43e3970 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.459486 11082 tablet_bootstrap.cc:654] T 60fdc299865446c89d3e5374d573029a P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:27.471194 11082 tablet_bootstrap.cc:492] T 60fdc299865446c89d3e5374d573029a P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:27.472060 10834 raft_consensus.cc:3037] T 9e697e7146e34841bd64660b45ebf231 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:27.472476 11082 ts_tablet_manager.cc:1397] T 60fdc299865446c89d3e5374d573029a P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.033s user 0.010s sys 0.000s
I20260925 09:57:27.474228 11709 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 233) took 1716 ms. Trace:
I20260925 09:57:27.474776 11082 raft_consensus.cc:348] T 60fdc299865446c89d3e5374d573029a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.475515 11082 raft_consensus.cc:374] T 60fdc299865446c89d3e5374d573029a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:27.475853 11082 raft_consensus.cc:729] T 60fdc299865446c89d3e5374d573029a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:27.474601 11709 rpcz_store.cc:276] 0925 09:57:25.757344 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:25.757438 (+ 94us) service_pool.cc:227] Handling call
0925 09:57:27.473238 (+1715800us) ts_tablet_manager.cc:1145] Deleting tablet 10657a81058241fc988d37e5ea7ff8cf
0925 09:57:27.473932 (+ 694us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:27.474176 (+ 244us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1648142}
W20260925 09:57:27.476995 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 10657a81058241fc988d37e5ea7ff8cf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 10657a81058241fc988d37e5ea7ff8cf already in progress: creating tablet
I20260925 09:57:27.492980 11709 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 238) took 1435 ms. Trace:
I20260925 09:57:27.493384 11709 rpcz_store.cc:276] 0925 09:57:26.057896 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:26.067043 (+ 9147us) service_pool.cc:227] Handling call
0925 09:57:27.476989 (+1409946us) ts_tablet_manager.cc:1145] Deleting tablet 417058493c6547c69e72347cef373292
0925 09:57:27.492587 (+ 15598us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:27.492914 (+ 327us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1401502}
W20260925 09:57:27.494783 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 417058493c6547c69e72347cef373292 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 417058493c6547c69e72347cef373292 already in progress: creating tablet
I20260925 09:57:27.480899 11082 consensus_queue.cc:260] T 60fdc299865446c89d3e5374d573029a P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.501034 11082 ts_tablet_manager.cc:1428] T 60fdc299865446c89d3e5374d573029a P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.028s user 0.005s sys 0.000s
I20260925 09:57:27.501991 11082 tablet_bootstrap.cc:492] T a44d9f2f7e3e48aeb0f31fa29ae1988a P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:27.502028 11709 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 244) took 1307 ms. Trace:
I20260925 09:57:27.502408 11709 rpcz_store.cc:276] 0925 09:57:26.194523 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:26.195421 (+ 898us) service_pool.cc:227] Handling call
0925 09:57:27.501718 (+1306297us) ts_tablet_manager.cc:1145] Deleting tablet cf775c9d9b114ddcb97dc6c6eca8e0fc
0925 09:57:27.501756 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:27.501979 (+ 223us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1305729}
W20260925 09:57:27.503489 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet cf775c9d9b114ddcb97dc6c6eca8e0fc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cf775c9d9b114ddcb97dc6c6eca8e0fc already in progress: creating tablet
I20260925 09:57:27.504125 11085 ts_tablet_manager.cc:1428] T aba253f22afa4200926f31a1b43e3970 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.090s user 0.005s sys 0.002s
I20260925 09:57:27.505278 11085 tablet_bootstrap.cc:492] T d9e454f82a4e4c429b58f32926319b7c P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
W20260925 09:57:27.508837 10379 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.9.60:36985: connect: Connection refused (error 111) [suppressed 7 similar messages]
I20260925 09:57:27.508769 10981 raft_consensus.cc:2454] T 3b9cdad4f9144daea800553abd378e0b P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:27.508839 11727 ts_tablet_manager.cc:1905] T 3706253e53d841aeaa43889ead13af32 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:27.510398 10619 leader_election.cc:302] T 3b9cdad4f9144daea800553abd378e0b P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:27.511606 11919 raft_consensus.cc:2781] T 3b9cdad4f9144daea800553abd378e0b P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:27.512111 11919 raft_consensus.cc:686] T 3b9cdad4f9144daea800553abd378e0b P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:27.513068 11919 consensus_queue.cc:237] T 3b9cdad4f9144daea800553abd378e0b P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.523020 11082 tablet_bootstrap.cc:654] T a44d9f2f7e3e48aeb0f31fa29ae1988a P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:27.524771 11977 raft_consensus.cc:482] T 3ff4cf556eee494d8cd590884466f09f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:27.525187 11977 raft_consensus.cc:504] T 3ff4cf556eee494d8cd590884466f09f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.527228 11977 leader_election.cc:288] T 3ff4cf556eee494d8cd590884466f09f P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
W20260925 09:57:27.533159 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 9a73a1f242e7430cbf52229c07b4dc82 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9a73a1f242e7430cbf52229c07b4dc82 already in progress: creating tablet
I20260925 09:57:27.509610 11084 tablet_bootstrap.cc:492] T b61444c31dbf4882ac748f861ea66818 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
W20260925 09:57:27.511747 10918 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 59153.6us
I20260925 09:57:27.516677 11892 raft_consensus.cc:482] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c43461c4ee854ae1ac52b873ccada318)
I20260925 09:57:27.536058 11892 raft_consensus.cc:504] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:27.536820 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 359090244c4748e8a2d45e3c77e89498 because the tablet was not found. No further retry: Not found: Tablet not found: 359090244c4748e8a2d45e3c77e89498
W20260925 09:57:27.539876 10919 leader_election.cc:341] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Not found: Tablet not found: 359090244c4748e8a2d45e3c77e89498
I20260925 09:57:27.541007 11892 leader_election.cc:288] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:27.545269 10981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ff4cf556eee494d8cd590884466f09f" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:27.545830 10981 raft_consensus.cc:2454] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:27.551688 11709 tablet_replica.cc:330] T d9e454f82a4e4c429b58f32926319b7c P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:27.553890 11709 raft_consensus.cc:2229] T d9e454f82a4e4c429b58f32926319b7c P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Raft consensus shutting down.
W20260925 09:57:27.555882 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 3039ec3febf848dba7c4e6fd38bb8053 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3039ec3febf848dba7c4e6fd38bb8053 already in progress: creating tablet
I20260925 09:57:27.555711 10619 leader_election.cc:302] T 3ff4cf556eee494d8cd590884466f09f P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:27.560125 11919 raft_consensus.cc:2781] T 3ff4cf556eee494d8cd590884466f09f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:27.560606 11709 pending_rounds.cc:62] T d9e454f82a4e4c429b58f32926319b7c P b8850818e2ee42a082802d0095c08d21: Trying to abort 1 pending ops.
W20260925 09:57:27.560499 10379 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.60:36985, user_credentials={real_user=slave}} blocked reactor thread for 51689.4us
I20260925 09:57:27.560737 11919 raft_consensus.cc:482] T 3ff4cf556eee494d8cd590884466f09f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:27.561893 11709 pending_rounds.cc:68] T d9e454f82a4e4c429b58f32926319b7c P b8850818e2ee42a082802d0095c08d21: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192682897068032 op_type: NO_OP noop_request { }
I20260925 09:57:27.561995 11892 raft_consensus.cc:482] T ab6bdba5ca5d41ae921a90c1f77006c5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:27.562436 11709 raft_consensus.cc:2866] T d9e454f82a4e4c429b58f32926319b7c P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:57:27.562400 11892 raft_consensus.cc:504] T ab6bdba5ca5d41ae921a90c1f77006c5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.562320 11919 raft_consensus.cc:3037] T 3ff4cf556eee494d8cd590884466f09f P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:27.563534 11709 raft_consensus.cc:2258] T d9e454f82a4e4c429b58f32926319b7c P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:27.565263 11709 ts_tablet_manager.cc:1905] T d9e454f82a4e4c429b58f32926319b7c P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:27.565408 10683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab6bdba5ca5d41ae921a90c1f77006c5" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:27.566606 10919 leader_election.cc:341] T ab6bdba5ca5d41ae921a90c1f77006c5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:27.567416 11892 leader_election.cc:288] T ab6bdba5ca5d41ae921a90c1f77006c5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
W20260925 09:57:27.568723 10379 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 -> Peer b8be057e0b6d40a3a4b90f2b1f4a0fa2 (127.10.9.60:36985): Couldn't send request to peer b8be057e0b6d40a3a4b90f2b1f4a0fa2. Status: Network error: Client connection negotiation failed: client connection to 127.10.9.60:36985: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260925 09:57:27.571997 10390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_3f8e7f40931b4423bf4df247624ef2bb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:27.573375 10390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3f8e7f40931b4423bf4df247624ef2bb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:27.588125 11084 tablet_bootstrap.cc:654] T b61444c31dbf4882ac748f861ea66818 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:27.597747 10390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_bd0dc294c2c049e9b9c9eddce6772a44"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:27.598847 10834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ff4cf556eee494d8cd590884466f09f" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:27.599248 10390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bd0dc294c2c049e9b9c9eddce6772a44 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:27.603780 10834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab6bdba5ca5d41ae921a90c1f77006c5" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:27.605321 10621 leader_election.cc:341] T 3ff4cf556eee494d8cd590884466f09f P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:27.639993 11082 tablet_bootstrap.cc:492] T a44d9f2f7e3e48aeb0f31fa29ae1988a P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
W20260925 09:57:27.612457 10918 leader_election.cc:341] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Not found: Tablet not found: 359090244c4748e8a2d45e3c77e89498
I20260925 09:57:27.640482 11082 ts_tablet_manager.cc:1397] T a44d9f2f7e3e48aeb0f31fa29ae1988a P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.139s user 0.035s sys 0.022s
I20260925 09:57:27.640604 10918 leader_election.cc:302] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:27.641304 11892 raft_consensus.cc:2726] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260925 09:57:27.642756 11082 raft_consensus.cc:348] T a44d9f2f7e3e48aeb0f31fa29ae1988a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:27.643702 10918 leader_election.cc:341] T ab6bdba5ca5d41ae921a90c1f77006c5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:27.644682 10918 leader_election.cc:302] T ab6bdba5ca5d41ae921a90c1f77006c5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:27.647174 11919 raft_consensus.cc:504] T 3ff4cf556eee494d8cd590884466f09f P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.647887 11082 raft_consensus.cc:374] T a44d9f2f7e3e48aeb0f31fa29ae1988a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:27.648471 11082 raft_consensus.cc:729] T a44d9f2f7e3e48aeb0f31fa29ae1988a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:27.649272 11901 raft_consensus.cc:2726] T ab6bdba5ca5d41ae921a90c1f77006c5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:27.649662 11082 consensus_queue.cc:260] T a44d9f2f7e3e48aeb0f31fa29ae1988a P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.650156 11892 raft_consensus.cc:482] T a436fe287a0d48a1821191adcd031d74 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:27.650640 11892 raft_consensus.cc:504] T a436fe287a0d48a1821191adcd031d74 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.653600 10683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a436fe287a0d48a1821191adcd031d74" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:27.654597 11961 raft_consensus.cc:482] T aba253f22afa4200926f31a1b43e3970 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:27.654881 11082 ts_tablet_manager.cc:1428] T a44d9f2f7e3e48aeb0f31fa29ae1988a P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.014s user 0.006s sys 0.001s
I20260925 09:57:27.655035 11961 raft_consensus.cc:504] T aba253f22afa4200926f31a1b43e3970 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.656889 11082 tablet_bootstrap.cc:492] T c2110d4ada2d45c2b07259374465a8df P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:27.656891 11961 leader_election.cc:288] T aba253f22afa4200926f31a1b43e3970 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
W20260925 09:57:27.658547 10919 leader_election.cc:341] T a436fe287a0d48a1821191adcd031d74 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:27.660549 10982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aba253f22afa4200926f31a1b43e3970" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:27.661281 10982 raft_consensus.cc:2379] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c43461c4ee854ae1ac52b873ccada318 in current term 1: Already voted for candidate b8850818e2ee42a082802d0095c08d21 in this term.
I20260925 09:57:27.661989 11082 tablet_bootstrap.cc:654] T c2110d4ada2d45c2b07259374465a8df P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:27.663681 11892 leader_election.cc:288] T a436fe287a0d48a1821191adcd031d74 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:27.666339 10390 master_service.cc:946] ReplaceTablet: received request to replace tablet 17bd30416fe14c0e9ab2e0c0bf5f2fb5 from {username='slave'} at 127.0.0.1:35140
I20260925 09:57:27.667869 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ff4cf556eee494d8cd590884466f09f" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:27.636951 11085 tablet_bootstrap.cc:654] T d9e454f82a4e4c429b58f32926319b7c P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:27.675963 10834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a436fe287a0d48a1821191adcd031d74" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:27.678438 10918 leader_election.cc:341] T a436fe287a0d48a1821191adcd031d74 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:27.678908 10918 leader_election.cc:302] T a436fe287a0d48a1821191adcd031d74 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:27.679849 11892 raft_consensus.cc:2726] T a436fe287a0d48a1821191adcd031d74 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:27.680878 10683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aba253f22afa4200926f31a1b43e3970" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:27.683151 10775 leader_election.cc:302] T aba253f22afa4200926f31a1b43e3970 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:27.684162 11961 raft_consensus.cc:3037] T aba253f22afa4200926f31a1b43e3970 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:27.688695 11084 tablet_bootstrap.cc:492] T b61444c31dbf4882ac748f861ea66818 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:27.689442 11084 ts_tablet_manager.cc:1397] T b61444c31dbf4882ac748f861ea66818 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.180s user 0.011s sys 0.000s
I20260925 09:57:27.691854 11084 raft_consensus.cc:348] T b61444c31dbf4882ac748f861ea66818 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.692607 11961 raft_consensus.cc:2726] T aba253f22afa4200926f31a1b43e3970 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:27.695690 11084 raft_consensus.cc:374] T b61444c31dbf4882ac748f861ea66818 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260925 09:57:27.697155 10621 leader_election.cc:341] T 3ff4cf556eee494d8cd590884466f09f P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:27.700966 11084 raft_consensus.cc:729] T b61444c31dbf4882ac748f861ea66818 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:27.702443 11084 consensus_queue.cc:260] T b61444c31dbf4882ac748f861ea66818 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.715713 11919 leader_election.cc:288] T 3ff4cf556eee494d8cd590884466f09f P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:27.716351 10982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ff4cf556eee494d8cd590884466f09f" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:27.716886 10982 raft_consensus.cc:3037] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:27.717476 11084 ts_tablet_manager.cc:1428] T b61444c31dbf4882ac748f861ea66818 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.028s user 0.005s sys 0.000s
I20260925 09:57:27.721683 11084 tablet_bootstrap.cc:492] T 92814437d53143a79f99199f8ee3a742 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:27.722870 10982 raft_consensus.cc:2454] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:27.723862 10619 leader_election.cc:302] T 3ff4cf556eee494d8cd590884466f09f P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:27.724943 11919 raft_consensus.cc:2781] T 3ff4cf556eee494d8cd590884466f09f P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:27.727517 11919 raft_consensus.cc:686] T 3ff4cf556eee494d8cd590884466f09f P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:27.728325 11919 consensus_queue.cc:237] T 3ff4cf556eee494d8cd590884466f09f P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.737499 11085 tablet_bootstrap.cc:492] T d9e454f82a4e4c429b58f32926319b7c P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:27.737928 11085 ts_tablet_manager.cc:1397] T d9e454f82a4e4c429b58f32926319b7c P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.233s user 0.010s sys 0.001s
I20260925 09:57:27.741902 11082 tablet_bootstrap.cc:492] T c2110d4ada2d45c2b07259374465a8df P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:27.741590 11085 raft_consensus.cc:348] T d9e454f82a4e4c429b58f32926319b7c P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.742321 11085 raft_consensus.cc:374] T d9e454f82a4e4c429b58f32926319b7c P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:27.742395 11082 ts_tablet_manager.cc:1397] T c2110d4ada2d45c2b07259374465a8df P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.086s user 0.007s sys 0.016s
I20260925 09:57:27.742580 11085 raft_consensus.cc:729] T d9e454f82a4e4c429b58f32926319b7c P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:27.743316 11085 consensus_queue.cc:260] T d9e454f82a4e4c429b58f32926319b7c P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.748138 11084 tablet_bootstrap.cc:654] T 92814437d53143a79f99199f8ee3a742 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:27.731287 11709 ts_tablet_manager.cc:1918] T d9e454f82a4e4c429b58f32926319b7c P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:27.760867 11727 ts_tablet_manager.cc:1918] T 3706253e53d841aeaa43889ead13af32 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:27.761371 11727 log.cc:1196] T 3706253e53d841aeaa43889ead13af32 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/3706253e53d841aeaa43889ead13af32
I20260925 09:57:27.762274 11727 ts_tablet_manager.cc:1939] T 3706253e53d841aeaa43889ead13af32 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:27.764472 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 218) took 3413 ms. Trace:
I20260925 09:57:27.769239 11709 log.cc:1196] T d9e454f82a4e4c429b58f32926319b7c P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/d9e454f82a4e4c429b58f32926319b7c
I20260925 09:57:27.769726 11082 raft_consensus.cc:348] T c2110d4ada2d45c2b07259374465a8df P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.770473 11082 raft_consensus.cc:374] T c2110d4ada2d45c2b07259374465a8df P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:27.770748 11082 raft_consensus.cc:729] T c2110d4ada2d45c2b07259374465a8df P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:27.771369 11082 consensus_queue.cc:260] T c2110d4ada2d45c2b07259374465a8df P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.776511 11082 ts_tablet_manager.cc:1428] T c2110d4ada2d45c2b07259374465a8df P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.034s user 0.003s sys 0.005s
I20260925 09:57:27.779279 11709 ts_tablet_manager.cc:1939] T d9e454f82a4e4c429b58f32926319b7c P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:27.781463 11961 raft_consensus.cc:482] T d9e454f82a4e4c429b58f32926319b7c P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:27.781910 11961 raft_consensus.cc:504] T d9e454f82a4e4c429b58f32926319b7c P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.783743 11961 leader_election.cc:288] T d9e454f82a4e4c429b58f32926319b7c P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:27.785616 11085 ts_tablet_manager.cc:1428] T d9e454f82a4e4c429b58f32926319b7c P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.047s user 0.007s sys 0.000s
W20260925 09:57:27.785624 10772 leader_election.cc:341] T d9e454f82a4e4c429b58f32926319b7c P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: d9e454f82a4e4c429b58f32926319b7c
I20260925 09:57:27.786515 11085 tablet_bootstrap.cc:492] T 3b9cdad4f9144daea800553abd378e0b P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:27.784329 10683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9e454f82a4e4c429b58f32926319b7c" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:27.796353 10775 leader_election.cc:341] T d9e454f82a4e4c429b58f32926319b7c P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: Tablet not RUNNING: STOPPED
I20260925 09:57:27.796715 10775 leader_election.cc:302] T d9e454f82a4e4c429b58f32926319b7c P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:27.797760 11082 tablet_bootstrap.cc:492] T 977a59175e5b43eeb6f23b938afd00a3 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:27.797986 11954 raft_consensus.cc:2726] T d9e454f82a4e4c429b58f32926319b7c P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:27.798834 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet d9e454f82a4e4c429b58f32926319b7c (table default.Table_95a4c3e1f6014e63811447ba4a0e34cf [id=626e547271e6456e81ccaa3b1e59bd97]) successfully deleted
W20260925 09:57:27.799502 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 69028aa7908c45b896da4ba8deef07b1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 69028aa7908c45b896da4ba8deef07b1 already in progress: creating tablet
W20260925 09:57:27.800084 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 67ce2e3325214aa6bdc3347e9e8f82b0 because the tablet was not found. No further retry: Not found: Tablet not found: 67ce2e3325214aa6bdc3347e9e8f82b0
W20260925 09:57:27.801358 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet a81530200cf14717bc7cbf67bb73f4e5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a81530200cf14717bc7cbf67bb73f4e5 already in progress: creating tablet
I20260925 09:57:27.802083 11709 tablet_replica.cc:330] T aba253f22afa4200926f31a1b43e3970 P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:27.802858 11709 raft_consensus.cc:2229] T aba253f22afa4200926f31a1b43e3970 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:27.803490 11727 rpcz_store.cc:276] 0925 09:57:24.351123 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:24.386007 (+ 34884us) service_pool.cc:227] Handling call
0925 09:57:27.448388 (+3062381us) ts_tablet_manager.cc:1145] Deleting tablet 3706253e53d841aeaa43889ead13af32
0925 09:57:27.448439 (+ 51us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:27.643742 (+195303us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:27.760619 (+116877us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:27.764416 (+ 3797us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3061063}
I20260925 09:57:27.804908 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 3706253e53d841aeaa43889ead13af32 (table default.Table_bbc3c50c06634d119ae96dcf649f44e7 [id=0260f0e6cdd6431db06788bde54ec38c]) successfully deleted
I20260925 09:57:27.779706 11084 tablet_bootstrap.cc:492] T 92814437d53143a79f99199f8ee3a742 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:27.805913 11084 ts_tablet_manager.cc:1397] T 92814437d53143a79f99199f8ee3a742 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.084s user 0.006s sys 0.004s
I20260925 09:57:27.807196 10816 tablet_service.cc:1467] Processing CreateTablet for tablet 2b1b2d66397b44f38056077d79941ac4 (DEFAULT_TABLE table=default.Table_94b396db46ba43d88015187a199247fc [id=78304acc330348238cdb35b6034337fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:27.807315 10663 tablet_service.cc:1467] Processing CreateTablet for tablet 2b1b2d66397b44f38056077d79941ac4 (DEFAULT_TABLE table=default.Table_94b396db46ba43d88015187a199247fc [id=78304acc330348238cdb35b6034337fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:27.808207 10816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b1b2d66397b44f38056077d79941ac4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:27.808337 10663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b1b2d66397b44f38056077d79941ac4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:27.808053 11084 raft_consensus.cc:348] T 92814437d53143a79f99199f8ee3a742 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.808779 11084 raft_consensus.cc:374] T 92814437d53143a79f99199f8ee3a742 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:27.809075 11084 raft_consensus.cc:729] T 92814437d53143a79f99199f8ee3a742 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:27.809854 11084 consensus_queue.cc:260] T 92814437d53143a79f99199f8ee3a742 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.824936 10390 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 17bd30416fe14c0e9ab2e0c0bf5f2fb5
I20260925 09:57:27.826198 10390 catalog_manager.cc:6565] ReplaceTablet: tablet 17bd30416fe14c0e9ab2e0c0bf5f2fb5 deleted and replaced by tablet 4a6b1284cddc4b2581931d8895212ca6
I20260925 09:57:27.827478 11085 tablet_bootstrap.cc:654] T 3b9cdad4f9144daea800553abd378e0b P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:27.827575 11082 tablet_bootstrap.cc:654] T 977a59175e5b43eeb6f23b938afd00a3 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:27.828004 10955 tablet_service.cc:1514] Processing DeleteTablet for tablet 17bd30416fe14c0e9ab2e0c0bf5f2fb5 with delete_type TABLET_DATA_DELETED (replaced by tablet 4a6b1284cddc4b2581931d8895212ca6) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:27.829756 11727 tablet_replica.cc:330] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:27.830899 11727 raft_consensus.cc:2229] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:27.831600 11727 raft_consensus.cc:2258] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:27.832711 10655 tablet_service.cc:1514] Processing DeleteTablet for tablet 17bd30416fe14c0e9ab2e0c0bf5f2fb5 with delete_type TABLET_DATA_DELETED (replaced by tablet 4a6b1284cddc4b2581931d8895212ca6) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:27.833946 11727 ts_tablet_manager.cc:1905] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:27.835788 11082 tablet_bootstrap.cc:492] T 977a59175e5b43eeb6f23b938afd00a3 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:27.836197 11082 ts_tablet_manager.cc:1397] T 977a59175e5b43eeb6f23b938afd00a3 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.039s user 0.006s sys 0.003s
I20260925 09:57:27.836284 10815 tablet_service.cc:1514] Processing DeleteTablet for tablet 17bd30416fe14c0e9ab2e0c0bf5f2fb5 with delete_type TABLET_DATA_DELETED (replaced by tablet 4a6b1284cddc4b2581931d8895212ca6) from {username='slave'} at 127.0.0.1:33344
W20260925 09:57:27.839164 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 17bd30416fe14c0e9ab2e0c0bf5f2fb5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 17bd30416fe14c0e9ab2e0c0bf5f2fb5 already in progress: creating tablet
I20260925 09:57:27.844076 11082 raft_consensus.cc:348] T 977a59175e5b43eeb6f23b938afd00a3 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.844794 11082 raft_consensus.cc:374] T 977a59175e5b43eeb6f23b938afd00a3 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:27.845053 11082 raft_consensus.cc:729] T 977a59175e5b43eeb6f23b938afd00a3 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:27.847834 11082 consensus_queue.cc:260] T 977a59175e5b43eeb6f23b938afd00a3 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.849381 10955 tablet_service.cc:1467] Processing CreateTablet for tablet 2b1b2d66397b44f38056077d79941ac4 (DEFAULT_TABLE table=default.Table_94b396db46ba43d88015187a199247fc [id=78304acc330348238cdb35b6034337fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:27.850091 10815 tablet_service.cc:1467] Processing CreateTablet for tablet dbc7c7b9a2ee437eabe10150bdbc2e4a (DEFAULT_TABLE table=default.Table_8ccc930f32b84671ab46826d4a9fceef [id=04481da81650427199dcc7b1ca10f793]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:27.851011 10815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbc7c7b9a2ee437eabe10150bdbc2e4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:27.850924 10655 tablet_service.cc:1467] Processing CreateTablet for tablet dbc7c7b9a2ee437eabe10150bdbc2e4a (DEFAULT_TABLE table=default.Table_8ccc930f32b84671ab46826d4a9fceef [id=04481da81650427199dcc7b1ca10f793]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:27.851630 11084 ts_tablet_manager.cc:1428] T 92814437d53143a79f99199f8ee3a742 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.045s user 0.006s sys 0.000s
I20260925 09:57:27.855185 11084 tablet_bootstrap.cc:492] T cf775c9d9b114ddcb97dc6c6eca8e0fc P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:27.855311 10658 tablet_service.cc:1467] Processing CreateTablet for tablet f3bba45d6b7241f9a07907c79b892f7d (DEFAULT_TABLE table=default.Table_0f18e1fd434a47d2828c80f67ab5107f [id=e8a1524ee2dc4f1c86ff27b9c4488147]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:27.855856 10655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbc7c7b9a2ee437eabe10150bdbc2e4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:27.856420 10658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3bba45d6b7241f9a07907c79b892f7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:27.856532 10956 tablet_service.cc:1467] Processing CreateTablet for tablet dbc7c7b9a2ee437eabe10150bdbc2e4a (DEFAULT_TABLE table=default.Table_8ccc930f32b84671ab46826d4a9fceef [id=04481da81650427199dcc7b1ca10f793]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:27.858626 10805 tablet_service.cc:1467] Processing CreateTablet for tablet f3bba45d6b7241f9a07907c79b892f7d (DEFAULT_TABLE table=default.Table_0f18e1fd434a47d2828c80f67ab5107f [id=e8a1524ee2dc4f1c86ff27b9c4488147]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:27.859848 10805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3bba45d6b7241f9a07907c79b892f7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:27.867769 10956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbc7c7b9a2ee437eabe10150bdbc2e4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:27.871659 10955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b1b2d66397b44f38056077d79941ac4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:27.875526 11709 pending_rounds.cc:62] T aba253f22afa4200926f31a1b43e3970 P b8850818e2ee42a082802d0095c08d21: Trying to abort 1 pending ops.
I20260925 09:57:27.875900 11082 ts_tablet_manager.cc:1428] T 977a59175e5b43eeb6f23b938afd00a3 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.039s user 0.005s sys 0.001s
I20260925 09:57:27.875837 11709 pending_rounds.cc:68] T aba253f22afa4200926f31a1b43e3970 P b8850818e2ee42a082802d0095c08d21: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192691345014784 op_type: NO_OP noop_request { }
I20260925 09:57:27.876420 11709 raft_consensus.cc:2866] T aba253f22afa4200926f31a1b43e3970 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:57:27.876736 11709 raft_consensus.cc:2258] T aba253f22afa4200926f31a1b43e3970 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:27.880095 11709 ts_tablet_manager.cc:1905] T aba253f22afa4200926f31a1b43e3970 P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:27.899701 11082 tablet_bootstrap.cc:492] T fe91b39f91274a7ca61e9618d8cbb475 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:27.901208 10388 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 59f0c6489a33433899bb6e62088121b6 (table default.Table_2d40588792754851bf107d378c5950dd [id=64032d286bf94c4cbdddde18adfb739e]). Expected version 1 got 0
I20260925 09:57:27.906356 11082 tablet_bootstrap.cc:654] T fe91b39f91274a7ca61e9618d8cbb475 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:27.913070 10953 tablet_service.cc:1467] Processing CreateTablet for tablet f3bba45d6b7241f9a07907c79b892f7d (DEFAULT_TABLE table=default.Table_0f18e1fd434a47d2828c80f67ab5107f [id=e8a1524ee2dc4f1c86ff27b9c4488147]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:27.918905 10960 tablet_service.cc:1467] Processing CreateTablet for tablet 88e53debe6de43e29b12e85e5171c8b5 (DEFAULT_TABLE table=default.Table_e22cff6ba0484ecf82c555938f037b69 [id=99951d40c1874dcfb3e8b6ae665395e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:27.919476 10812 tablet_service.cc:1467] Processing CreateTablet for tablet 56f8c08025b94c6eb8b429bd7b0408aa (DEFAULT_TABLE table=default.Table_2b0b0b9f52034c03aa2c4f041b28ec3d [id=1f8215ee448f4d59802bf0420b81b699]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:27.920485 10812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56f8c08025b94c6eb8b429bd7b0408aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:27.927824 11085 tablet_bootstrap.cc:492] T 3b9cdad4f9144daea800553abd378e0b P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:27.928339 11085 ts_tablet_manager.cc:1397] T 3b9cdad4f9144daea800553abd378e0b P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.142s user 0.008s sys 0.002s
I20260925 09:57:27.931998 10816 tablet_service.cc:1467] Processing CreateTablet for tablet 88e53debe6de43e29b12e85e5171c8b5 (DEFAULT_TABLE table=default.Table_e22cff6ba0484ecf82c555938f037b69 [id=99951d40c1874dcfb3e8b6ae665395e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:27.933028 10816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88e53debe6de43e29b12e85e5171c8b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:27.949710 10953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3bba45d6b7241f9a07907c79b892f7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:27.951583 10658 tablet_service.cc:1467] Processing CreateTablet for tablet 88e53debe6de43e29b12e85e5171c8b5 (DEFAULT_TABLE table=default.Table_e22cff6ba0484ecf82c555938f037b69 [id=99951d40c1874dcfb3e8b6ae665395e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:27.951785 10667 tablet_service.cc:1467] Processing CreateTablet for tablet cc295f7a9169475f9a9235e657be629f (DEFAULT_TABLE table=default.Table_0af2360d761342cf9f057be5f265aa41 [id=99ba81c4d0dd497398e048a05759419a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:27.952049 10961 tablet_service.cc:1467] Processing CreateTablet for tablet 56f8c08025b94c6eb8b429bd7b0408aa (DEFAULT_TABLE table=default.Table_2b0b0b9f52034c03aa2c4f041b28ec3d [id=1f8215ee448f4d59802bf0420b81b699]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:27.952672 10658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88e53debe6de43e29b12e85e5171c8b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:27.953366 10960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88e53debe6de43e29b12e85e5171c8b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:27.959496 10957 tablet_service.cc:1467] Processing CreateTablet for tablet cc295f7a9169475f9a9235e657be629f (DEFAULT_TABLE table=default.Table_0af2360d761342cf9f057be5f265aa41 [id=99ba81c4d0dd497398e048a05759419a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:27.960525 10957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc295f7a9169475f9a9235e657be629f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:27.969358 10667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc295f7a9169475f9a9235e657be629f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:27.975571 10961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56f8c08025b94c6eb8b429bd7b0408aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:27.945497 11085 raft_consensus.cc:348] T 3b9cdad4f9144daea800553abd378e0b P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.976325 11085 raft_consensus.cc:374] T 3b9cdad4f9144daea800553abd378e0b P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:27.976567 11085 raft_consensus.cc:729] T 3b9cdad4f9144daea800553abd378e0b P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:27.902283 10388 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet ef22f64036684dce9d7c69c37722f468 (table default.Table_0d7d27d05491401089db5f65b43e86f7 [id=a20216b9b3074e408bee74e39c5819b4]). Expected version 1 got 0
I20260925 09:57:27.980710 10388 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 93e1b5837aff48d7907890c40dfcd0d0 (table default.Table_f8b067e4e4cf498bbf44d7bf24c29363 [id=23d14ed1d2d647fc8b3b5ef40c7dc0bb]). Expected version 1 got 0
I20260925 09:57:27.981874 11085 consensus_queue.cc:260] T 3b9cdad4f9144daea800553abd378e0b P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:27.982400 10388 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet c2110d4ada2d45c2b07259374465a8df (table default.Table_44ddd064d56640a0a0e3dd8fd38344fd [id=8307778a24ee442b8f7e89c09652309e]). Expected version 1 got 0
I20260925 09:57:27.982785 11709 ts_tablet_manager.cc:1918] T aba253f22afa4200926f31a1b43e3970 P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:27.983099 11709 log.cc:1196] T aba253f22afa4200926f31a1b43e3970 P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/aba253f22afa4200926f31a1b43e3970
I20260925 09:57:27.983110 10388 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 5608c5f9b65b4da7b2fbb56417c92d88 (table default.Table_1b4a040b8d60467a869df4b51c09cf46 [id=e46d114e5d484b66bb15eb0416b5ce0d]). Expected version 1 got 0
I20260925 09:57:27.984175 11085 ts_tablet_manager.cc:1428] T 3b9cdad4f9144daea800553abd378e0b P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.056s user 0.005s sys 0.002s
I20260925 09:57:27.984764 11709 ts_tablet_manager.cc:1939] T aba253f22afa4200926f31a1b43e3970 P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:27.985051 11085 tablet_bootstrap.cc:492] T 2d087d3e35814c90a03eccd90eeea86c P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:27.990693 10958 tablet_service.cc:1467] Processing CreateTablet for tablet a58cf1dc81454710a9ad1a8baf155c9f (DEFAULT_TABLE table=default.Table_0478cdcfe9d64b66ba0e3fbc638f9a3b [id=58904eafabaa40eeb24a79dba89ae346]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:27.992686 10963 tablet_service.cc:1467] Processing CreateTablet for tablet c52745a8cdee461aba7485b0531245d6 (DEFAULT_TABLE table=default.Table_8e186a99ccba425b9b588041d2364a40 [id=2d6bc0e2aaee4b1483b01fc678505bb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:27.992731 10815 tablet_service.cc:1467] Processing CreateTablet for tablet a58cf1dc81454710a9ad1a8baf155c9f (DEFAULT_TABLE table=default.Table_0478cdcfe9d64b66ba0e3fbc638f9a3b [id=58904eafabaa40eeb24a79dba89ae346]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:27.993541 11084 tablet_bootstrap.cc:654] T cf775c9d9b114ddcb97dc6c6eca8e0fc P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:27.994789 10806 tablet_service.cc:1467] Processing CreateTablet for tablet c52745a8cdee461aba7485b0531245d6 (DEFAULT_TABLE table=default.Table_8e186a99ccba425b9b588041d2364a40 [id=2d6bc0e2aaee4b1483b01fc678505bb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:27.995174 10811 tablet_service.cc:1467] Processing CreateTablet for tablet f1d31dca90df414a83be7c9e404b8d30 (DEFAULT_TABLE table=default.Table_1f25b6a5838b40758e7645d5d47ad04e [id=584cddf4e10c46d8827735eb5bdbe5d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:27.996975 10663 tablet_service.cc:1467] Processing CreateTablet for tablet 56f8c08025b94c6eb8b429bd7b0408aa (DEFAULT_TABLE table=default.Table_2b0b0b9f52034c03aa2c4f041b28ec3d [id=1f8215ee448f4d59802bf0420b81b699]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:27.998121 10663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56f8c08025b94c6eb8b429bd7b0408aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:28.006556 10958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a58cf1dc81454710a9ad1a8baf155c9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:28.007462 10963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c52745a8cdee461aba7485b0531245d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:28.011849 10815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a58cf1dc81454710a9ad1a8baf155c9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:28.012356 10811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1d31dca90df414a83be7c9e404b8d30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:28.015700 10806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c52745a8cdee461aba7485b0531245d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:28.020175 10805 tablet_service.cc:1467] Processing CreateTablet for tablet cc295f7a9169475f9a9235e657be629f (DEFAULT_TABLE table=default.Table_0af2360d761342cf9f057be5f265aa41 [id=99ba81c4d0dd497398e048a05759419a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:28.021234 10805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc295f7a9169475f9a9235e657be629f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:28.025243 11727 ts_tablet_manager.cc:1918] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.1
I20260925 09:57:28.025713 11727 log.cc:1196] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/359090244c4748e8a2d45e3c77e89498
I20260925 09:57:28.026604 11727 ts_tablet_manager.cc:1939] T 359090244c4748e8a2d45e3c77e89498 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:28.023204 11085 tablet_bootstrap.cc:654] T 2d087d3e35814c90a03eccd90eeea86c P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:27.989444 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet aba253f22afa4200926f31a1b43e3970 (table default.Table_bc338e8ca7174647948301512850ce00 [id=7d130f66062344458770b2b943b4485a]) successfully deleted
W20260925 09:57:28.039222 10380 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 49795us
W20260925 09:57:28.039849 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 17bd30416fe14c0e9ab2e0c0bf5f2fb5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 17bd30416fe14c0e9ab2e0c0bf5f2fb5 already in progress: creating tablet
I20260925 09:57:28.041808 10657 tablet_service.cc:1467] Processing CreateTablet for tablet c52745a8cdee461aba7485b0531245d6 (DEFAULT_TABLE table=default.Table_8e186a99ccba425b9b588041d2364a40 [id=2d6bc0e2aaee4b1483b01fc678505bb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:28.042897 10657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c52745a8cdee461aba7485b0531245d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:28.055402 11085 tablet_bootstrap.cc:492] T 2d087d3e35814c90a03eccd90eeea86c P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:28.055915 11085 ts_tablet_manager.cc:1397] T 2d087d3e35814c90a03eccd90eeea86c P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.071s user 0.009s sys 0.003s
I20260925 09:57:28.061582 11085 raft_consensus.cc:348] T 2d087d3e35814c90a03eccd90eeea86c P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:28.062413 11085 raft_consensus.cc:374] T 2d087d3e35814c90a03eccd90eeea86c P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:28.062731 11085 raft_consensus.cc:729] T 2d087d3e35814c90a03eccd90eeea86c P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:28.063171 10388 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 3d652a76d96f4dbeafde15f3c16534a6 (table default.Table_be0f658aa41f4188bc16db37e96f4ad1 [id=73083c1560cb4db5a12964387cb49b73]). Expected version 1 got 0
I20260925 09:57:28.064185 10388 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 26fae3fbfe374f348a1729c21ec751b9 (table default.Table_0011387b45014de4b24b466f4cb907f7 [id=e3f7aa5e988c4b6c98be5cd3d465245d]). Expected version 1 got 0
I20260925 09:57:28.068627 10666 tablet_service.cc:1467] Processing CreateTablet for tablet f1d31dca90df414a83be7c9e404b8d30 (DEFAULT_TABLE table=default.Table_1f25b6a5838b40758e7645d5d47ad04e [id=584cddf4e10c46d8827735eb5bdbe5d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:28.069653 10666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1d31dca90df414a83be7c9e404b8d30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:28.075789 11085 consensus_queue.cc:260] T 2d087d3e35814c90a03eccd90eeea86c P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:28.082439 11085 ts_tablet_manager.cc:1428] T 2d087d3e35814c90a03eccd90eeea86c P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.026s user 0.006s sys 0.000s
I20260925 09:57:28.083328 11085 tablet_bootstrap.cc:492] T a44d9f2f7e3e48aeb0f31fa29ae1988a P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:28.086716 11082 tablet_bootstrap.cc:492] T fe91b39f91274a7ca61e9618d8cbb475 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:28.087105 11082 ts_tablet_manager.cc:1397] T fe91b39f91274a7ca61e9618d8cbb475 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.188s user 0.057s sys 0.089s
I20260925 09:57:28.089182 11082 raft_consensus.cc:348] T fe91b39f91274a7ca61e9618d8cbb475 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:28.089802 11082 raft_consensus.cc:374] T fe91b39f91274a7ca61e9618d8cbb475 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:28.090021 11082 raft_consensus.cc:729] T fe91b39f91274a7ca61e9618d8cbb475 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:28.100183 10655 tablet_service.cc:1467] Processing CreateTablet for tablet a58cf1dc81454710a9ad1a8baf155c9f (DEFAULT_TABLE table=default.Table_0478cdcfe9d64b66ba0e3fbc638f9a3b [id=58904eafabaa40eeb24a79dba89ae346]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:28.101261 10655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a58cf1dc81454710a9ad1a8baf155c9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:28.103586 10664 tablet_service.cc:1467] Processing CreateTablet for tablet 139fe14ba82a4137b59115b6d592a7c4 (DEFAULT_TABLE table=default.Table_ec0c08cc69f14fe5b33b93b0ca637da2 [id=f0092641ff814f0696f88a32ade365df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:28.104453 10664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 139fe14ba82a4137b59115b6d592a7c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:28.122486 11919 raft_consensus.cc:482] T a44d9f2f7e3e48aeb0f31fa29ae1988a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:28.122915 11919 raft_consensus.cc:504] T a44d9f2f7e3e48aeb0f31fa29ae1988a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:28.125279 11919 leader_election.cc:288] T a44d9f2f7e3e48aeb0f31fa29ae1988a P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:28.126458 10661 tablet_service.cc:1467] Processing CreateTablet for tablet 2d79538d6474433fbda26c91645a0619 (DEFAULT_TABLE table=default.Table_7af6fc9b30f54185a51b24ba24892166 [id=64e1495a3bde44a7b8fbc18d254293d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:28.137838 10952 tablet_service.cc:1467] Processing CreateTablet for tablet 139fe14ba82a4137b59115b6d592a7c4 (DEFAULT_TABLE table=default.Table_ec0c08cc69f14fe5b33b93b0ca637da2 [id=f0092641ff814f0696f88a32ade365df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:28.138830 10391 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 37bab60afe7a4c758a08ef7c9c24aba0
I20260925 09:57:28.139163 10952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 139fe14ba82a4137b59115b6d592a7c4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:28.140228 10619 leader_election.cc:341] T a44d9f2f7e3e48aeb0f31fa29ae1988a P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: a44d9f2f7e3e48aeb0f31fa29ae1988a
I20260925 09:57:28.140333 10391 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:35140 (request call id 266) took 1291 ms. Trace:
I20260925 09:57:28.140609 10391 rpcz_store.cc:276] 0925 09:57:26.848941 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:26.849067 (+ 126us) service_pool.cc:227] Handling call
0925 09:57:26.863474 (+ 14407us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2adb466 00007fabf061cb34 00007fabf061cac2 00007fabf061c9f1 00007fabf061c96c 00007fabf061bc22 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:26.864426 (+ 952us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:57:26.865842 (+ 1416us) catalog_manager.cc:2752] Modifying in-memory table state
0925 09:57:26.865955 (+ 113us) catalog_manager.cc:2761] Locking tablets
0925 09:57:26.866905 (+ 950us) catalog_manager.cc:2775] Removing table and tablets from system table
0925 09:57:27.097768 (+230863us) catalog_manager.cc:2793] Removing table from by-name map
0925 09:57:27.097957 (+ 189us) catalog_manager.cc:2809] Committing in-memory state
0925 09:57:28.138736 (+1040779us) catalog_manager.cc:2813] Aborting table tasks
0925 09:57:28.140278 (+ 1542us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:27.005413 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:27.005743 (+ 330us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:27.005766 (+ 23us) write_op.cc:435] Acquired schema lock
0925 09:57:27.005779 (+ 13us) tablet.cc:660] Decoding operations
0925 09:57:27.006064 (+ 285us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:27.006104 (+ 40us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:27.006124 (+ 20us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:27.006364 (+ 240us) tablet.cc:699] Row locks acquired
0925 09:57:27.006377 (+ 13us) write_op.cc:260] PREPARE: finished
0925 09:57:27.006471 (+ 94us) write_op.cc:270] Start()
0925 09:57:27.006540 (+ 69us) write_op.cc:276] Timestamp: P: 1790330247006397 usec, L: 0
0925 09:57:27.006555 (+ 15us) op_driver.cc:331] REPLICATION: starting
0925 09:57:27.006959 (+ 404us) log.cc:842] Serialized 722 byte log entry
0925 09:57:27.092901 (+ 85942us) op_driver.cc:444] REPLICATION: finished
0925 09:57:27.094369 (+ 1468us) write_op.cc:301] APPLY: starting
0925 09:57:27.094423 (+ 54us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:27.094526 (+ 103us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:27.094618 (+ 92us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:27.094868 (+ 250us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:27.095015 (+ 147us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 09:57:27.095061 (+ 46us) write_op.cc:312] APPLY: finished
0925 09:57:27.095246 (+ 185us) log.cc:842] Serialized 29 byte log entry
0925 09:57:27.095374 (+ 128us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:27.095472 (+ 98us) write_op.cc:454] Released schema lock
0925 09:57:27.095504 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:27.095687 (+ 183us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:27.095717 (+ 30us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":30061312,"child_traces":[["op",{"apply.queue_time_us":1334,"apply.run_cpu_time_us":1539,"apply.run_wall_time_us":1538,"num_ops":2,"prepare.queue_time_us":133858,"prepare.run_cpu_time_us":1715,"prepare.run_wall_time_us":1714,"replication_time_us":86285}]]}
I20260925 09:57:28.145355 10954 tablet_service.cc:1467] Processing CreateTablet for tablet 2d79538d6474433fbda26c91645a0619 (DEFAULT_TABLE table=default.Table_7af6fc9b30f54185a51b24ba24892166 [id=64e1495a3bde44a7b8fbc18d254293d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:28.090588 11082 consensus_queue.cc:260] T fe91b39f91274a7ca61e9618d8cbb475 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:28.146463 10391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_19555c81edae449ba3e2c82cc089ea01"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:28.147017 10959 tablet_service.cc:1467] Processing CreateTablet for tablet f1d31dca90df414a83be7c9e404b8d30 (DEFAULT_TABLE table=default.Table_1f25b6a5838b40758e7645d5d47ad04e [id=584cddf4e10c46d8827735eb5bdbe5d2]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 09:57:28.147840 10391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_19555c81edae449ba3e2c82cc089ea01 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:28.104853 11892 raft_consensus.cc:482] T 96a80382bb7d4395a14ad9ae972f4a25 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:28.149822 11901 raft_consensus.cc:482] T 4e3120347c8e4a3eab57d6562ae40e8d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:28.150139 11892 raft_consensus.cc:504] T 96a80382bb7d4395a14ad9ae972f4a25 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:28.150251 11901 raft_consensus.cc:504] T 4e3120347c8e4a3eab57d6562ae40e8d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:28.152043 11892 leader_election.cc:288] T 96a80382bb7d4395a14ad9ae972f4a25 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:28.153347 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 219) took 3680 ms. Trace:
I20260925 09:57:28.153671 11727 rpcz_store.cc:276] 0925 09:57:24.472655 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:24.473654 (+ 999us) service_pool.cc:227] Handling call
0925 09:57:27.805595 (+3331941us) ts_tablet_manager.cc:1145] Deleting tablet 359090244c4748e8a2d45e3c77e89498
0925 09:57:27.829633 (+ 24038us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:27.967572 (+137939us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:28.024986 (+ 57414us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:28.153288 (+128302us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3328488}
I20260925 09:57:28.154320 10683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e3120347c8e4a3eab57d6562ae40e8d" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:28.154764 11727 tablet_replica.cc:330] T 28d60531970a4ab58ce5ef99cb3a96bf P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:28.155426 10678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96a80382bb7d4395a14ad9ae972f4a25" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:28.156821 10919 leader_election.cc:341] T 4e3120347c8e4a3eab57d6562ae40e8d P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:28.156965 10959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1d31dca90df414a83be7c9e404b8d30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:28.157816 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 359090244c4748e8a2d45e3c77e89498 (table default.Table_3f0e11249b90447595581ed433d20c5b [id=d0cfc580b2c64772b90d93e44f1fdb85]) successfully deleted
I20260925 09:57:28.152045 11901 leader_election.cc:288] T 4e3120347c8e4a3eab57d6562ae40e8d P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:28.167776 10962 tablet_service.cc:1514] Processing DeleteTablet for tablet 37bab60afe7a4c758a08ef7c9c24aba0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:26 UTC) from {username='slave'} at 127.0.0.1:53560
W20260925 09:57:28.168592 10919 leader_election.cc:341] T 96a80382bb7d4395a14ad9ae972f4a25 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:28.169552 10954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d79538d6474433fbda26c91645a0619. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:28.171319 10378 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet b3d9a3cf5bda4cf6bc1fa3fae1cb9288 (table default.Table_cc9f467e078c448f87392bf324b040a8 [id=4cefdaddfd6143468df4502cd35afac4], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet b3d9a3cf5bda4cf6bc1fa3fae1cb9288 consensus state has no leader
I20260925 09:57:28.172163 11919 raft_consensus.cc:482] T 60fdc299865446c89d3e5374d573029a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:28.172703 11919 raft_consensus.cc:504] T 60fdc299865446c89d3e5374d573029a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:28.175756 10667 tablet_service.cc:1514] Processing DeleteTablet for tablet 37bab60afe7a4c758a08ef7c9c24aba0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:26 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:28.175925 11919 leader_election.cc:288] T 60fdc299865446c89d3e5374d573029a P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:28.176800 11727 raft_consensus.cc:2229] T 28d60531970a4ab58ce5ef99cb3a96bf P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260925 09:57:28.177316 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 37bab60afe7a4c758a08ef7c9c24aba0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 37bab60afe7a4c758a08ef7c9c24aba0 already in progress: creating tablet
I20260925 09:57:28.177501 11727 raft_consensus.cc:2258] T 28d60531970a4ab58ce5ef99cb3a96bf P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:28.178844 10391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_9d3d93d30cc442bcbabe7edfbf665469"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:28.180348 10391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9d3d93d30cc442bcbabe7edfbf665469 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 09:57:28.182262 10619 leader_election.cc:341] T 60fdc299865446c89d3e5374d573029a P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 60fdc299865446c89d3e5374d573029a
I20260925 09:57:28.182994 11082 ts_tablet_manager.cc:1428] T fe91b39f91274a7ca61e9618d8cbb475 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.096s user 0.005s sys 0.001s
I20260925 09:57:28.186026 10387 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 3d652a76d96f4dbeafde15f3c16534a6 (table default.Table_be0f658aa41f4188bc16db37e96f4ad1 [id=73083c1560cb4db5a12964387cb49b73]). Expected version 1 got 0
I20260925 09:57:28.188835 10387 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 26fae3fbfe374f348a1729c21ec751b9 (table default.Table_0011387b45014de4b24b466f4cb907f7 [id=e3f7aa5e988c4b6c98be5cd3d465245d]). Expected version 1 got 0
I20260925 09:57:28.189884 10678 raft_consensus.cc:3037] T c2110d4ada2d45c2b07259374465a8df P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:28.199581 10661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d79538d6474433fbda26c91645a0619. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:28.200326 10962 tablet_service.cc:1467] Processing CreateTablet for tablet f0bf545f70ae4cd6a1ad2540c7158ecc (DEFAULT_TABLE table=default.Table_25edfe533c64421da7700ac5d60265c3 [id=1469f44b239d454caa97056bd5803255]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:28.201551 10962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0bf545f70ae4cd6a1ad2540c7158ecc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:28.202654 11082 tablet_bootstrap.cc:492] T c7cd1d6a603642afa2c8c737079961b5 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:28.224220 11901 raft_consensus.cc:482] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:28.224691 11901 raft_consensus.cc:504] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:28.226634 11901 leader_election.cc:288] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:28.227334 11082 tablet_bootstrap.cc:654] T c7cd1d6a603642afa2c8c737079961b5 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:28.191354 10387 catalog_manager.cc:5495] T 2d087d3e35814c90a03eccd90eeea86c P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:28.229420 11084 tablet_bootstrap.cc:492] T cf775c9d9b114ddcb97dc6c6eca8e0fc P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:28.229825 11084 ts_tablet_manager.cc:1397] T cf775c9d9b114ddcb97dc6c6eca8e0fc P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.375s user 0.014s sys 0.001s
I20260925 09:57:28.230053 10683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb9c51f7e7fd4c029be2a710cb9462eb" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:28.231564 10919 leader_election.cc:341] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:28.237149 11084 raft_consensus.cc:348] T cf775c9d9b114ddcb97dc6c6eca8e0fc P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:28.237835 11084 raft_consensus.cc:374] T cf775c9d9b114ddcb97dc6c6eca8e0fc P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:28.238086 11084 raft_consensus.cc:729] T cf775c9d9b114ddcb97dc6c6eca8e0fc P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:28.238772 11084 consensus_queue.cc:260] T cf775c9d9b114ddcb97dc6c6eca8e0fc P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:28.252436 11892 raft_consensus.cc:482] T 350904aa5957496d90a2aa2e637b591c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:28.252905 11892 raft_consensus.cc:504] T 350904aa5957496d90a2aa2e637b591c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:28.253379 11084 ts_tablet_manager.cc:1428] T cf775c9d9b114ddcb97dc6c6eca8e0fc P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.023s user 0.007s sys 0.000s
I20260925 09:57:28.265084 10391 master_service.cc:946] ReplaceTablet: received request to replace tablet b3d9a3cf5bda4cf6bc1fa3fae1cb9288 from {username='slave'} at 127.0.0.1:35140
I20260925 09:57:28.270000 11901 raft_consensus.cc:482] T 92814437d53143a79f99199f8ee3a742 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:28.262975 10393 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 26fae3fbfe374f348a1729c21ec751b9 (table default.Table_0011387b45014de4b24b466f4cb907f7 [id=e3f7aa5e988c4b6c98be5cd3d465245d]). Expected version 1 got 0
I20260925 09:57:28.292946 10393 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 3d652a76d96f4dbeafde15f3c16534a6 (table default.Table_be0f658aa41f4188bc16db37e96f4ad1 [id=73083c1560cb4db5a12964387cb49b73]). Expected version 1 got 0
I20260925 09:57:28.296881 10393 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet c2110d4ada2d45c2b07259374465a8df (table default.Table_44ddd064d56640a0a0e3dd8fd38344fd [id=8307778a24ee442b8f7e89c09652309e]). Expected version 1 got 0
I20260925 09:57:28.298661 10393 catalog_manager.cc:5495] T df006163f72743b594baf1a3efd1c771 P c43461c4ee854ae1ac52b873ccada318 reported cstate change: term changed from 0 to 1, leader changed from <none> to c43461c4ee854ae1ac52b873ccada318 (127.10.9.2). New cstate: current_term: 1 leader_uuid: "c43461c4ee854ae1ac52b873ccada318" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:28.309302 10391 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet b3d9a3cf5bda4cf6bc1fa3fae1cb9288
I20260925 09:57:28.320828 10655 tablet_service.cc:1514] Processing DeleteTablet for tablet b3d9a3cf5bda4cf6bc1fa3fae1cb9288 with delete_type TABLET_DATA_DELETED (replaced by tablet f46095351ea64812a7fdcfc79cd92c1f) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:28.322777 10391 catalog_manager.cc:6565] ReplaceTablet: tablet b3d9a3cf5bda4cf6bc1fa3fae1cb9288 deleted and replaced by tablet f46095351ea64812a7fdcfc79cd92c1f
W20260925 09:57:28.324115 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet b3d9a3cf5bda4cf6bc1fa3fae1cb9288 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b3d9a3cf5bda4cf6bc1fa3fae1cb9288 already in progress: creating tablet
I20260925 09:57:28.326251 10391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_7a34d8a020ed46dc8177c60c3acfb2ec"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:28.327621 10391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7a34d8a020ed46dc8177c60c3acfb2ec in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:28.339589 10667 tablet_service.cc:1467] Processing CreateTablet for tablet f0bf545f70ae4cd6a1ad2540c7158ecc (DEFAULT_TABLE table=default.Table_25edfe533c64421da7700ac5d60265c3 [id=1469f44b239d454caa97056bd5803255]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:28.340818 10667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0bf545f70ae4cd6a1ad2540c7158ecc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:28.351419 11727 ts_tablet_manager.cc:1905] T 28d60531970a4ab58ce5ef99cb3a96bf P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:28.353889 11084 tablet_bootstrap.cc:492] T aca5987b14b64152887ac5ee584b257a P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:28.359067 11084 tablet_bootstrap.cc:654] T aca5987b14b64152887ac5ee584b257a P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:28.361202 11892 leader_election.cc:288] T 350904aa5957496d90a2aa2e637b591c P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:28.361804 11892 raft_consensus.cc:482] T a5d872c3620646e5ac33ceb95de2cbd4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:28.362380 11892 raft_consensus.cc:504] T a5d872c3620646e5ac33ceb95de2cbd4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:28.370949 11084 tablet_bootstrap.cc:492] T aca5987b14b64152887ac5ee584b257a P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:28.371374 11084 ts_tablet_manager.cc:1397] T aca5987b14b64152887ac5ee584b257a P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.018s user 0.010s sys 0.001s
I20260925 09:57:28.373816 11084 raft_consensus.cc:348] T aca5987b14b64152887ac5ee584b257a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:28.374473 11084 raft_consensus.cc:374] T aca5987b14b64152887ac5ee584b257a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:28.374737 11084 raft_consensus.cc:729] T aca5987b14b64152887ac5ee584b257a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:28.376775 10391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_926b545b446b47679a2c440a43597ebc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:28.378185 10391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_926b545b446b47679a2c440a43597ebc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:28.391549 11901 raft_consensus.cc:504] T 92814437d53143a79f99199f8ee3a742 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:28.395685 10683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92814437d53143a79f99199f8ee3a742" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:28.398317 10678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "350904aa5957496d90a2aa2e637b591c" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:28.375306 11084 consensus_queue.cc:260] T aca5987b14b64152887ac5ee584b257a P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:28.402032 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e3120347c8e4a3eab57d6562ae40e8d" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:28.404556 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a44d9f2f7e3e48aeb0f31fa29ae1988a" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:28.406046 10391 master_service.cc:946] ReplaceTablet: received request to replace tablet abfc8f390d434c47a8c4990f8406c270 from {username='slave'} at 127.0.0.1:35140
I20260925 09:57:28.428356 11082 tablet_bootstrap.cc:492] T c7cd1d6a603642afa2c8c737079961b5 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:28.430420 12019 raft_consensus.cc:482] T 977a59175e5b43eeb6f23b938afd00a3 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:28.433259 12019 raft_consensus.cc:504] T 977a59175e5b43eeb6f23b938afd00a3 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:28.436065 12019 leader_election.cc:288] T 977a59175e5b43eeb6f23b938afd00a3 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:28.436782 11082 ts_tablet_manager.cc:1397] T c7cd1d6a603642afa2c8c737079961b5 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.234s user 0.015s sys 0.000s
I20260925 09:57:28.438601 10812 tablet_service.cc:1467] Processing CreateTablet for tablet 139fe14ba82a4137b59115b6d592a7c4 (DEFAULT_TABLE table=default.Table_ec0c08cc69f14fe5b33b93b0ca637da2 [id=f0092641ff814f0696f88a32ade365df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:28.439774 10812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 139fe14ba82a4137b59115b6d592a7c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:28.463635 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60fdc299865446c89d3e5374d573029a" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:28.464349 10839 raft_consensus.cc:2454] T 60fdc299865446c89d3e5374d573029a P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:28.466068 10834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96a80382bb7d4395a14ad9ae972f4a25" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:28.482118 11085 tablet_bootstrap.cc:654] T a44d9f2f7e3e48aeb0f31fa29ae1988a P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:28.490726 10834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "977a59175e5b43eeb6f23b938afd00a3" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:28.509424 10810 tablet_service.cc:1514] Processing DeleteTablet for tablet b3d9a3cf5bda4cf6bc1fa3fae1cb9288 with delete_type TABLET_DATA_DELETED (replaced by tablet f46095351ea64812a7fdcfc79cd92c1f) from {username='slave'} at 127.0.0.1:33344
W20260925 09:57:28.509719 10621 leader_election.cc:341] T a44d9f2f7e3e48aeb0f31fa29ae1988a P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:28.510157 10621 leader_election.cc:302] T a44d9f2f7e3e48aeb0f31fa29ae1988a P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8850818e2ee42a082802d0095c08d21; no voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:28.512228 11981 raft_consensus.cc:2726] T a44d9f2f7e3e48aeb0f31fa29ae1988a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:28.512182 10807 tablet_service.cc:1467] Processing CreateTablet for tablet f0bf545f70ae4cd6a1ad2540c7158ecc (DEFAULT_TABLE table=default.Table_25edfe533c64421da7700ac5d60265c3 [id=1469f44b239d454caa97056bd5803255]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:28.513370 10807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0bf545f70ae4cd6a1ad2540c7158ecc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:28.516469 10806 tablet_service.cc:1467] Processing CreateTablet for tablet 2d79538d6474433fbda26c91645a0619 (DEFAULT_TABLE table=default.Table_7af6fc9b30f54185a51b24ba24892166 [id=64e1495a3bde44a7b8fbc18d254293d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:28.517752 10806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d79538d6474433fbda26c91645a0619. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:28.527542 10621 leader_election.cc:302] T 60fdc299865446c89d3e5374d573029a P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters: be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:28.528967 12028 raft_consensus.cc:2781] T 60fdc299865446c89d3e5374d573029a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:28.529349 12028 raft_consensus.cc:482] T 60fdc299865446c89d3e5374d573029a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:28.529664 12028 raft_consensus.cc:3037] T 60fdc299865446c89d3e5374d573029a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:28.563110 11082 raft_consensus.cc:348] T c7cd1d6a603642afa2c8c737079961b5 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:28.564107 11082 raft_consensus.cc:374] T c7cd1d6a603642afa2c8c737079961b5 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:28.564440 11082 raft_consensus.cc:729] T c7cd1d6a603642afa2c8c737079961b5 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:28.565245 11082 consensus_queue.cc:260] T c7cd1d6a603642afa2c8c737079961b5 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:28.568264 10621 leader_election.cc:341] T 977a59175e5b43eeb6f23b938afd00a3 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:28.561093 10815 tablet_service.cc:1514] Processing DeleteTablet for tablet 37bab60afe7a4c758a08ef7c9c24aba0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:26 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:28.422101 11084 ts_tablet_manager.cc:1428] T aca5987b14b64152887ac5ee584b257a P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.050s user 0.005s sys 0.000s
W20260925 09:57:28.399643 10919 leader_election.cc:341] T 92814437d53143a79f99199f8ee3a742 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:28.616374 10919 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 216755us
W20260925 09:57:28.616955 10919 leader_election.cc:341] T 350904aa5957496d90a2aa2e637b591c P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:28.616955 11084 tablet_bootstrap.cc:492] T 61aa6380d30a4b21b70ee2d321247bd8 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:28.394881 11901 leader_election.cc:288] T 92814437d53143a79f99199f8ee3a742 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:28.629840 10655 tablet_service.cc:1467] Processing CreateTablet for tablet bbcfc4dca399496fa41105eff0c71c3b (DEFAULT_TABLE table=default.Table_19555c81edae449ba3e2c82cc089ea01 [id=dc7e88f34db14952a32655a01690bf6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:28.630896 10655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbcfc4dca399496fa41105eff0c71c3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:28.634509 11084 tablet_bootstrap.cc:654] T 61aa6380d30a4b21b70ee2d321247bd8 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:28.645984 10391 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet abfc8f390d434c47a8c4990f8406c270
I20260925 09:57:28.647759 10391 catalog_manager.cc:6565] ReplaceTablet: tablet abfc8f390d434c47a8c4990f8406c270 deleted and replaced by tablet b025f8ba34694ba09cb01ca959d3629c
I20260925 09:57:28.648403 10661 tablet_service.cc:1514] Processing DeleteTablet for tablet abfc8f390d434c47a8c4990f8406c270 with delete_type TABLET_DATA_DELETED (replaced by tablet b025f8ba34694ba09cb01ca959d3629c) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:28.649360 11084 tablet_bootstrap.cc:492] T 61aa6380d30a4b21b70ee2d321247bd8 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:28.649716 11084 ts_tablet_manager.cc:1397] T 61aa6380d30a4b21b70ee2d321247bd8 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.033s user 0.007s sys 0.005s
W20260925 09:57:28.650830 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet abfc8f390d434c47a8c4990f8406c270 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet abfc8f390d434c47a8c4990f8406c270 already in progress: creating tablet
I20260925 09:57:28.658093 10834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb9c51f7e7fd4c029be2a710cb9462eb" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:28.662715 10661 tablet_service.cc:1467] Processing CreateTablet for tablet 5d6ebe0c2ccc48a09b616816953d83fe (DEFAULT_TABLE table=default.Table_13782377553b4270abed8a7beae0a959 [id=bfeffc2083a6422ba2ecd971e45206fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:28.663631 10661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d6ebe0c2ccc48a09b616816953d83fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:28.665401 11892 leader_election.cc:288] T a5d872c3620646e5ac33ceb95de2cbd4 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:28.668282 11085 tablet_bootstrap.cc:492] T a44d9f2f7e3e48aeb0f31fa29ae1988a P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:28.679464 10663 tablet_service.cc:1467] Processing CreateTablet for tablet 30e04d90c962442cadf7bdec6f827dac (DEFAULT_TABLE table=default.Table_63486992aef8441d8596bf9a0e3b98a9 [id=e76f309fb4d041ccafa093f734a0803c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:28.680436 10663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30e04d90c962442cadf7bdec6f827dac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:28.670471 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "350904aa5957496d90a2aa2e637b591c" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:28.671023 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5d872c3620646e5ac33ceb95de2cbd4" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:28.677627 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92814437d53143a79f99199f8ee3a742" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:28.680454 11961 raft_consensus.cc:482] T 3b9cdad4f9144daea800553abd378e0b P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:28.685168 11961 raft_consensus.cc:504] T 3b9cdad4f9144daea800553abd378e0b P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:28.687016 11961 leader_election.cc:288] T 3b9cdad4f9144daea800553abd378e0b P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:28.669566 10834 raft_consensus.cc:3037] T 2d087d3e35814c90a03eccd90eeea86c P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:28.691329 10390 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet c2110d4ada2d45c2b07259374465a8df (table default.Table_44ddd064d56640a0a0e3dd8fd38344fd [id=8307778a24ee442b8f7e89c09652309e]). Expected version 1 got 0
I20260925 09:57:28.695338 12024 raft_consensus.cc:482] T 2d087d3e35814c90a03eccd90eeea86c P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:28.695788 12024 raft_consensus.cc:504] T 2d087d3e35814c90a03eccd90eeea86c P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:28.696022 10661 tablet_service.cc:1467] Processing CreateTablet for tablet d289c2fb23df499a8118de92d82f0d19 (DEFAULT_TABLE table=default.Table_ff6257b50da64b15b5f006b7f178ca4a [id=a9e004c9068643c0bf14928761061424]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:28.696921 10661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d289c2fb23df499a8118de92d82f0d19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:28.668658 11085 ts_tablet_manager.cc:1397] T a44d9f2f7e3e48aeb0f31fa29ae1988a P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.585s user 0.014s sys 0.001s
I20260925 09:57:28.703780 10664 tablet_service.cc:1467] Processing CreateTablet for tablet 572c83b4a7c84fdab1fdbb0fbdb0a53d (DEFAULT_TABLE table=default.Table_04c588671331474fac69a076bb87ba84 [id=b8b7198fdaa24d72ae13b7275685c555]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:28.703991 10655 tablet_service.cc:1467] Processing CreateTablet for tablet 25ac35fb43d34c9b85b8e61ad6e386a6 (DEFAULT_TABLE table=default.Table_bd0dc294c2c049e9b9c9eddce6772a44 [id=887ce3cafef64210bb69d424ed5dae8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:28.704985 10390 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 3d652a76d96f4dbeafde15f3c16534a6 (table default.Table_be0f658aa41f4188bc16db37e96f4ad1 [id=73083c1560cb4db5a12964387cb49b73]). Expected version 1 got 0
W20260925 09:57:28.705109 10918 leader_election.cc:341] T 4e3120347c8e4a3eab57d6562ae40e8d P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:28.705513 10918 leader_election.cc:302] T 4e3120347c8e4a3eab57d6562ae40e8d P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:28.705971 11085 raft_consensus.cc:348] T a44d9f2f7e3e48aeb0f31fa29ae1988a P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:28.706588 11085 raft_consensus.cc:374] T a44d9f2f7e3e48aeb0f31fa29ae1988a P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:28.706926 10655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25ac35fb43d34c9b85b8e61ad6e386a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:28.706868 11085 raft_consensus.cc:729] T a44d9f2f7e3e48aeb0f31fa29ae1988a P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:28.707542 10664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 572c83b4a7c84fdab1fdbb0fbdb0a53d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:28.696447 11084 raft_consensus.cc:348] T 61aa6380d30a4b21b70ee2d321247bd8 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:28.708042 11084 raft_consensus.cc:374] T 61aa6380d30a4b21b70ee2d321247bd8 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:28.708257 11084 raft_consensus.cc:729] T 61aa6380d30a4b21b70ee2d321247bd8 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:28.708875 11084 consensus_queue.cc:260] T 61aa6380d30a4b21b70ee2d321247bd8 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:28.717888 12019 raft_consensus.cc:482] T fe91b39f91274a7ca61e9618d8cbb475 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:28.718322 12019 raft_consensus.cc:504] T fe91b39f91274a7ca61e9618d8cbb475 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:28.719491 12028 raft_consensus.cc:504] T 60fdc299865446c89d3e5374d573029a P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:28.720010 12019 leader_election.cc:288] T fe91b39f91274a7ca61e9618d8cbb475 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:28.721269 12028 leader_election.cc:288] T 60fdc299865446c89d3e5374d573029a P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:28.725672 10835 raft_consensus.cc:3037] T 3b9cdad4f9144daea800553abd378e0b P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:28.721180 10390 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 5608c5f9b65b4da7b2fbb56417c92d88 (table default.Table_1b4a040b8d60467a869df4b51c09cf46 [id=e46d114e5d484b66bb15eb0416b5ce0d]). Expected version 1 got 0
I20260925 09:57:28.727280 10390 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 26fae3fbfe374f348a1729c21ec751b9 (table default.Table_0011387b45014de4b24b466f4cb907f7 [id=e3f7aa5e988c4b6c98be5cd3d465245d]). Expected version 1 got 0
I20260925 09:57:28.732367 10663 tablet_service.cc:1467] Processing CreateTablet for tablet d7459730813f486e9cb9aeffe29509a2 (DEFAULT_TABLE table=default.Table_94f22b074c71401b9899866c584a0190 [id=626d465d30ba46bb866f81913ee77770]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:28.733280 10663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7459730813f486e9cb9aeffe29509a2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:28.733095 10919 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 88928us
I20260925 09:57:28.734611 10683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5d872c3620646e5ac33ceb95de2cbd4" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:28.736560 10658 tablet_service.cc:1467] Processing CreateTablet for tablet 086fc96ca247461db2ecd864139a228c (DEFAULT_TABLE table=default.Table_3f8e7f40931b4423bf4df247624ef2bb [id=7d28f94e106046a8b5be332be51f9905]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:28.737454 10658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 086fc96ca247461db2ecd864139a228c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:28.738131 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet b3d9a3cf5bda4cf6bc1fa3fae1cb9288 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b3d9a3cf5bda4cf6bc1fa3fae1cb9288 already in progress: creating tablet
W20260925 09:57:28.739342 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 37bab60afe7a4c758a08ef7c9c24aba0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 37bab60afe7a4c758a08ef7c9c24aba0 already in progress: creating tablet
I20260925 09:57:28.745065 12019 consensus_queue.cc:1035] T 2d087d3e35814c90a03eccd90eeea86c P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:28.745458 12030 consensus_queue.cc:1035] T 3b9cdad4f9144daea800553abd378e0b P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 09:57:28.746052 10619 leader_election.cc:341] T 977a59175e5b43eeb6f23b938afd00a3 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 977a59175e5b43eeb6f23b938afd00a3
I20260925 09:57:28.746541 10619 leader_election.cc:302] T 977a59175e5b43eeb6f23b938afd00a3 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8850818e2ee42a082802d0095c08d21; no voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:28.747532 12030 raft_consensus.cc:2726] T 977a59175e5b43eeb6f23b938afd00a3 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:28.748706 10918 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 43623.3us
W20260925 09:57:28.749316 10918 leader_election.cc:341] T 96a80382bb7d4395a14ad9ae972f4a25 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:28.749650 10918 leader_election.cc:302] T 96a80382bb7d4395a14ad9ae972f4a25 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:28.750821 10667 tablet_service.cc:1467] Processing CreateTablet for tablet 4a6b1284cddc4b2581931d8895212ca6 (DEFAULT_TABLE table=default.Table_4688c2ec003a44218fedb9f17dc0f5e3 [id=0e904cb2f51c411aa5fbdde8907a695d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:28.751816 10667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a6b1284cddc4b2581931d8895212ca6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:28.753990 12024 leader_election.cc:288] T 2d087d3e35814c90a03eccd90eeea86c P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:28.755035 10664 tablet_service.cc:1467] Processing CreateTablet for tablet 2015c10f503e4751ad9ff77404fca50b (DEFAULT_TABLE table=default.Table_fad50b13f9024bf8b4df345241f8c32b [id=351cf96e6f86425cb5077c576bc006db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:28.760685 10664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2015c10f503e4751ad9ff77404fca50b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:28.762244 10384 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5608c5f9b65b4da7b2fbb56417c92d88 consensus state has no leader
I20260925 09:57:28.762707 10384 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:35140 (request call id 272) took 1576 ms. Trace:
I20260925 09:57:28.763013 10384 rpcz_store.cc:276] 0925 09:57:27.186727 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:27.186868 (+ 141us) service_pool.cc:227] Handling call
0925 09:57:27.187765 (+ 897us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:57:27.188438 (+ 673us) catalog_manager.cc:3616] Apply alter schema
0925 09:57:27.189007 (+ 569us) catalog_manager.cc:3639] Acquired catalog manager lock
0925 09:57:27.189178 (+ 171us) catalog_manager.cc:3786] Updating metadata on disk
0925 09:57:27.268319 (+ 79141us) catalog_manager.cc:3817] Committing alterations to in-memory state
0925 09:57:28.762656 (+1494337us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:27.221266 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:27.221616 (+ 350us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:27.221637 (+ 21us) write_op.cc:435] Acquired schema lock
0925 09:57:27.221648 (+ 11us) tablet.cc:660] Decoding operations
0925 09:57:27.230345 (+ 8697us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:27.230382 (+ 37us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:27.230404 (+ 22us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:27.230537 (+ 133us) tablet.cc:699] Row locks acquired
0925 09:57:27.230550 (+ 13us) write_op.cc:260] PREPARE: finished
0925 09:57:27.230651 (+ 101us) write_op.cc:270] Start()
0925 09:57:27.230723 (+ 72us) write_op.cc:276] Timestamp: P: 1790330247230606 usec, L: 0
0925 09:57:27.230739 (+ 16us) op_driver.cc:331] REPLICATION: starting
0925 09:57:27.231209 (+ 470us) log.cc:842] Serialized 405 byte log entry
0925 09:57:27.256186 (+ 24977us) op_driver.cc:444] REPLICATION: finished
0925 09:57:27.258284 (+ 2098us) write_op.cc:301] APPLY: starting
0925 09:57:27.258357 (+ 73us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:27.258435 (+ 78us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:27.258527 (+ 92us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:27.258652 (+ 125us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:27.258758 (+ 106us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:27.258795 (+ 37us) write_op.cc:312] APPLY: finished
0925 09:57:27.258943 (+ 148us) log.cc:842] Serialized 23 byte log entry
0925 09:57:27.259073 (+ 130us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:27.259121 (+ 48us) write_op.cc:454] Released schema lock
0925 09:57:27.259147 (+ 26us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:27.259294 (+ 147us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:27.259327 (+ 33us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1990,"apply.run_cpu_time_us":1222,"apply.run_wall_time_us":10282,"num_ops":1,"prepare.queue_time_us":31088,"prepare.run_cpu_time_us":1727,"prepare.run_wall_time_us":10133,"replication_time_us":25384}]]}
I20260925 09:57:28.768092 10384 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_4f4ab1f0170a42e2826f1e3143803a6b" } modify_external_catalogs: true
I20260925 09:57:28.768728 10384 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_4f4ab1f0170a42e2826f1e3143803a6b" } modify_external_catalogs: true
I20260925 09:57:28.776674 10683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b9cdad4f9144daea800553abd378e0b" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:28.778250 10683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d087d3e35814c90a03eccd90eeea86c" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:28.783564 11727 ts_tablet_manager.cc:1918] T 28d60531970a4ab58ce5ef99cb3a96bf P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:57:28.783954 11727 log.cc:1196] T 28d60531970a4ab58ce5ef99cb3a96bf P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/28d60531970a4ab58ce5ef99cb3a96bf
I20260925 09:57:28.785681 10384 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet e9faf70be8ac43c79305980d685a28eb
I20260925 09:57:28.784415 11892 raft_consensus.cc:2726] T 4e3120347c8e4a3eab57d6562ae40e8d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:28.793412 11727 ts_tablet_manager.cc:1939] T 28d60531970a4ab58ce5ef99cb3a96bf P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:28.795217 12021 raft_consensus.cc:2726] T 96a80382bb7d4395a14ad9ae972f4a25 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:28.804908 10384 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_14a20618504e490289d610b4f74cea84"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:28.782791 11961 consensus_queue.cc:1035] T df006163f72743b594baf1a3efd1c771 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:28.812439 10663 tablet_service.cc:1514] Processing DeleteTablet for tablet e9faf70be8ac43c79305980d685a28eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:28 UTC) from {username='slave'} at 127.0.0.1:37876
W20260925 09:57:28.813581 10384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_14a20618504e490289d610b4f74cea84 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:28.815480 11082 ts_tablet_manager.cc:1428] T c7cd1d6a603642afa2c8c737079961b5 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.378s user 0.006s sys 0.001s
I20260925 09:57:28.816452 11082 tablet_bootstrap.cc:492] T 2358acdaa5104e3eacbab8bf182b8f30 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:28.819942 10661 tablet_service.cc:1467] Processing CreateTablet for tablet d5f28d1c19eb4e849dfa6b0bf532b67e (DEFAULT_TABLE table=default.Table_548921f3f0fd49069b0c73f688f4fa36 [id=f769116d89a545ff977f369477811e7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:28.821076 11084 ts_tablet_manager.cc:1428] T 61aa6380d30a4b21b70ee2d321247bd8 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.171s user 0.006s sys 0.001s
I20260925 09:57:28.822119 11084 tablet_bootstrap.cc:492] T f62c62715c9b42de826c1a66fd431eab P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
W20260925 09:57:28.815675 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet e9faf70be8ac43c79305980d685a28eb because the tablet was not found. No further retry: Not found: Tablet not found: e9faf70be8ac43c79305980d685a28eb
I20260925 09:57:28.822000 10391 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 3d652a76d96f4dbeafde15f3c16534a6 (table default.Table_be0f658aa41f4188bc16db37e96f4ad1 [id=73083c1560cb4db5a12964387cb49b73]). Expected version 1 got 0
I20260925 09:57:28.829372 10663 tablet_service.cc:1467] Processing CreateTablet for tablet 476567ad4efa4043b6536f3f28d2ee28 (DEFAULT_TABLE table=default.Table_8f6c92e47621419db4bc9e3e2dd5ce12 [id=85a14c8ac1ab4bef8df5a1572d2b0e54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:28.830941 10391 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet c2110d4ada2d45c2b07259374465a8df (table default.Table_44ddd064d56640a0a0e3dd8fd38344fd [id=8307778a24ee442b8f7e89c09652309e]). Expected version 1 got 0
I20260925 09:57:28.834331 11901 raft_consensus.cc:482] T b61444c31dbf4882ac748f861ea66818 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:28.834731 11901 raft_consensus.cc:504] T b61444c31dbf4882ac748f861ea66818 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:28.836766 11901 leader_election.cc:288] T b61444c31dbf4882ac748f861ea66818 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:28.837286 11901 raft_consensus.cc:482] T 8aff53cfd4994c629a408ee8250280c5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:28.838711 11901 raft_consensus.cc:504] T 8aff53cfd4994c629a408ee8250280c5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:28.840092 10819 tablet_service.cc:1514] Processing DeleteTablet for tablet abfc8f390d434c47a8c4990f8406c270 with delete_type TABLET_DATA_DELETED (replaced by tablet b025f8ba34694ba09cb01ca959d3629c) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:28.840641 10815 tablet_service.cc:1467] Processing CreateTablet for tablet 5d6ebe0c2ccc48a09b616816953d83fe (DEFAULT_TABLE table=default.Table_13782377553b4270abed8a7beae0a959 [id=bfeffc2083a6422ba2ecd971e45206fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:28.841566 10815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d6ebe0c2ccc48a09b616816953d83fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:28.842751 11901 leader_election.cc:288] T 8aff53cfd4994c629a408ee8250280c5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:28.847065 10391 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 26fae3fbfe374f348a1729c21ec751b9 (table default.Table_0011387b45014de4b24b466f4cb907f7 [id=e3f7aa5e988c4b6c98be5cd3d465245d]). Expected version 1 got 0
I20260925 09:57:28.851253 10667 tablet_service.cc:1467] Processing CreateTablet for tablet bc1d7b25d6ba42bc91130927c32d4ae6 (DEFAULT_TABLE table=default.Table_7c70dabea5b347c8b156b4b1c72703eb [id=0f92f8e3fdcc49cc906a25117f2fb4da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:28.853410 10982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b9cdad4f9144daea800553abd378e0b" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:28.854048 10982 raft_consensus.cc:2379] T 3b9cdad4f9144daea800553abd378e0b P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c43461c4ee854ae1ac52b873ccada318 in current term 1: Already voted for candidate b8850818e2ee42a082802d0095c08d21 in this term.
I20260925 09:57:28.857730 11085 consensus_queue.cc:260] T a44d9f2f7e3e48aeb0f31fa29ae1988a P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:28.859269 10978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d087d3e35814c90a03eccd90eeea86c" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:28.859841 10978 raft_consensus.cc:2454] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:28.860131 11085 ts_tablet_manager.cc:1428] T a44d9f2f7e3e48aeb0f31fa29ae1988a P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.156s user 0.005s sys 0.000s
I20260925 09:57:28.860996 11085 tablet_bootstrap.cc:492] T c7cd1d6a603642afa2c8c737079961b5 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:28.866909 12030 consensus_queue.cc:1035] T 2d087d3e35814c90a03eccd90eeea86c P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 09:57:28.875730 10918 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 126433us
W20260925 09:57:28.876369 10918 leader_election.cc:341] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:28.876751 10918 leader_election.cc:302] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
W20260925 09:57:28.877638 10918 leader_election.cc:341] T 350904aa5957496d90a2aa2e637b591c P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:28.878859 10918 leader_election.cc:302] T 350904aa5957496d90a2aa2e637b591c P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
W20260925 09:57:28.880223 10918 leader_election.cc:341] T a5d872c3620646e5ac33ceb95de2cbd4 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:28.882105 10918 leader_election.cc:341] T 92814437d53143a79f99199f8ee3a742 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:28.878043 12039 raft_consensus.cc:2726] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:28.880623 12040 raft_consensus.cc:2726] T 350904aa5957496d90a2aa2e637b591c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:28.883704 12044 tablet.cc:1722] T c2110d4ada2d45c2b07259374465a8df P c43461c4ee854ae1ac52b873ccada318: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:28.884184 10918 leader_election.cc:302] T 92814437d53143a79f99199f8ee3a742 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:28.886921 12040 raft_consensus.cc:2726] T 92814437d53143a79f99199f8ee3a742 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:28.889619 10683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b61444c31dbf4882ac748f861ea66818" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:28.891908 10683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aff53cfd4994c629a408ee8250280c5" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:28.894019 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 228) took 3662 ms. Trace:
I20260925 09:57:28.894364 11727 rpcz_store.cc:276] 0925 09:57:25.231189 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:25.231310 (+ 121us) service_pool.cc:227] Handling call
0925 09:57:28.154603 (+2923293us) ts_tablet_manager.cc:1145] Deleting tablet 28d60531970a4ab58ce5ef99cb3a96bf
0925 09:57:28.154648 (+ 45us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:28.685264 (+530616us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:28.783315 (+ 98051us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:28.893962 (+110647us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":98566,"tablet-delete.queue_time_us":2922767}
I20260925 09:57:28.895607 11727 tablet_replica.cc:330] T 67ce2e3325214aa6bdc3347e9e8f82b0 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:28.896462 11727 raft_consensus.cc:2229] T 67ce2e3325214aa6bdc3347e9e8f82b0 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:28.913410 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b61444c31dbf4882ac748f861ea66818" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:28.916801 10391 catalog_manager.cc:5495] T 3ff4cf556eee494d8cd590884466f09f P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:28.924278 10391 catalog_manager.cc:5495] T 3b9cdad4f9144daea800553abd378e0b P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:28.925757 10819 tablet_service.cc:1467] Processing CreateTablet for tablet d289c2fb23df499a8118de92d82f0d19 (DEFAULT_TABLE table=default.Table_ff6257b50da64b15b5f006b7f178ca4a [id=a9e004c9068643c0bf14928761061424]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:28.926839 10819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d289c2fb23df499a8118de92d82f0d19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:28.945555 10385 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: default.Table_f8b067e4e4cf498bbf44d7bf24c29363 [id=23d14ed1d2d647fc8b3b5ef40c7dc0bb] alter complete (version 1)
I20260925 09:57:28.948308 10805 tablet_service.cc:1467] Processing CreateTablet for tablet bbcfc4dca399496fa41105eff0c71c3b (DEFAULT_TABLE table=default.Table_19555c81edae449ba3e2c82cc089ea01 [id=dc7e88f34db14952a32655a01690bf6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:28.949805 10805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbcfc4dca399496fa41105eff0c71c3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:28.951514 10392 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet b555eeba505740988c704df5c734e7c0
I20260925 09:57:28.953022 10392 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:35140 (request call id 265) took 2105 ms. Trace:
I20260925 09:57:28.953281 10392 rpcz_store.cc:276] 0925 09:57:26.848161 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:26.853406 (+ 5245us) service_pool.cc:227] Handling call
0925 09:57:26.868936 (+ 15530us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2adb466 00007fabf061cb34 00007fabf061cac2 00007fabf061c9f1 00007fabf061c96c 00007fabf061bc22 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:26.871327 (+ 2391us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:57:26.879000 (+ 7673us) catalog_manager.cc:2752] Modifying in-memory table state
0925 09:57:26.879137 (+ 137us) catalog_manager.cc:2761] Locking tablets
0925 09:57:26.974248 (+ 95111us) catalog_manager.cc:2775] Removing table and tablets from system table
0925 09:57:27.160096 (+185848us) catalog_manager.cc:2793] Removing table from by-name map
0925 09:57:27.160324 (+ 228us) catalog_manager.cc:2809] Committing in-memory state
0925 09:57:28.951425 (+1791101us) catalog_manager.cc:2813] Aborting table tasks
0925 09:57:28.952970 (+ 1545us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:27.042971 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:27.084982 (+ 42011us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:27.085008 (+ 26us) write_op.cc:435] Acquired schema lock
0925 09:57:27.085019 (+ 11us) tablet.cc:660] Decoding operations
0925 09:57:27.085348 (+ 329us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:27.085380 (+ 32us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:27.091484 (+ 6104us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:27.091787 (+ 303us) tablet.cc:699] Row locks acquired
0925 09:57:27.091805 (+ 18us) write_op.cc:260] PREPARE: finished
0925 09:57:27.091908 (+ 103us) write_op.cc:270] Start()
0925 09:57:27.091984 (+ 76us) write_op.cc:276] Timestamp: P: 1790330247091826 usec, L: 0
0925 09:57:27.092000 (+ 16us) op_driver.cc:331] REPLICATION: starting
0925 09:57:27.092474 (+ 474us) log.cc:842] Serialized 762 byte log entry
0925 09:57:27.158091 (+ 65617us) op_driver.cc:444] REPLICATION: finished
0925 09:57:27.158358 (+ 267us) write_op.cc:301] APPLY: starting
0925 09:57:27.158412 (+ 54us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:27.158518 (+ 106us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:27.158663 (+ 145us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:27.158934 (+ 271us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:27.159081 (+ 147us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 09:57:27.159126 (+ 45us) write_op.cc:312] APPLY: finished
0925 09:57:27.159316 (+ 190us) log.cc:842] Serialized 29 byte log entry
0925 09:57:27.159483 (+ 167us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:27.159547 (+ 64us) write_op.cc:454] Released schema lock
0925 09:57:27.159578 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:27.159752 (+ 174us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:27.159775 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":33899776,"child_traces":[["op",{"apply.queue_time_us":89,"apply.run_cpu_time_us":1625,"apply.run_wall_time_us":1625,"num_ops":2,"prepare.queue_time_us":66486,"prepare.run_cpu_time_us":2000,"prepare.run_wall_time_us":77734,"replication_time_us":66013,"spinlock_wait_cycles":25472}]]}
I20260925 09:57:28.963810 12030 consensus_queue.cc:1035] T 3b9cdad4f9144daea800553abd378e0b P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:28.966640 10655 tablet_service.cc:1467] Processing CreateTablet for tablet 815b4b1bd914433798dcb5deec6ec79c (DEFAULT_TABLE table=default.Table_bc338e8ca7174647948301512850ce00 [id=7d130f66062344458770b2b943b4485a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:28.966629 10666 tablet_service.cc:1514] Processing DeleteTablet for tablet b555eeba505740988c704df5c734e7c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:26 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:28.967029 10658 tablet_service.cc:1467] Processing CreateTablet for tablet 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef (DEFAULT_TABLE table=default.Table_40181c2d583b42b89dbb7e1f589f5317 [id=cda0a13cf8024d98942bd16e2b9ecbe8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:28.971580 10807 tablet_service.cc:1467] Processing CreateTablet for tablet 572c83b4a7c84fdab1fdbb0fbdb0a53d (DEFAULT_TABLE table=default.Table_04c588671331474fac69a076bb87ba84 [id=b8b7198fdaa24d72ae13b7275685c555]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:28.971903 10810 tablet_service.cc:1467] Processing CreateTablet for tablet d7459730813f486e9cb9aeffe29509a2 (DEFAULT_TABLE table=default.Table_94f22b074c71401b9899866c584a0190 [id=626d465d30ba46bb866f81913ee77770]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:28.972189 10816 tablet_service.cc:1467] Processing CreateTablet for tablet 086fc96ca247461db2ecd864139a228c (DEFAULT_TABLE table=default.Table_3f8e7f40931b4423bf4df247624ef2bb [id=7d28f94e106046a8b5be332be51f9905]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:28.972582 10807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 572c83b4a7c84fdab1fdbb0fbdb0a53d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:28.973084 10816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 086fc96ca247461db2ecd864139a228c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:28.973593 10810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7459730813f486e9cb9aeffe29509a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:28.967603 10392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_c91ec477f2e541c394b6cc12607a0d3e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:28.980950 10392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c91ec477f2e541c394b6cc12607a0d3e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:28.983618 10666 tablet_service.cc:1514] Processing DeleteTablet for tablet d9e454f82a4e4c429b58f32926319b7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:22 UTC) from {username='slave'} at 127.0.0.1:37876
W20260925 09:57:28.984066 10385 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.1:56563 (request call id 35) took 2630 ms (client timeout 1000 ms). Trace:
W20260925 09:57:28.987658 10621 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 119271us
I20260925 09:57:28.994135 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aff53cfd4994c629a408ee8250280c5" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:28.991964 10812 tablet_service.cc:1467] Processing CreateTablet for tablet 30e04d90c962442cadf7bdec6f827dac (DEFAULT_TABLE table=default.Table_63486992aef8441d8596bf9a0e3b98a9 [id=e76f309fb4d041ccafa093f734a0803c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:29.005175 10812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30e04d90c962442cadf7bdec6f827dac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:29.012625 10807 tablet_service.cc:1467] Processing CreateTablet for tablet 4a6b1284cddc4b2581931d8895212ca6 (DEFAULT_TABLE table=default.Table_4688c2ec003a44218fedb9f17dc0f5e3 [id=0e904cb2f51c411aa5fbdde8907a695d]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 09:57:29.017982 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet abfc8f390d434c47a8c4990f8406c270 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet abfc8f390d434c47a8c4990f8406c270 already in progress: creating tablet
I20260925 09:57:28.899850 12039 raft_consensus.cc:482] T 69a96874eea1409a9a5a06b5787fbd99 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:29.019047 11727 raft_consensus.cc:2258] T 67ce2e3325214aa6bdc3347e9e8f82b0 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:29.018983 12039 raft_consensus.cc:504] T 69a96874eea1409a9a5a06b5787fbd99 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:29.021279 11727 ts_tablet_manager.cc:1905] T 67ce2e3325214aa6bdc3347e9e8f82b0 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:29.013043 10810 tablet_service.cc:1467] Processing CreateTablet for tablet 2015c10f503e4751ad9ff77404fca50b (DEFAULT_TABLE table=default.Table_fad50b13f9024bf8b4df345241f8c32b [id=351cf96e6f86425cb5077c576bc006db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:29.014415 10811 tablet_service.cc:1514] Processing DeleteTablet for tablet e9faf70be8ac43c79305980d685a28eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:28 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:29.024420 10810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2015c10f503e4751ad9ff77404fca50b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:29.016340 10808 tablet_service.cc:1467] Processing CreateTablet for tablet d5f28d1c19eb4e849dfa6b0bf532b67e (DEFAULT_TABLE table=default.Table_548921f3f0fd49069b0c73f688f4fa36 [id=f769116d89a545ff977f369477811e7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:29.026327 10808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5f28d1c19eb4e849dfa6b0bf532b67e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:29.030144 10807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a6b1284cddc4b2581931d8895212ca6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:29.017696 10813 tablet_service.cc:1467] Processing CreateTablet for tablet 476567ad4efa4043b6536f3f28d2ee28 (DEFAULT_TABLE table=default.Table_8f6c92e47621419db4bc9e3e2dd5ce12 [id=85a14c8ac1ab4bef8df5a1572d2b0e54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:29.031437 10390 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: default.Table_0d7d27d05491401089db5f65b43e86f7 [id=a20216b9b3074e408bee74e39c5819b4] alter complete (version 1)
I20260925 09:57:29.032411 10813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 476567ad4efa4043b6536f3f28d2ee28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:29.018957 10809 tablet_service.cc:1467] Processing CreateTablet for tablet bc1d7b25d6ba42bc91130927c32d4ae6 (DEFAULT_TABLE table=default.Table_7c70dabea5b347c8b156b4b1c72703eb [id=0f92f8e3fdcc49cc906a25117f2fb4da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:29.036420 10809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc1d7b25d6ba42bc91130927c32d4ae6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:29.020123 10818 tablet_service.cc:1467] Processing CreateTablet for tablet 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef (DEFAULT_TABLE table=default.Table_40181c2d583b42b89dbb7e1f589f5317 [id=cda0a13cf8024d98942bd16e2b9ecbe8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:29.039989 10818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:29.021297 10814 tablet_service.cc:1467] Processing CreateTablet for tablet 815b4b1bd914433798dcb5deec6ec79c (DEFAULT_TABLE table=default.Table_bc338e8ca7174647948301512850ce00 [id=7d130f66062344458770b2b943b4485a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:29.043995 10814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 815b4b1bd914433798dcb5deec6ec79c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:29.021993 10817 tablet_service.cc:1514] Processing DeleteTablet for tablet b555eeba505740988c704df5c734e7c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:26 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:29.048137 12027 tablet_replica.cc:330] T b555eeba505740988c704df5c734e7c0 P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:29.049299 12027 raft_consensus.cc:2229] T b555eeba505740988c704df5c734e7c0 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:29.049880 12027 raft_consensus.cc:2258] T b555eeba505740988c704df5c734e7c0 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:29.064443 12027 ts_tablet_manager.cc:1905] T b555eeba505740988c704df5c734e7c0 P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 09:57:29.059376 10393 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.2:40761 (request call id 34) took 1427 ms (client timeout 1000 ms). Trace:
I20260925 09:57:28.999143 11082 tablet_bootstrap.cc:654] T 2358acdaa5104e3eacbab8bf182b8f30 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:29.066214 10666 tablet_service.cc:1514] Processing DeleteTablet for tablet aba253f22afa4200926f31a1b43e3970 with delete_type TABLET_DATA_DELETED (replaced by tablet 815b4b1bd914433798dcb5deec6ec79c) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:29.112413 10657 tablet_service.cc:1467] Processing CreateTablet for tablet e329410c044c40a4933550730f46001e (DEFAULT_TABLE table=default.Table_a5e309b8633e4779a51120c48f91df43 [id=c276a09404734f019e9773a244bcb199]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:29.116122 10659 tablet_service.cc:1467] Processing CreateTablet for tablet e9faf70be8ac43c79305980d685a28eb (DEFAULT_TABLE table=default.Table_4f4ab1f0170a42e2826f1e3143803a6b [id=86ea83e3afc04657ad2aff23057b36d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:29.132933 10661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5f28d1c19eb4e849dfa6b0bf532b67e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:29.139432 10663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 476567ad4efa4043b6536f3f28d2ee28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:29.140012 10655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 815b4b1bd914433798dcb5deec6ec79c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:29.140534 11709 tablet_replica.cc:330] T b555eeba505740988c704df5c734e7c0 P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:29.142966 10658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:29.143532 10386 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.2:40761 (request call id 33) took 2622 ms (client timeout 1000 ms). Trace:
I20260925 09:57:29.143703 10667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc1d7b25d6ba42bc91130927c32d4ae6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:29.144198 10659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9faf70be8ac43c79305980d685a28eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:29.144691 11709 raft_consensus.cc:2229] T b555eeba505740988c704df5c734e7c0 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:29.145191 11709 raft_consensus.cc:2258] T b555eeba505740988c704df5c734e7c0 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 09:57:29.143862 10386 rpcz_store.cc:269] 0925 09:57:26.520940 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:26.559784 (+ 38844us) service_pool.cc:227] Handling call
0925 09:57:26.650113 (+ 90329us) spinlock_profiling.cc:243] Waited 81.7 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2ad9c3e 00007fabf0610fb4 00007fabf0610f42 00007fabf0610e71 00007fabf0610dec 00007fabf06100a2 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:29.143459 (+2493346us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:27.212693 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:27.213072 (+ 379us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:27.213100 (+ 28us) write_op.cc:435] Acquired schema lock
0925 09:57:27.213117 (+ 17us) tablet.cc:660] Decoding operations
0925 09:57:27.213388 (+ 271us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:27.213423 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:27.213445 (+ 22us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:27.213599 (+ 154us) tablet.cc:699] Row locks acquired
0925 09:57:27.213612 (+ 13us) write_op.cc:260] PREPARE: finished
0925 09:57:27.216937 (+ 3325us) write_op.cc:270] Start()
0925 09:57:27.217032 (+ 95us) write_op.cc:276] Timestamp: P: 1790330247216880 usec, L: 0
0925 09:57:27.217048 (+ 16us) op_driver.cc:331] REPLICATION: starting
0925 09:57:27.220959 (+ 3911us) log.cc:842] Serialized 524 byte log entry
0925 09:57:27.255984 (+ 35025us) op_driver.cc:444] REPLICATION: finished
0925 09:57:27.256867 (+ 883us) write_op.cc:301] APPLY: starting
0925 09:57:27.256921 (+ 54us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:27.257005 (+ 84us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:27.257104 (+ 99us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:27.257277 (+ 173us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:27.257408 (+ 131us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:27.257446 (+ 38us) write_op.cc:312] APPLY: finished
0925 09:57:27.257613 (+ 167us) log.cc:842] Serialized 23 byte log entry
0925 09:57:27.257755 (+ 142us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:27.257813 (+ 58us) write_op.cc:454] Released schema lock
0925 09:57:27.257844 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:27.258010 (+ 166us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:27.258035 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":23544,"reported_tablets":31,"spinlock_wait_cycles":187863936,"child_traces":[["op",{"apply.queue_time_us":734,"apply.run_cpu_time_us":1315,"apply.run_wall_time_us":1315,"num_ops":1,"prepare.queue_time_us":30815,"prepare.run_cpu_time_us":1896,"prepare.run_wall_time_us":8471,"replication_time_us":38856}]]}
I20260925 09:57:29.146787 11709 ts_tablet_manager.cc:1905] T b555eeba505740988c704df5c734e7c0 P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:29.147644 10657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e329410c044c40a4933550730f46001e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:28.984423 10385 rpcz_store.cc:269] 0925 09:57:26.353276 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:26.353428 (+ 152us) service_pool.cc:227] Handling call
0925 09:57:26.376209 (+ 22781us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2ad9c3e 00007fabf0610fb4 00007fabf0610f42 00007fabf0610e71 00007fabf0610dec 00007fabf06100a2 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:28.984001 (+2607792us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:26.930364 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:26.930740 (+ 376us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:26.930764 (+ 24us) write_op.cc:435] Acquired schema lock
0925 09:57:26.930778 (+ 14us) tablet.cc:660] Decoding operations
0925 09:57:26.931095 (+ 317us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:26.931138 (+ 43us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:26.931163 (+ 25us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:26.931425 (+ 262us) tablet.cc:699] Row locks acquired
0925 09:57:26.931439 (+ 14us) write_op.cc:260] PREPARE: finished
0925 09:57:26.931550 (+ 111us) write_op.cc:270] Start()
0925 09:57:26.931630 (+ 80us) write_op.cc:276] Timestamp: P: 1790330246931465 usec, L: 0
0925 09:57:26.931645 (+ 15us) op_driver.cc:331] REPLICATION: starting
0925 09:57:26.962095 (+ 30450us) log.cc:842] Serialized 895 byte log entry
0925 09:57:26.967906 (+ 5811us) op_driver.cc:444] REPLICATION: finished
0925 09:57:26.968158 (+ 252us) write_op.cc:301] APPLY: starting
0925 09:57:26.968208 (+ 50us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:26.968307 (+ 99us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:26.968457 (+ 150us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:26.968750 (+ 293us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:26.968886 (+ 136us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 09:57:26.968931 (+ 45us) write_op.cc:312] APPLY: finished
0925 09:57:26.969090 (+ 159us) log.cc:842] Serialized 29 byte log entry
0925 09:57:26.969213 (+ 123us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:26.969279 (+ 66us) write_op.cc:454] Released schema lock
0925 09:57:26.969313 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:26.969483 (+ 170us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:26.969506 (+ 23us) write_op.cc:454] Released schema lock
Related trace 'op':
0925 09:57:27.120912 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:27.121270 (+ 358us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:27.121303 (+ 33us) write_op.cc:435] Acquired schema lock
0925 09:57:27.121316 (+ 13us) tablet.cc:660] Decoding operations
0925 09:57:27.121568 (+ 252us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:27.121604 (+ 36us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:27.121625 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:27.134724 (+ 13099us) tablet.cc:699] Row locks acquired
0925 09:57:27.134740 (+ 16us) write_op.cc:260] PREPARE: finished
0925 09:57:27.134854 (+ 114us) write_op.cc:270] Start()
0925 09:57:27.134937 (+ 83us) write_op.cc:276] Timestamp: P: 1790330247134812 usec, L: 0
0925 09:57:27.134953 (+ 16us) op_driver.cc:331] REPLICATION: starting
0925 09:57:27.137673 (+ 2720us) log.cc:842] Serialized 356 byte log entry
0925 09:57:27.191446 (+ 53773us) op_driver.cc:444] REPLICATION: finished
0925 09:57:27.192111 (+ 665us) write_op.cc:301] APPLY: starting
0925 09:57:27.192159 (+ 48us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:27.192247 (+ 88us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:27.192338 (+ 91us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:27.192523 (+ 185us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:27.192646 (+ 123us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:27.192689 (+ 43us) write_op.cc:312] APPLY: finished
0925 09:57:27.192849 (+ 160us) log.cc:842] Serialized 23 byte log entry
0925 09:57:27.192998 (+ 149us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:27.193054 (+ 56us) write_op.cc:454] Released schema lock
0925 09:57:27.193088 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:27.193256 (+ 168us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:27.193278 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":31662,"reported_tablets":18,"spinlock_wait_cycles":26781440,"child_traces":[["op",{"apply.queue_time_us":126,"apply.run_cpu_time_us":1519,"apply.run_wall_time_us":1518,"num_ops":2,"prepare.queue_time_us":210693,"prepare.run_cpu_time_us":2220,"prepare.run_wall_time_us":32074,"raft.queue_time_us":87,"raft.run_cpu_time_us":668,"raft.run_wall_time_us":1428,"replication_time_us":36200,"spinlock_wait_cycles":67584}],["op",{"apply.queue_time_us":538,"apply.run_cpu_time_us":1335,"apply.run_wall_time_us":1334,"num_ops":1,"prepare.queue_time_us":140668,"prepare.run_cpu_time_us":1893,"prepare.run_wall_time_us":17000,"replication_time_us":56399}]]}
W20260925 09:57:29.066874 10393 rpcz_store.cc:269] 0925 09:57:27.630839 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:27.630948 (+ 109us) service_pool.cc:227] Handling call
0925 09:57:29.057925 (+1426977us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:28.330398 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:28.330706 (+ 308us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:28.330728 (+ 22us) write_op.cc:435] Acquired schema lock
0925 09:57:28.330739 (+ 11us) tablet.cc:660] Decoding operations
0925 09:57:28.330997 (+ 258us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:28.331035 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:28.331055 (+ 20us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:28.331180 (+ 125us) tablet.cc:699] Row locks acquired
0925 09:57:28.331193 (+ 13us) write_op.cc:260] PREPARE: finished
0925 09:57:28.331292 (+ 99us) write_op.cc:270] Start()
0925 09:57:28.331368 (+ 76us) write_op.cc:276] Timestamp: P: 1790330248331243 usec, L: 0
0925 09:57:28.331384 (+ 16us) op_driver.cc:331] REPLICATION: starting
0925 09:57:28.331843 (+ 459us) log.cc:842] Serialized 524 byte log entry
0925 09:57:28.337831 (+ 5988us) op_driver.cc:444] REPLICATION: finished
0925 09:57:28.338290 (+ 459us) write_op.cc:301] APPLY: starting
0925 09:57:28.338336 (+ 46us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:28.338423 (+ 87us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:28.338502 (+ 79us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:28.338641 (+ 139us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:28.338752 (+ 111us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:28.338789 (+ 37us) write_op.cc:312] APPLY: finished
0925 09:57:28.338931 (+ 142us) log.cc:842] Serialized 23 byte log entry
0925 09:57:28.339040 (+ 109us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:28.339096 (+ 56us) write_op.cc:454] Released schema lock
0925 09:57:28.339122 (+ 26us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:28.339281 (+ 159us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:28.339309 (+ 28us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":31,"child_traces":[["op",{"apply.queue_time_us":302,"apply.run_cpu_time_us":1211,"apply.run_wall_time_us":1211,"num_ops":1,"prepare.queue_time_us":5392,"prepare.run_cpu_time_us":1699,"prepare.run_wall_time_us":1699,"raft.queue_time_us":142,"raft.run_cpu_time_us":558,"raft.run_wall_time_us":558,"replication_time_us":6383}]]}
I20260925 09:57:29.165556 10393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_47277d736fdb436a9d4e8c03f5ec80a9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:28.908154 12040 raft_consensus.cc:482] T cf775c9d9b114ddcb97dc6c6eca8e0fc P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:29.204864 12040 raft_consensus.cc:504] T cf775c9d9b114ddcb97dc6c6eca8e0fc P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:29.206928 12040 leader_election.cc:288] T cf775c9d9b114ddcb97dc6c6eca8e0fc P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:28.918752 10954 tablet_service.cc:1514] Processing DeleteTablet for tablet abfc8f390d434c47a8c4990f8406c270 with delete_type TABLET_DATA_DELETED (replaced by tablet b025f8ba34694ba09cb01ca959d3629c) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:28.919337 10959 tablet_service.cc:1467] Processing CreateTablet for tablet bbcfc4dca399496fa41105eff0c71c3b (DEFAULT_TABLE table=default.Table_19555c81edae449ba3e2c82cc089ea01 [id=dc7e88f34db14952a32655a01690bf6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:29.208393 10959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbcfc4dca399496fa41105eff0c71c3b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:29.208545 10393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_47277d736fdb436a9d4e8c03f5ec80a9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 09:57:29.156478 10387 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.1:56563 (request call id 36) took 1747 ms (client timeout 1000 ms). Trace:
I20260925 09:57:28.920364 10957 tablet_service.cc:1467] Processing CreateTablet for tablet 5d6ebe0c2ccc48a09b616816953d83fe (DEFAULT_TABLE table=default.Table_13782377553b4270abed8a7beae0a959 [id=bfeffc2083a6422ba2ecd971e45206fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:28.921317 10963 tablet_service.cc:1467] Processing CreateTablet for tablet 30e04d90c962442cadf7bdec6f827dac (DEFAULT_TABLE table=default.Table_63486992aef8441d8596bf9a0e3b98a9 [id=e76f309fb4d041ccafa093f734a0803c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:29.212070 10963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30e04d90c962442cadf7bdec6f827dac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:29.216009 10957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d6ebe0c2ccc48a09b616816953d83fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:28.923844 10958 tablet_service.cc:1514] Processing DeleteTablet for tablet b3d9a3cf5bda4cf6bc1fa3fae1cb9288 with delete_type TABLET_DATA_DELETED (replaced by tablet f46095351ea64812a7fdcfc79cd92c1f) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:28.929455 10961 tablet_service.cc:1467] Processing CreateTablet for tablet d289c2fb23df499a8118de92d82f0d19 (DEFAULT_TABLE table=default.Table_ff6257b50da64b15b5f006b7f178ca4a [id=a9e004c9068643c0bf14928761061424]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:29.220127 10961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d289c2fb23df499a8118de92d82f0d19. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:29.222641 10378 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet b3d9a3cf5bda4cf6bc1fa3fae1cb9288 (table default.Table_cc9f467e078c448f87392bf324b040a8 [id=4cefdaddfd6143468df4502cd35afac4], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet b3d9a3cf5bda4cf6bc1fa3fae1cb9288 consensus state has no leader
I20260925 09:57:28.932077 10953 tablet_service.cc:1467] Processing CreateTablet for tablet 25ac35fb43d34c9b85b8e61ad6e386a6 (DEFAULT_TABLE table=default.Table_bd0dc294c2c049e9b9c9eddce6772a44 [id=887ce3cafef64210bb69d424ed5dae8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:29.224138 10953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25ac35fb43d34c9b85b8e61ad6e386a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:28.933449 10956 tablet_service.cc:1467] Processing CreateTablet for tablet 572c83b4a7c84fdab1fdbb0fbdb0a53d (DEFAULT_TABLE table=default.Table_04c588671331474fac69a076bb87ba84 [id=b8b7198fdaa24d72ae13b7275685c555]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:29.228219 10956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 572c83b4a7c84fdab1fdbb0fbdb0a53d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:29.228039 10386 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_aad29b2508b744aaba08bf51b98cd527" } new_table_name: "default.Table_dfc8dd8209e3467b872784a505f0d78c" modify_external_catalogs: true
I20260925 09:57:29.201733 10385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_5bdcdd02d06948d6b95e9c2ea4c22471"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:29.229518 10385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5bdcdd02d06948d6b95e9c2ea4c22471 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:28.933943 10955 tablet_service.cc:1467] Processing CreateTablet for tablet 086fc96ca247461db2ecd864139a228c (DEFAULT_TABLE table=default.Table_3f8e7f40931b4423bf4df247624ef2bb [id=7d28f94e106046a8b5be332be51f9905]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:29.231801 10664 tablet_service.cc:1467] Processing CreateTablet for tablet c47783e4ea694561bf008def26655884 (DEFAULT_TABLE table=default.Table_6248142ab04c4496a3db73e70549b09b [id=87469318fa894ad18bf840b9be6160d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:29.232184 10955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 086fc96ca247461db2ecd864139a228c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:29.232682 10664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c47783e4ea694561bf008def26655884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:28.934458 10960 tablet_service.cc:1467] Processing CreateTablet for tablet d7459730813f486e9cb9aeffe29509a2 (DEFAULT_TABLE table=default.Table_94f22b074c71401b9899866c584a0190 [id=626d465d30ba46bb866f81913ee77770]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:29.235978 10960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7459730813f486e9cb9aeffe29509a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:29.237358 10389 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:35140 (request call id 264) took 2413 ms. Trace:
I20260925 09:57:28.936218 10951 tablet_service.cc:1467] Processing CreateTablet for tablet 4a6b1284cddc4b2581931d8895212ca6 (DEFAULT_TABLE table=default.Table_4688c2ec003a44218fedb9f17dc0f5e3 [id=0e904cb2f51c411aa5fbdde8907a695d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:29.240075 10951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a6b1284cddc4b2581931d8895212ca6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:29.237641 10389 rpcz_store.cc:276] 0925 09:57:26.824150 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:26.824286 (+ 136us) service_pool.cc:227] Handling call
0925 09:57:26.858994 (+ 34708us) spinlock_profiling.cc:243] Waited 33.9 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2adb7d6 00007fabf0621634 00007fabf06215c2 00007fabf06214f1 00007fabf062146c 00007fabf0620722 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:26.859676 (+ 682us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:57:26.860277 (+ 601us) catalog_manager.cc:3616] Apply alter schema
0925 09:57:26.860822 (+ 545us) catalog_manager.cc:3639] Acquired catalog manager lock
0925 09:57:26.860994 (+ 172us) catalog_manager.cc:3786] Updating metadata on disk
0925 09:57:26.982904 (+121910us) catalog_manager.cc:3817] Committing alterations to in-memory state
0925 09:57:29.237298 (+2254394us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:26.965076 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:26.965400 (+ 324us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:26.965419 (+ 19us) write_op.cc:435] Acquired schema lock
0925 09:57:26.965431 (+ 12us) tablet.cc:660] Decoding operations
0925 09:57:26.965650 (+ 219us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:26.965679 (+ 29us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:26.965698 (+ 19us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:26.965825 (+ 127us) tablet.cc:699] Row locks acquired
0925 09:57:26.965837 (+ 12us) write_op.cc:260] PREPARE: finished
0925 09:57:26.965925 (+ 88us) write_op.cc:270] Start()
0925 09:57:26.965991 (+ 66us) write_op.cc:276] Timestamp: P: 1790330246965858 usec, L: 0
0925 09:57:26.966005 (+ 14us) op_driver.cc:331] REPLICATION: starting
0925 09:57:26.966396 (+ 391us) log.cc:842] Serialized 405 byte log entry
0925 09:57:26.979019 (+ 12623us) op_driver.cc:444] REPLICATION: finished
0925 09:57:26.981279 (+ 2260us) write_op.cc:301] APPLY: starting
0925 09:57:26.981343 (+ 64us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:26.981514 (+ 171us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:26.981683 (+ 169us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:26.981988 (+ 305us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:26.982251 (+ 263us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:26.982333 (+ 82us) write_op.cc:312] APPLY: finished
0925 09:57:26.982508 (+ 175us) log.cc:842] Serialized 23 byte log entry
0925 09:57:26.982614 (+ 106us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:26.982690 (+ 76us) write_op.cc:454] Released schema lock
0925 09:57:26.982721 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:26.983149 (+ 428us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:26.983181 (+ 32us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":78078720,"child_traces":[["op",{"apply.queue_time_us":2112,"apply.run_cpu_time_us":1811,"apply.run_wall_time_us":2058,"num_ops":1,"prepare.queue_time_us":102885,"prepare.run_cpu_time_us":1473,"prepare.run_wall_time_us":1472,"replication_time_us":12960}]]}
I20260925 09:57:29.241345 10952 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53560 (request call id 270) took 1105 ms. Trace:
I20260925 09:57:29.241686 10952 rpcz_store.cc:276] 0925 09:57:28.135499 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:28.136921 (+ 1422us) service_pool.cc:227] Handling call
0925 09:57:28.138619 (+ 1698us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0925 09:57:28.138726 (+ 107us) ts_tablet_manager.cc:676] Creating new metadata...
0925 09:57:28.823568 (+684842us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:29.240750 (+417182us) tablet_replica.cc:205] Creating consensus instance
0925 09:57:29.241206 (+ 456us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260925 09:57:28.944558 10950 tablet_service.cc:1467] Processing CreateTablet for tablet 2015c10f503e4751ad9ff77404fca50b (DEFAULT_TABLE table=default.Table_fad50b13f9024bf8b4df345241f8c32b [id=351cf96e6f86425cb5077c576bc006db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:29.244122 10950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2015c10f503e4751ad9ff77404fca50b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:28.950271 10949 tablet_service.cc:1514] Processing DeleteTablet for tablet e9faf70be8ac43c79305980d685a28eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:28 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:29.249147 11709 ts_tablet_manager.cc:1918] T b555eeba505740988c704df5c734e7c0 P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:57:29.261798 11709 log.cc:1196] T b555eeba505740988c704df5c734e7c0 P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/b555eeba505740988c704df5c734e7c0
I20260925 09:57:29.262892 11709 ts_tablet_manager.cc:1939] T b555eeba505740988c704df5c734e7c0 P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:29.264536 12054 tablet.cc:1722] T c2110d4ada2d45c2b07259374465a8df P b8850818e2ee42a082802d0095c08d21: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:29.273311 10389 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_755c3887714845d2bd8bb30bd444adef" } modify_external_catalogs: true
I20260925 09:57:29.276250 11961 consensus_queue.cc:1035] T df006163f72743b594baf1a3efd1c771 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:29.277052 11082 tablet_bootstrap.cc:492] T 2358acdaa5104e3eacbab8bf182b8f30 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:29.277482 11082 ts_tablet_manager.cc:1397] T 2358acdaa5104e3eacbab8bf182b8f30 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.461s user 0.008s sys 0.000s
I20260925 09:57:29.278029 10772 leader_election.cc:302] T 3b9cdad4f9144daea800553abd378e0b P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:29.278733 11961 raft_consensus.cc:2726] T 3b9cdad4f9144daea800553abd378e0b P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:29.280225 11085 tablet_bootstrap.cc:654] T c7cd1d6a603642afa2c8c737079961b5 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:29.280510 10772 leader_election.cc:302] T 2d087d3e35814c90a03eccd90eeea86c P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21
I20260925 09:57:29.281214 11961 raft_consensus.cc:2781] T 2d087d3e35814c90a03eccd90eeea86c P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 09:57:29.281553 11961 raft_consensus.cc:482] T 2d087d3e35814c90a03eccd90eeea86c P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:29.281896 11961 raft_consensus.cc:3037] T 2d087d3e35814c90a03eccd90eeea86c P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:57:29.285650 10819 tablet_service.cc:1467] Processing CreateTablet for tablet 25ac35fb43d34c9b85b8e61ad6e386a6 (DEFAULT_TABLE table=default.Table_bd0dc294c2c049e9b9c9eddce6772a44 [id=887ce3cafef64210bb69d424ed5dae8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:29.286659 10819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25ac35fb43d34c9b85b8e61ad6e386a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:29.289789 11961 raft_consensus.cc:504] T 2d087d3e35814c90a03eccd90eeea86c P c43461c4ee854ae1ac52b873ccada318 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:29.293035 11961 leader_election.cc:288] T 2d087d3e35814c90a03eccd90eeea86c P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:29.295599 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe91b39f91274a7ca61e9618d8cbb475" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:29.299527 12040 raft_consensus.cc:482] T aca5987b14b64152887ac5ee584b257a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:29.299942 12040 raft_consensus.cc:504] T aca5987b14b64152887ac5ee584b257a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:29.301688 12040 leader_election.cc:288] T aca5987b14b64152887ac5ee584b257a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:29.306226 10978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d087d3e35814c90a03eccd90eeea86c" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:29.306734 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60fdc299865446c89d3e5374d573029a" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:29.307200 10835 raft_consensus.cc:3037] T 60fdc299865446c89d3e5374d573029a P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:29.308040 10958 tablet_service.cc:1467] Processing CreateTablet for tablet e9faf70be8ac43c79305980d685a28eb (DEFAULT_TABLE table=default.Table_4f4ab1f0170a42e2826f1e3143803a6b [id=86ea83e3afc04657ad2aff23057b36d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:29.309929 10958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9faf70be8ac43c79305980d685a28eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:29.313474 10835 raft_consensus.cc:2454] T 60fdc299865446c89d3e5374d573029a P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:29.316691 10834 raft_consensus.cc:1226] T 2d087d3e35814c90a03eccd90eeea86c P c43461c4ee854ae1ac52b873ccada318 [term 2 FOLLOWER]: Rejecting Update request from peer b8850818e2ee42a082802d0095c08d21 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260925 09:57:29.001122 11084 tablet_bootstrap.cc:654] T f62c62715c9b42de826c1a66fd431eab P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:29.013312 12041 raft_consensus.cc:482] T dba14a44a9de492886d629441606767e P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:29.320086 12041 raft_consensus.cc:504] T dba14a44a9de492886d629441606767e P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:29.322186 12041 leader_election.cc:288] T dba14a44a9de492886d629441606767e P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:29.021420 12039 leader_election.cc:288] T 69a96874eea1409a9a5a06b5787fbd99 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
W20260925 09:57:28.899974 10919 leader_election.cc:341] T a5d872c3620646e5ac33ceb95de2cbd4 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:29.322832 10919 leader_election.cc:302] T a5d872c3620646e5ac33ceb95de2cbd4 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
W20260925 09:57:29.323570 10919 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 423598us
I20260925 09:57:29.055110 10948 tablet_service.cc:1467] Processing CreateTablet for tablet d5f28d1c19eb4e849dfa6b0bf532b67e (DEFAULT_TABLE table=default.Table_548921f3f0fd49069b0c73f688f4fa36 [id=f769116d89a545ff977f369477811e7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:29.073348 10947 tablet_service.cc:1467] Processing CreateTablet for tablet 476567ad4efa4043b6536f3f28d2ee28 (DEFAULT_TABLE table=default.Table_8f6c92e47621419db4bc9e3e2dd5ce12 [id=85a14c8ac1ab4bef8df5a1572d2b0e54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:29.081986 10946 tablet_service.cc:1467] Processing CreateTablet for tablet bc1d7b25d6ba42bc91130927c32d4ae6 (DEFAULT_TABLE table=default.Table_7c70dabea5b347c8b156b4b1c72703eb [id=0f92f8e3fdcc49cc906a25117f2fb4da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:29.083703 10945 tablet_service.cc:1467] Processing CreateTablet for tablet 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef (DEFAULT_TABLE table=default.Table_40181c2d583b42b89dbb7e1f589f5317 [id=cda0a13cf8024d98942bd16e2b9ecbe8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:29.085187 10944 tablet_service.cc:1467] Processing CreateTablet for tablet 815b4b1bd914433798dcb5deec6ec79c (DEFAULT_TABLE table=default.Table_bc338e8ca7174647948301512850ce00 [id=7d130f66062344458770b2b943b4485a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:29.274761 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 28d60531970a4ab58ce5ef99cb3a96bf (table default.Table_ce09d3a8d8e84973894f33b18bbfdb22 [id=6851500c7dba44f1bf353bb79bafd97a]) successfully deleted
W20260925 09:57:29.209143 10387 rpcz_store.cc:269] 0925 09:57:27.409370 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:27.409490 (+ 120us) service_pool.cc:227] Handling call
0925 09:57:29.156105 (+1746615us) spinlock_profiling.cc:243] Waited 41 ms on lock 0x7b2400003008. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabe99faf11 00007fabe9b7e851 00007fabe9b7e702 00007fabf2a15b09 00007fabea5a7480 00007fabea5b43cf 00007fabf2ada6d4 00007fabf0610fb4 00007fabf0610f42 00007fabf0610e71 00007fabf0610dec
0925 09:57:29.156416 (+ 311us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:28.230550 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:28.230922 (+ 372us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:28.230945 (+ 23us) write_op.cc:435] Acquired schema lock
0925 09:57:28.230956 (+ 11us) tablet.cc:660] Decoding operations
0925 09:57:28.231206 (+ 250us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:28.231248 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:28.231269 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:28.231435 (+ 166us) tablet.cc:699] Row locks acquired
0925 09:57:28.231448 (+ 13us) write_op.cc:260] PREPARE: finished
0925 09:57:28.231553 (+ 105us) write_op.cc:270] Start()
0925 09:57:28.231643 (+ 90us) write_op.cc:276] Timestamp: P: 1790330248231512 usec, L: 0
0925 09:57:28.231657 (+ 14us) op_driver.cc:331] REPLICATION: starting
0925 09:57:28.232109 (+ 452us) log.cc:842] Serialized 524 byte log entry
0925 09:57:28.240345 (+ 8236us) op_driver.cc:444] REPLICATION: finished
0925 09:57:28.240622 (+ 277us) write_op.cc:301] APPLY: starting
0925 09:57:28.240674 (+ 52us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:28.240756 (+ 82us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:28.240860 (+ 104us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:28.241010 (+ 150us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:28.241133 (+ 123us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:28.241173 (+ 40us) write_op.cc:312] APPLY: finished
0925 09:57:28.241330 (+ 157us) log.cc:842] Serialized 23 byte log entry
0925 09:57:28.241455 (+ 125us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:28.241509 (+ 54us) write_op.cc:454] Released schema lock
0925 09:57:28.241537 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:28.241693 (+ 156us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:28.241726 (+ 33us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":23,"spinlock_wait_cycles":94363392,"child_traces":[["op",{"apply.queue_time_us":155,"apply.run_cpu_time_us":1239,"apply.run_wall_time_us":1239,"num_ops":1,"prepare.queue_time_us":107,"prepare.run_cpu_time_us":1817,"prepare.run_wall_time_us":1821,"raft.queue_time_us":145,"raft.run_cpu_time_us":508,"raft.run_wall_time_us":507,"replication_time_us":8626}]]}
W20260925 09:57:29.327618 10377 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 52883.8us
I20260925 09:57:29.327957 10387 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_926b545b446b47679a2c440a43597ebc" } modify_external_catalogs: true
I20260925 09:57:29.291193 10392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_56638c187b5547f1b35fe75c2454fc97"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:29.123948 10982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe91b39f91274a7ca61e9618d8cbb475" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:29.135177 11901 raft_consensus.cc:482] T fe91b39f91274a7ca61e9618d8cbb475 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260925 09:57:29.329160 10392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_56638c187b5547f1b35fe75c2454fc97 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 09:57:28.996222 10918 leader_election.cc:341] T b61444c31dbf4882ac748f861ea66818 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:29.333587 10918 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 337376us
W20260925 09:57:29.334182 10918 leader_election.cc:341] T 8aff53cfd4994c629a408ee8250280c5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:29.335721 11901 raft_consensus.cc:504] T fe91b39f91274a7ca61e9618d8cbb475 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:29.337543 11901 leader_election.cc:288] T fe91b39f91274a7ca61e9618d8cbb475 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:29.337980 12039 raft_consensus.cc:2726] T a5d872c3620646e5ac33ceb95de2cbd4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:29.338913 10919 leader_election.cc:341] T b61444c31dbf4882ac748f861ea66818 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:29.339187 10387 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_926b545b446b47679a2c440a43597ebc" } modify_external_catalogs: true
I20260925 09:57:29.202217 10962 tablet_service.cc:1514] Processing DeleteTablet for tablet b555eeba505740988c704df5c734e7c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:26 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:29.330768 10946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc1d7b25d6ba42bc91130927c32d4ae6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:29.340792 10389 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_755c3887714845d2bd8bb30bd444adef" } modify_external_catalogs: true
I20260925 09:57:29.349571 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf775c9d9b114ddcb97dc6c6eca8e0fc" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:29.349952 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69a96874eea1409a9a5a06b5787fbd99" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:29.350600 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aca5987b14b64152887ac5ee584b257a" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:29.351718 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dba14a44a9de492886d629441606767e" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:29.338824 10982 raft_consensus.cc:2454] T fe91b39f91274a7ca61e9618d8cbb475 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:29.359771 12039 raft_consensus.cc:482] T 37bab60afe7a4c758a08ef7c9c24aba0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:29.364162 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe91b39f91274a7ca61e9618d8cbb475" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:29.369042 10949 tablet_service.cc:1467] Processing CreateTablet for tablet c47783e4ea694561bf008def26655884 (DEFAULT_TABLE table=default.Table_6248142ab04c4496a3db73e70549b09b [id=87469318fa894ad18bf840b9be6160d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:29.375377 10944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 815b4b1bd914433798dcb5deec6ec79c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:29.376770 10918 leader_election.cc:341] T cf775c9d9b114ddcb97dc6c6eca8e0fc P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:29.377413 10918 leader_election.cc:341] T 69a96874eea1409a9a5a06b5787fbd99 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:29.378050 10918 leader_election.cc:341] T aca5987b14b64152887ac5ee584b257a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:29.378739 10918 leader_election.cc:341] T dba14a44a9de492886d629441606767e P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:29.379290 10918 leader_election.cc:341] T fe91b39f91274a7ca61e9618d8cbb475 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:29.397668 11082 raft_consensus.cc:348] T 2358acdaa5104e3eacbab8bf182b8f30 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:29.398360 11082 raft_consensus.cc:374] T 2358acdaa5104e3eacbab8bf182b8f30 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:29.398602 11082 raft_consensus.cc:729] T 2358acdaa5104e3eacbab8bf182b8f30 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:29.360194 12039 raft_consensus.cc:504] T 37bab60afe7a4c758a08ef7c9c24aba0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:29.369653 11901 raft_consensus.cc:482] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260925 09:57:29.403939 10386 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b025f8ba34694ba09cb01ca959d3629c has no consensus state
I20260925 09:57:29.410985 10948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5f28d1c19eb4e849dfa6b0bf532b67e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:29.413623 10945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:29.431346 12057 tablet.cc:1722] T c2110d4ada2d45c2b07259374465a8df P be97ab5d75934a648d4ab76341193ec0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:29.434383 10919 leader_election.cc:302] T b61444c31dbf4882ac748f861ea66818 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
W20260925 09:57:29.435195 10919 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 110528us
W20260925 09:57:29.435760 10919 leader_election.cc:341] T 8aff53cfd4994c629a408ee8250280c5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:29.436085 10919 leader_election.cc:302] T 8aff53cfd4994c629a408ee8250280c5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:29.443225 12041 raft_consensus.cc:2726] T 8aff53cfd4994c629a408ee8250280c5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:29.407038 11901 raft_consensus.cc:504] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:29.448376 10834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5608c5f9b65b4da7b2fbb56417c92d88" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:29.446852 10949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c47783e4ea694561bf008def26655884. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:29.449776 10918 leader_election.cc:341] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:29.459730 11085 tablet_bootstrap.cc:492] T c7cd1d6a603642afa2c8c737079961b5 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:29.451588 10952 tablet_service.cc:1467] Processing CreateTablet for tablet ac8230d622214346bc7412bfc6cdb203 (DEFAULT_TABLE table=default.Table_b5a8166b631b40b998c23247d1382925 [id=b3eb418db8fe47ea98aa4c46a9d5e39e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:29.461148 12039 leader_election.cc:288] T 37bab60afe7a4c758a08ef7c9c24aba0 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:29.445035 12031 raft_consensus.cc:2726] T b61444c31dbf4882ac748f861ea66818 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:29.456676 10947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 476567ad4efa4043b6536f3f28d2ee28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:29.465039 10952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac8230d622214346bc7412bfc6cdb203. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:29.449841 11901 leader_election.cc:288] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:29.466882 11084 tablet_bootstrap.cc:492] T f62c62715c9b42de826c1a66fd431eab P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:29.463927 10834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37bab60afe7a4c758a08ef7c9c24aba0" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:29.467293 11084 ts_tablet_manager.cc:1397] T f62c62715c9b42de826c1a66fd431eab P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.645s user 0.011s sys 0.001s
W20260925 09:57:29.468077 10918 leader_election.cc:341] T 37bab60afe7a4c758a08ef7c9c24aba0 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:29.468871 11085 ts_tablet_manager.cc:1397] T c7cd1d6a603642afa2c8c737079961b5 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.608s user 0.010s sys 0.000s
I20260925 09:57:29.471133 11085 raft_consensus.cc:348] T c7cd1d6a603642afa2c8c737079961b5 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:29.479888 11085 raft_consensus.cc:374] T c7cd1d6a603642afa2c8c737079961b5 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:29.480193 11085 raft_consensus.cc:729] T c7cd1d6a603642afa2c8c737079961b5 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:29.481004 11085 consensus_queue.cc:260] T c7cd1d6a603642afa2c8c737079961b5 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:29.481964 10959 tablet_service.cc:1467] Processing CreateTablet for tablet e329410c044c40a4933550730f46001e (DEFAULT_TABLE table=default.Table_a5e309b8633e4779a51120c48f91df43 [id=c276a09404734f019e9773a244bcb199]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:29.482971 10959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e329410c044c40a4933550730f46001e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:29.483599 11085 ts_tablet_manager.cc:1428] T c7cd1d6a603642afa2c8c737079961b5 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.014s user 0.005s sys 0.000s
I20260925 09:57:29.486500 10387 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 88329fd5bd364d3a9a696fd56cbe292a
I20260925 09:57:29.487030 11085 tablet_bootstrap.cc:492] T 6161774f61ae48ab9764a05f2188a80b P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:29.493848 10387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_8f56e9a1b3504017bfabda8fd9b359ba"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:29.399142 11082 consensus_queue.cc:260] T 2358acdaa5104e3eacbab8bf182b8f30 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:29.495814 10387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8f56e9a1b3504017bfabda8fd9b359ba in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:29.497466 11085 tablet_bootstrap.cc:654] T 6161774f61ae48ab9764a05f2188a80b P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:29.520150 11085 tablet_bootstrap.cc:492] T 6161774f61ae48ab9764a05f2188a80b P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:29.521589 11085 ts_tablet_manager.cc:1397] T 6161774f61ae48ab9764a05f2188a80b P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.037s user 0.009s sys 0.003s
I20260925 09:57:29.522406 11727 ts_tablet_manager.cc:1918] T 67ce2e3325214aa6bdc3347e9e8f82b0 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:29.529919 11085 raft_consensus.cc:348] T 6161774f61ae48ab9764a05f2188a80b P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:29.530561 11085 raft_consensus.cc:374] T 6161774f61ae48ab9764a05f2188a80b P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:29.530803 11085 raft_consensus.cc:729] T 6161774f61ae48ab9764a05f2188a80b P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:29.536902 11085 consensus_queue.cc:260] T 6161774f61ae48ab9764a05f2188a80b P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:29.545603 11085 ts_tablet_manager.cc:1428] T 6161774f61ae48ab9764a05f2188a80b P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.023s user 0.003s sys 0.001s
I20260925 09:57:29.545079 10392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_d0a68d52a7f445538f3308aabc6591db"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:29.546434 11085 tablet_bootstrap.cc:492] T 2358acdaa5104e3eacbab8bf182b8f30 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
W20260925 09:57:29.547019 10392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d0a68d52a7f445538f3308aabc6591db in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:29.561160 11085 tablet_bootstrap.cc:654] T 2358acdaa5104e3eacbab8bf182b8f30 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:29.564083 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet e9faf70be8ac43c79305980d685a28eb because the tablet was not found. No further retry: Not found: Tablet not found: e9faf70be8ac43c79305980d685a28eb
I20260925 09:57:29.571321 11961 raft_consensus.cc:482] T a44d9f2f7e3e48aeb0f31fa29ae1988a P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:29.571868 11961 raft_consensus.cc:504] T a44d9f2f7e3e48aeb0f31fa29ae1988a P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:29.574568 11961 leader_election.cc:288] T a44d9f2f7e3e48aeb0f31fa29ae1988a P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
W20260925 09:57:29.579037 10772 leader_election.cc:341] T a44d9f2f7e3e48aeb0f31fa29ae1988a P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: a44d9f2f7e3e48aeb0f31fa29ae1988a
I20260925 09:57:29.582338 10384 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 26fae3fbfe374f348a1729c21ec751b9 (table default.Table_0011387b45014de4b24b466f4cb907f7 [id=e3f7aa5e988c4b6c98be5cd3d465245d]). Expected version 1 got 0
I20260925 09:57:29.583932 10384 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 59f0c6489a33433899bb6e62088121b6 (table default.Table_01f37e9d4f5b4dc5a66a24eac603810d [id=64032d286bf94c4cbdddde18adfb739e]). Expected version 2 got 1
W20260925 09:57:29.585626 10391 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.1:56563 (request call id 37) took 1131 ms (client timeout 1000 ms). Trace:
I20260925 09:57:29.593048 10389 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 92814437d53143a79f99199f8ee3a742
I20260925 09:57:29.614195 10386 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_8cde0139b88d439eaee476479a937bf3" } new_table_name: "default.Table_bd6b1d4faeb94310be9fdecc693e5b96" modify_external_catalogs: true
I20260925 09:57:29.618098 10393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_7c6bc1b63b0a45c5b67bc1fe7021a101"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:29.643528 10393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7c6bc1b63b0a45c5b67bc1fe7021a101 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:29.645334 10805 tablet_service.cc:1514] Processing DeleteTablet for tablet 60fdc299865446c89d3e5374d573029a with delete_type TABLET_DATA_DELETED (replaced by tablet 61aa6380d30a4b21b70ee2d321247bd8) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:29.646493 11085 tablet_bootstrap.cc:492] T 2358acdaa5104e3eacbab8bf182b8f30 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:29.646927 11085 ts_tablet_manager.cc:1397] T 2358acdaa5104e3eacbab8bf182b8f30 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.101s user 0.017s sys 0.013s
I20260925 09:57:29.649120 11085 raft_consensus.cc:348] T 2358acdaa5104e3eacbab8bf182b8f30 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:29.619100 10387 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_e613b02df28b46f2b371344af4971233" } new_table_name: "default.Table_0db927e92a3b4970a7efea3fb765e65e" modify_external_catalogs: true
I20260925 09:57:29.649806 11085 raft_consensus.cc:374] T 2358acdaa5104e3eacbab8bf182b8f30 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:29.650043 11085 raft_consensus.cc:729] T 2358acdaa5104e3eacbab8bf182b8f30 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:29.650940 10812 tablet_service.cc:1514] Processing DeleteTablet for tablet aba253f22afa4200926f31a1b43e3970 with delete_type TABLET_DATA_DELETED (replaced by tablet 815b4b1bd914433798dcb5deec6ec79c) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:29.652222 11085 consensus_queue.cc:260] T 2358acdaa5104e3eacbab8bf182b8f30 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:29.656661 11085 ts_tablet_manager.cc:1428] T 2358acdaa5104e3eacbab8bf182b8f30 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.009s user 0.005s sys 0.001s
I20260925 09:57:29.657452 11085 tablet_bootstrap.cc:492] T fe91b39f91274a7ca61e9618d8cbb475 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
W20260925 09:57:29.652878 10391 rpcz_store.cc:269] 0925 09:57:28.454192 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:28.648276 (+194084us) service_pool.cc:227] Handling call
0925 09:57:29.102529 (+454253us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:29.102553 (+ 24us) write_op.cc:454] Released schema lock
0925 09:57:29.585564 (+483011us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:29.083970 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:29.084299 (+ 329us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:29.084322 (+ 23us) write_op.cc:435] Acquired schema lock
0925 09:57:29.084335 (+ 13us) tablet.cc:660] Decoding operations
0925 09:57:29.084608 (+ 273us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:29.084643 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:29.084663 (+ 20us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:29.084917 (+ 254us) tablet.cc:699] Row locks acquired
0925 09:57:29.084929 (+ 12us) write_op.cc:260] PREPARE: finished
0925 09:57:29.085026 (+ 97us) write_op.cc:270] Start()
0925 09:57:29.085100 (+ 74us) write_op.cc:276] Timestamp: P: 1790330249084953 usec, L: 0
0925 09:57:29.085114 (+ 14us) op_driver.cc:331] REPLICATION: starting
0925 09:57:29.085572 (+ 458us) log.cc:842] Serialized 895 byte log entry
0925 09:57:29.093514 (+ 7942us) op_driver.cc:444] REPLICATION: finished
0925 09:57:29.093893 (+ 379us) write_op.cc:301] APPLY: starting
0925 09:57:29.093942 (+ 49us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:29.094046 (+ 104us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:29.094144 (+ 98us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:29.094371 (+ 227us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:29.094530 (+ 159us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 09:57:29.094579 (+ 49us) write_op.cc:312] APPLY: finished
0925 09:57:29.094756 (+ 177us) log.cc:842] Serialized 29 byte log entry
0925 09:57:29.094890 (+ 134us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:29.094958 (+ 68us) write_op.cc:454] Released schema lock
0925 09:57:29.094986 (+ 28us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":107,"reported_tablets":27,"child_traces":[["op",{"apply.queue_time_us":255,"apply.run_cpu_time_us":1259,"apply.run_wall_time_us":1258,"num_ops":2,"prepare.queue_time_us":552,"prepare.run_cpu_time_us":1889,"prepare.run_wall_time_us":1889,"raft.queue_time_us":2510,"raft.run_cpu_time_us":532,"raft.run_wall_time_us":571,"replication_time_us":8336}]]}
I20260925 09:57:29.669785 11085 tablet_bootstrap.cc:654] T fe91b39f91274a7ca61e9618d8cbb475 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:29.620805 10390 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.3:41469 (request call id 26) took 1844 ms (client timeout 1000 ms). Trace:
W20260925 09:57:29.675951 10390 rpcz_store.cc:269] 0925 09:57:27.776225 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:27.826981 (+ 50756us) service_pool.cc:227] Handling call
0925 09:57:28.915911 (+1088930us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:28.915936 (+ 25us) write_op.cc:454] Released schema lock
0925 09:57:29.620731 (+704795us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:28.861012 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:28.861376 (+ 364us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:28.861400 (+ 24us) write_op.cc:435] Acquired schema lock
0925 09:57:28.861413 (+ 13us) tablet.cc:660] Decoding operations
0925 09:57:28.861642 (+ 229us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:28.861681 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:28.861702 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:28.861853 (+ 151us) tablet.cc:699] Row locks acquired
0925 09:57:28.861870 (+ 17us) write_op.cc:260] PREPARE: finished
0925 09:57:28.861985 (+ 115us) write_op.cc:270] Start()
0925 09:57:28.862068 (+ 83us) write_op.cc:276] Timestamp: P: 1790330248861931 usec, L: 0
0925 09:57:28.862083 (+ 15us) op_driver.cc:331] REPLICATION: starting
0925 09:57:28.862604 (+ 521us) log.cc:842] Serialized 356 byte log entry
0925 09:57:28.880361 (+ 17757us) op_driver.cc:444] REPLICATION: finished
0925 09:57:28.880700 (+ 339us) write_op.cc:301] APPLY: starting
0925 09:57:28.880746 (+ 46us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:28.880835 (+ 89us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:28.880928 (+ 93us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:28.881081 (+ 153us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:28.881197 (+ 116us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:28.881237 (+ 40us) write_op.cc:312] APPLY: finished
0925 09:57:28.881413 (+ 176us) log.cc:842] Serialized 23 byte log entry
0925 09:57:28.881569 (+ 156us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:28.881650 (+ 81us) write_op.cc:454] Released schema lock
0925 09:57:28.881695 (+ 45us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":322955,"reported_tablets":63,"child_traces":[["op",{"apply.queue_time_us":217,"apply.run_cpu_time_us":1196,"apply.run_wall_time_us":1196,"num_ops":1,"prepare.queue_time_us":35479,"prepare.run_cpu_time_us":1988,"prepare.run_wall_time_us":1989,"raft.queue_time_us":3615,"raft.run_cpu_time_us":499,"raft.run_wall_time_us":500,"replication_time_us":18214}]]}
I20260925 09:57:29.707805 10812 tablet_service.cc:1514] Processing DeleteTablet for tablet cf35f39bcd6a41128acb2a4936fcc316 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:16 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:29.711731 10812 tablet_service.cc:1514] Processing DeleteTablet for tablet ec1b78e61e8d46cb8b87f60786059658 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:21 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:29.713578 12027 ts_tablet_manager.cc:1918] T b555eeba505740988c704df5c734e7c0 P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:57:29.639299 10384 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 3d652a76d96f4dbeafde15f3c16534a6 (table default.Table_be0f658aa41f4188bc16db37e96f4ad1 [id=73083c1560cb4db5a12964387cb49b73]). Expected version 1 got 0
I20260925 09:57:29.692884 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet b555eeba505740988c704df5c734e7c0 (table default.Table_48f368d811494119be0dc534e2fdcfbd [id=3e32026d175b4b92a3ee036f17e54b9d]) successfully deleted
W20260925 09:57:29.732023 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet d9e454f82a4e4c429b58f32926319b7c because the tablet was not found. No further retry: Not found: Tablet not found: d9e454f82a4e4c429b58f32926319b7c
W20260925 09:57:29.732597 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet aba253f22afa4200926f31a1b43e3970 because the tablet was not found. No further retry: Not found: Tablet not found: aba253f22afa4200926f31a1b43e3970
I20260925 09:57:29.697022 10653 tablet_service.cc:1514] Processing DeleteTablet for tablet 92814437d53143a79f99199f8ee3a742 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:29 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:29.698256 10389 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_66b8a45b14bf441d80ab32ee756de608" } modify_external_catalogs: true
I20260925 09:57:29.736433 10389 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_66b8a45b14bf441d80ab32ee756de608" } modify_external_catalogs: true
I20260925 09:57:29.697520 10654 tablet_service.cc:1514] Processing DeleteTablet for tablet 977a59175e5b43eeb6f23b938afd00a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:18 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:29.705538 10672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37bab60afe7a4c758a08ef7c9c24aba0" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:29.738637 10384 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet c2110d4ada2d45c2b07259374465a8df (table default.Table_44ddd064d56640a0a0e3dd8fd38344fd [id=8307778a24ee442b8f7e89c09652309e]). Expected version 1 got 0
I20260925 09:57:29.699462 10391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_574ab2069f754ba398e8d4a2e98817cf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:29.706161 10665 tablet_service.cc:1514] Processing DeleteTablet for tablet 60fdc299865446c89d3e5374d573029a with delete_type TABLET_DATA_DELETED (replaced by tablet 61aa6380d30a4b21b70ee2d321247bd8) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:29.739504 11709 tablet_replica.cc:330] T 977a59175e5b43eeb6f23b938afd00a3 P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
W20260925 09:57:29.703912 10388 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.3:41469 (request call id 25) took 3093 ms (client timeout 1000 ms). Trace:
I20260925 09:57:29.718691 10676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe91b39f91274a7ca61e9618d8cbb475" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:29.497933 11082 ts_tablet_manager.cc:1428] T 2358acdaa5104e3eacbab8bf182b8f30 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.220s user 0.008s sys 0.000s
W20260925 09:57:29.740150 10388 rpcz_store.cc:269] 0925 09:57:26.610342 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:26.616956 (+ 6614us) service_pool.cc:227] Handling call
0925 09:57:26.649704 (+ 32748us) spinlock_profiling.cc:243] Waited 22.8 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2ad9c3e 00007fabf0610fb4 00007fabf0610f42 00007fabf0610e71 00007fabf0610dec 00007fabf06100a2 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:29.703845 (+3054141us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":116334,"reported_tablets":61,"spinlock_wait_cycles":52552576}
I20260925 09:57:29.740760 10676 raft_consensus.cc:2454] T fe91b39f91274a7ca61e9618d8cbb475 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be97ab5d75934a648d4ab76341193ec0 in term 0.
I20260925 09:57:29.720474 10678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d087d3e35814c90a03eccd90eeea86c" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
W20260925 09:57:29.742043 10391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_574ab2069f754ba398e8d4a2e98817cf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:29.742233 11082 tablet_bootstrap.cc:492] T 6161774f61ae48ab9764a05f2188a80b P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:29.721076 10679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf775c9d9b114ddcb97dc6c6eca8e0fc" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:29.722229 10681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a44d9f2f7e3e48aeb0f31fa29ae1988a" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:29.744776 10681 raft_consensus.cc:2454] T a44d9f2f7e3e48aeb0f31fa29ae1988a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
I20260925 09:57:29.725512 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aca5987b14b64152887ac5ee584b257a" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:29.708076 10668 tablet_service.cc:1514] Processing DeleteTablet for tablet a44d9f2f7e3e48aeb0f31fa29ae1988a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:18 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:29.728955 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69a96874eea1409a9a5a06b5787fbd99" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:29.721675 10680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dba14a44a9de492886d629441606767e" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:29.706080 10673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5608c5f9b65b4da7b2fbb56417c92d88" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:29.707139 10666 tablet_service.cc:1467] Processing CreateTablet for tablet ac8230d622214346bc7412bfc6cdb203 (DEFAULT_TABLE table=default.Table_b5a8166b631b40b998c23247d1382925 [id=b3eb418db8fe47ea98aa4c46a9d5e39e]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 09:57:29.753121 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 92814437d53143a79f99199f8ee3a742 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 92814437d53143a79f99199f8ee3a742 already in progress: creating tablet
I20260925 09:57:29.754083 10666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac8230d622214346bc7412bfc6cdb203. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:29.757473 10621 leader_election.cc:341] T fe91b39f91274a7ca61e9618d8cbb475 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:29.758397 10621 leader_election.cc:302] T 60fdc299865446c89d3e5374d573029a P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:29.761592 12065 raft_consensus.cc:2781] T 60fdc299865446c89d3e5374d573029a P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:29.763398 12065 raft_consensus.cc:686] T 60fdc299865446c89d3e5374d573029a P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:29.768476 12065 consensus_queue.cc:237] T 60fdc299865446c89d3e5374d573029a P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:29.773188 10619 leader_election.cc:341] T 60fdc299865446c89d3e5374d573029a P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 60fdc299865446c89d3e5374d573029a
I20260925 09:57:29.781307 10645 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:37864 (request call id 22) took 1252 ms. Trace:
I20260925 09:57:29.784595 10645 rpcz_store.cc:276] 0925 09:57:28.528966 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:28.529074 (+ 108us) service_pool.cc:227] Handling call
0925 09:57:28.855575 (+326501us) spinlock_profiling.cc:243] Waited 83 ms on lock 0x7b54000e2754. stack: 00007fb965334033 00007fb965333f30 00007fb964eb5534 00000000004db2b4 00000000004db249 00007fb96c240471 00007fb96c2b5268 00007fb96d2a16cd 00007fb96bb49e04 00007fb96bb49d92 00007fb96bb49cc1 00007fb96bb49c3c 00007fb96bb48ef2 00007fb96a06a94c 00007fb96a069d86 00007fb96a06972f
0925 09:57:29.781123 (+925548us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":190987648}
I20260925 09:57:29.720167 10812 tablet_service.cc:1514] Processing DeleteTablet for tablet 3be0eccc65564bfba3d6e5c398c61fb6 with delete_type TABLET_DATA_DELETED (replaced by tablet cc295f7a9169475f9a9235e657be629f) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:29.748529 12068 raft_consensus.cc:482] T c7cd1d6a603642afa2c8c737079961b5 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:29.794135 12027 log.cc:1196] T b555eeba505740988c704df5c734e7c0 P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/b555eeba505740988c704df5c734e7c0
I20260925 09:57:29.794005 12068 raft_consensus.cc:504] T c7cd1d6a603642afa2c8c737079961b5 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:29.794986 12027 ts_tablet_manager.cc:1939] T b555eeba505740988c704df5c734e7c0 P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:29.796036 12068 leader_election.cc:288] T c7cd1d6a603642afa2c8c737079961b5 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:29.783986 10805 tablet_service.cc:1467] Processing CreateTablet for tablet e329410c044c40a4933550730f46001e (DEFAULT_TABLE table=default.Table_a5e309b8633e4779a51120c48f91df43 [id=c276a09404734f019e9773a244bcb199]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:29.785593 10808 tablet_service.cc:1467] Processing CreateTablet for tablet e9faf70be8ac43c79305980d685a28eb (DEFAULT_TABLE table=default.Table_4f4ab1f0170a42e2826f1e3143803a6b [id=86ea83e3afc04657ad2aff23057b36d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:29.786653 10809 tablet_service.cc:1467] Processing CreateTablet for tablet c47783e4ea694561bf008def26655884 (DEFAULT_TABLE table=default.Table_6248142ab04c4496a3db73e70549b09b [id=87469318fa894ad18bf840b9be6160d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:29.787688 10814 tablet_service.cc:1467] Processing CreateTablet for tablet ac8230d622214346bc7412bfc6cdb203 (DEFAULT_TABLE table=default.Table_b5a8166b631b40b998c23247d1382925 [id=b3eb418db8fe47ea98aa4c46a9d5e39e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:29.788266 10806 tablet_service.cc:1514] Processing DeleteTablet for tablet 92814437d53143a79f99199f8ee3a742 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:29 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:29.798146 10805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e329410c044c40a4933550730f46001e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:29.798699 10809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c47783e4ea694561bf008def26655884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:29.798704 10386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_7d5661d0c59f4ac2912f9cb12f58886a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:29.800175 10386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7d5661d0c59f4ac2912f9cb12f58886a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:29.803866 10814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac8230d622214346bc7412bfc6cdb203. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:29.807976 10808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9faf70be8ac43c79305980d685a28eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:29.524755 12031 raft_consensus.cc:482] T 61aa6380d30a4b21b70ee2d321247bd8 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:29.513360 11901 raft_consensus.cc:482] T 5246648245e04e26be2abf1dbf706688 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:29.527616 10963 tablet_service.cc:1514] Processing DeleteTablet for tablet 3706253e53d841aeaa43889ead13af32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:23 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:29.532817 10957 tablet_service.cc:1514] Processing DeleteTablet for tablet 28d60531970a4ab58ce5ef99cb3a96bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:24 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:29.535050 10950 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a31b999f0fd447eacff13de71aa3fff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:23 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:29.602546 10956 tablet_service.cc:1514] Processing DeleteTablet for tablet 977a59175e5b43eeb6f23b938afd00a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:18 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:29.605476 10955 tablet_service.cc:1514] Processing DeleteTablet for tablet 359090244c4748e8a2d45e3c77e89498 with delete_type TABLET_DATA_DELETED (replaced by tablet b39f9e7102ff4a9c96a2cc5d2676f03d) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:29.607352 11084 raft_consensus.cc:348] T f62c62715c9b42de826c1a66fd431eab P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:29.630911 12039 raft_consensus.cc:482] T 3be0eccc65564bfba3d6e5c398c61fb6 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b8850818e2ee42a082802d0095c08d21)
W20260925 09:57:29.832998 10775 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 137115us
I20260925 09:57:29.833132 11084 raft_consensus.cc:374] T f62c62715c9b42de826c1a66fd431eab P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:29.833395 11084 raft_consensus.cc:729] T f62c62715c9b42de826c1a66fd431eab P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:29.687565 10951 tablet_service.cc:1514] Processing DeleteTablet for tablet 92814437d53143a79f99199f8ee3a742 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:29 UTC) from {username='slave'} at 127.0.0.1:53560
W20260925 09:57:29.719841 10919 consensus_peers.cc:487] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0 -> Peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Couldn't send request to peer b8850818e2ee42a082802d0095c08d21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b555eeba505740988c704df5c734e7c0. This is attempt 1: this message will repeat every 5th retry.
W20260925 09:57:29.834532 10387 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2f07dc9548364870923165f4ea524310 consensus state has no leader
W20260925 09:57:29.834700 10919 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 114962us
W20260925 09:57:29.737414 10918 consensus_peers.cc:487] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
W20260925 09:57:29.835533 10918 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 98205us
I20260925 09:57:29.826026 10978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7cd1d6a603642afa2c8c737079961b5" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:29.838864 10387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_5a48ae73ce6b4d6e8eb352d18dd62ad5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:29.840355 10387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5a48ae73ce6b4d6e8eb352d18dd62ad5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:29.844566 10388 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_0507f47042b9445abaf8671c979f8994" } modify_external_catalogs: true
I20260925 09:57:29.845046 10388 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_0507f47042b9445abaf8671c979f8994" } modify_external_catalogs: true
I20260925 09:57:29.815600 12030 consensus_queue.cc:1035] T 3ff4cf556eee494d8cd590884466f09f P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:29.824952 12054 tablet.cc:1722] T 59f0c6489a33433899bb6e62088121b6 P b8850818e2ee42a082802d0095c08d21: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260925 09:57:29.859545 10621 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 62243.3us
I20260925 09:57:29.860783 11085 tablet_bootstrap.cc:492] T fe91b39f91274a7ca61e9618d8cbb475 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:29.861685 11961 raft_consensus.cc:482] T 6161774f61ae48ab9764a05f2188a80b P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:29.864609 11961 raft_consensus.cc:504] T 6161774f61ae48ab9764a05f2188a80b P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:29.866452 11961 leader_election.cc:288] T 6161774f61ae48ab9764a05f2188a80b P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:29.867334 11085 ts_tablet_manager.cc:1397] T fe91b39f91274a7ca61e9618d8cbb475 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.210s user 0.007s sys 0.005s
I20260925 09:57:29.868050 12030 consensus_queue.cc:1046] T 2d087d3e35814c90a03eccd90eeea86c P b8850818e2ee42a082802d0095c08d21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:29.870313 10389 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 3ff4cf556eee494d8cd590884466f09f
I20260925 09:57:29.797520 12027 tablet_replica.cc:330] T 60fdc299865446c89d3e5374d573029a P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:29.872197 12027 raft_consensus.cc:2229] T 60fdc299865446c89d3e5374d573029a P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:29.872650 12027 raft_consensus.cc:2258] T 60fdc299865446c89d3e5374d573029a P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:29.874460 12027 ts_tablet_manager.cc:1905] T 60fdc299865446c89d3e5374d573029a P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:29.874925 10668 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ff4cf556eee494d8cd590884466f09f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:29 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:29.881611 10978 raft_consensus.cc:2454] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
W20260925 09:57:29.861584 10384 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.2:40761 (request call id 35) took 958 ms (client timeout 1000 ms). Trace:
I20260925 09:57:29.882114 12080 raft_consensus.cc:482] T c7cd1d6a603642afa2c8c737079961b5 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260925 09:57:29.882294 10384 rpcz_store.cc:269] 0925 09:57:28.903298 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:28.920687 (+ 17389us) service_pool.cc:227] Handling call
0925 09:57:29.861519 (+940832us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":94729,"reported_tablets":34}
W20260925 09:57:29.882901 10919 leader_election.cc:341] T 37bab60afe7a4c758a08ef7c9c24aba0 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:29.826185 10619 leader_election.cc:302] T fe91b39f91274a7ca61e9618d8cbb475 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
W20260925 09:57:29.884723 10918 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 48413.7us
I20260925 09:57:29.882570 12080 raft_consensus.cc:504] T c7cd1d6a603642afa2c8c737079961b5 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:29.885073 10772 leader_election.cc:302] T c7cd1d6a603642afa2c8c737079961b5 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:29.883265 10919 leader_election.cc:302] T 37bab60afe7a4c758a08ef7c9c24aba0 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:29.833946 11084 consensus_queue.cc:260] T f62c62715c9b42de826c1a66fd431eab P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:29.845116 11727 log.cc:1196] T 67ce2e3325214aa6bdc3347e9e8f82b0 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/67ce2e3325214aa6bdc3347e9e8f82b0
I20260925 09:57:29.831687 12031 raft_consensus.cc:504] T 61aa6380d30a4b21b70ee2d321247bd8 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:29.878940 11901 raft_consensus.cc:504] T 5246648245e04e26be2abf1dbf706688 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:29.880237 12039 raft_consensus.cc:504] T 3be0eccc65564bfba3d6e5c398c61fb6 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:29.887112 11727 ts_tablet_manager.cc:1939] T 67ce2e3325214aa6bdc3347e9e8f82b0 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
W20260925 09:57:29.884675 10619 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 58582.9us
I20260925 09:57:29.889768 12040 raft_consensus.cc:2726] T 37bab60afe7a4c758a08ef7c9c24aba0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:29.891070 12080 leader_election.cc:288] T c7cd1d6a603642afa2c8c737079961b5 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:29.899700 11961 raft_consensus.cc:2781] T c7cd1d6a603642afa2c8c737079961b5 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:29.900126 11961 raft_consensus.cc:482] T c7cd1d6a603642afa2c8c737079961b5 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:29.900482 11961 raft_consensus.cc:3037] T c7cd1d6a603642afa2c8c737079961b5 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:29.902272 12030 raft_consensus.cc:2781] T fe91b39f91274a7ca61e9618d8cbb475 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:29.902693 12030 raft_consensus.cc:482] T fe91b39f91274a7ca61e9618d8cbb475 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:29.903059 12030 raft_consensus.cc:3037] T fe91b39f91274a7ca61e9618d8cbb475 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:29.903826 10949 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ff4cf556eee494d8cd590884466f09f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:29 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:29.907528 12031 leader_election.cc:288] T 61aa6380d30a4b21b70ee2d321247bd8 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:29.908950 12039 leader_election.cc:288] T 3be0eccc65564bfba3d6e5c398c61fb6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:29.910357 11901 leader_election.cc:288] T 5246648245e04e26be2abf1dbf706688 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:29.911849 12054 tablet.cc:1722] T 26fae3fbfe374f348a1729c21ec751b9 P b8850818e2ee42a082802d0095c08d21: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:29.914456 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61aa6380d30a4b21b70ee2d321247bd8" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:29.915642 10834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5246648245e04e26be2abf1dbf706688" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:29.916877 10918 leader_election.cc:341] T 3be0eccc65564bfba3d6e5c398c61fb6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Not found: Tablet not found: 3be0eccc65564bfba3d6e5c398c61fb6
W20260925 09:57:29.917873 10918 leader_election.cc:341] T 61aa6380d30a4b21b70ee2d321247bd8 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:29.922382 10918 leader_election.cc:341] T 5246648245e04e26be2abf1dbf706688 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:29.924324 10620 outbound_call.cc:322] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.10.9.61:33047, user_credentials={real_user=slave}} blocked reactor thread for 45896.4us
I20260925 09:57:29.926944 12057 tablet.cc:1722] T c588b0c05a1a4f3ea545066183e76502 P be97ab5d75934a648d4ab76341193ec0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:29.930263 12065 raft_consensus.cc:3032] T 2d087d3e35814c90a03eccd90eeea86c P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Stepping down as leader of term 1
I20260925 09:57:29.930809 12065 raft_consensus.cc:729] T 2d087d3e35814c90a03eccd90eeea86c P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:29.931653 12065 consensus_queue.cc:260] T 2d087d3e35814c90a03eccd90eeea86c P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:29.932717 12065 raft_consensus.cc:3037] T 2d087d3e35814c90a03eccd90eeea86c P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:57:29.942498 10387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_7dfed08625c8462dbccb81cd1ceed336"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:29.943837 11085 raft_consensus.cc:348] T fe91b39f91274a7ca61e9618d8cbb475 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:29.944564 11085 raft_consensus.cc:374] T fe91b39f91274a7ca61e9618d8cbb475 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260925 09:57:29.944693 10387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7dfed08625c8462dbccb81cd1ceed336 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:29.944870 11085 raft_consensus.cc:729] T fe91b39f91274a7ca61e9618d8cbb475 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:29.945618 11085 consensus_queue.cc:260] T fe91b39f91274a7ca61e9618d8cbb475 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:29.953308 10391 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_04c588671331474fac69a076bb87ba84" } new_table_name: "default.Table_8398f7e9c7a64f688bd18521e99cb669" modify_external_catalogs: true
I20260925 09:57:29.957935 11085 ts_tablet_manager.cc:1428] T fe91b39f91274a7ca61e9618d8cbb475 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.088s user 0.004s sys 0.000s
I20260925 09:57:29.967031 10978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7cd1d6a603642afa2c8c737079961b5" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:29.967725 12041 raft_consensus.cc:482] T f62c62715c9b42de826c1a66fd431eab P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:29.967783 10978 raft_consensus.cc:2454] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:29.968214 12041 raft_consensus.cc:504] T f62c62715c9b42de826c1a66fd431eab P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:29.970116 12041 leader_election.cc:288] T f62c62715c9b42de826c1a66fd431eab P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:29.978498 10619 leader_election.cc:302] T c7cd1d6a603642afa2c8c737079961b5 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:29.979271 12065 raft_consensus.cc:2781] T c7cd1d6a603642afa2c8c737079961b5 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:29.979617 12065 raft_consensus.cc:482] T c7cd1d6a603642afa2c8c737079961b5 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:29.979930 12065 raft_consensus.cc:3037] T c7cd1d6a603642afa2c8c737079961b5 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:29.984548 11085 tablet_bootstrap.cc:492] T 3ff4cf556eee494d8cd590884466f09f P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:29.986905 12065 raft_consensus.cc:504] T c7cd1d6a603642afa2c8c737079961b5 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:29.965417 10775 leader_election.cc:302] T 2d087d3e35814c90a03eccd90eeea86c P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:29.989169 10775 leader_election.cc:302] T a44d9f2f7e3e48aeb0f31fa29ae1988a P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters: be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:29.989810 12067 raft_consensus.cc:2726] T 2d087d3e35814c90a03eccd90eeea86c P c43461c4ee854ae1ac52b873ccada318 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260925 09:57:29.990209 12065 leader_election.cc:288] T c7cd1d6a603642afa2c8c737079961b5 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:29.990339 12067 raft_consensus.cc:2781] T a44d9f2f7e3e48aeb0f31fa29ae1988a P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:29.990646 12067 raft_consensus.cc:482] T a44d9f2f7e3e48aeb0f31fa29ae1988a P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:29.990936 12067 raft_consensus.cc:3037] T a44d9f2f7e3e48aeb0f31fa29ae1988a P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:57:29.991123 10391 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 572c83b4a7c84fdab1fdbb0fbdb0a53d consensus state has no leader
I20260925 09:57:29.996202 10973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7cd1d6a603642afa2c8c737079961b5" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:29.996806 10973 raft_consensus.cc:3037] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:30.004098 11961 raft_consensus.cc:504] T c7cd1d6a603642afa2c8c737079961b5 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:30.005223 10973 raft_consensus.cc:2454] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:30.006297 10619 leader_election.cc:302] T c7cd1d6a603642afa2c8c737079961b5 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:30.006357 10673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7cd1d6a603642afa2c8c737079961b5" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:30.007138 10673 raft_consensus.cc:2379] T c7cd1d6a603642afa2c8c737079961b5 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c43461c4ee854ae1ac52b873ccada318 in current term 1: Already voted for candidate b8850818e2ee42a082802d0095c08d21 in this term.
I20260925 09:57:30.008054 12065 raft_consensus.cc:2781] T c7cd1d6a603642afa2c8c737079961b5 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:30.007845 12096 tablet.cc:1722] T 59f0c6489a33433899bb6e62088121b6 P be97ab5d75934a648d4ab76341193ec0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 09:57:30.008503 12065 raft_consensus.cc:686] T c7cd1d6a603642afa2c8c737079961b5 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:30.009303 12065 consensus_queue.cc:237] T c7cd1d6a603642afa2c8c737079961b5 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:30.015964 10919 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 133085us
I20260925 09:57:30.020077 12071 raft_consensus.cc:482] T 2358acdaa5104e3eacbab8bf182b8f30 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:30.020474 12071 raft_consensus.cc:504] T 2358acdaa5104e3eacbab8bf182b8f30 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:30.022557 12071 leader_election.cc:288] T 2358acdaa5104e3eacbab8bf182b8f30 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:30.025827 12030 raft_consensus.cc:504] T fe91b39f91274a7ca61e9618d8cbb475 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:30.027966 10673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6161774f61ae48ab9764a05f2188a80b" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:30.029788 10673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7cd1d6a603642afa2c8c737079961b5" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:30.032399 11961 leader_election.cc:288] T c7cd1d6a603642afa2c8c737079961b5 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:30.037752 10668 tablet_service.cc:1467] Processing CreateTablet for tablet a3edf29118e048c5a59b4ff2643ed206 (DEFAULT_TABLE table=default.Table_c91ec477f2e541c394b6cc12607a0d3e [id=49e1579274a540f58414076daa019f31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.039901 10387 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_c22088d959e84c3cbabadde4d3445178" } new_table_name: "default.Table_7a8e20521b384bfa8f659eaaf46bae54" modify_external_catalogs: true
I20260925 09:57:30.040865 10665 tablet_service.cc:1467] Processing CreateTablet for tablet 451d1984f9b144b68f0681837f8eacf1 (DEFAULT_TABLE table=default.Table_7a34d8a020ed46dc8177c60c3acfb2ec [id=8b7604f5a95c4cebb355038aa0471564]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.043035 12030 leader_election.cc:288] T fe91b39f91274a7ca61e9618d8cbb475 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:30.018908 12067 raft_consensus.cc:504] T a44d9f2f7e3e48aeb0f31fa29ae1988a P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:30.046767 10772 leader_election.cc:341] T 6161774f61ae48ab9764a05f2188a80b P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 6161774f61ae48ab9764a05f2188a80b
W20260925 09:57:30.047159 10772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 66721.6us
I20260925 09:57:30.049851 10919 leader_election.cc:302] T fe91b39f91274a7ca61e9618d8cbb475 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
W20260925 09:57:30.050997 10919 leader_election.cc:341] T cf775c9d9b114ddcb97dc6c6eca8e0fc P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:30.051432 10919 leader_election.cc:302] T cf775c9d9b114ddcb97dc6c6eca8e0fc P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:30.053014 10388 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet c7cd1d6a603642afa2c8c737079961b5
I20260925 09:57:30.054574 10654 tablet_service.cc:1514] Processing DeleteTablet for tablet c7cd1d6a603642afa2c8c737079961b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:29 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:30.061311 10388 master_service.cc:946] ReplaceTablet: received request to replace tablet 61aa6380d30a4b21b70ee2d321247bd8 from {username='slave'} at 127.0.0.1:35140
I20260925 09:57:29.824940 11082 tablet_bootstrap.cc:654] T 6161774f61ae48ab9764a05f2188a80b P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:30.064144 10775 leader_election.cc:341] T 6161774f61ae48ab9764a05f2188a80b P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:30.064653 10775 leader_election.cc:302] T 6161774f61ae48ab9764a05f2188a80b P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:30.065626 11961 raft_consensus.cc:2726] T 6161774f61ae48ab9764a05f2188a80b P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:30.067956 12067 leader_election.cc:288] T a44d9f2f7e3e48aeb0f31fa29ae1988a P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:30.068996 12101 tablet.cc:1722] T c588b0c05a1a4f3ea545066183e76502 P c43461c4ee854ae1ac52b873ccada318: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:30.039263 11085 tablet_bootstrap.cc:654] T 3ff4cf556eee494d8cd590884466f09f P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:30.017638 11084 ts_tablet_manager.cc:1428] T f62c62715c9b42de826c1a66fd431eab P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.550s user 0.003s sys 0.008s
I20260925 09:57:30.029088 10973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2358acdaa5104e3eacbab8bf182b8f30" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:30.041626 10951 tablet_service.cc:1467] Processing CreateTablet for tablet a3edf29118e048c5a59b4ff2643ed206 (DEFAULT_TABLE table=default.Table_c91ec477f2e541c394b6cc12607a0d3e [id=49e1579274a540f58414076daa019f31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.045421 10982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe91b39f91274a7ca61e9618d8cbb475" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:30.072340 10982 raft_consensus.cc:3037] T fe91b39f91274a7ca61e9618d8cbb475 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:30.074308 10673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a44d9f2f7e3e48aeb0f31fa29ae1988a" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:30.074867 10673 raft_consensus.cc:3037] T a44d9f2f7e3e48aeb0f31fa29ae1988a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:30.076036 10819 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:33344 (request call id 277) took 1116 ms. Trace:
I20260925 09:57:30.076411 10819 rpcz_store.cc:276] 0925 09:57:28.970434 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:29.285058 (+314624us) service_pool.cc:227] Handling call
0925 09:57:29.286288 (+ 1230us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0925 09:57:29.286361 (+ 73us) ts_tablet_manager.cc:676] Creating new metadata...
0925 09:57:29.511017 (+224656us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:29.807324 (+296307us) tablet_replica.cc:205] Creating consensus instance
0925 09:57:30.075980 (+268656us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260925 09:57:30.081984 10673 raft_consensus.cc:2454] T a44d9f2f7e3e48aeb0f31fa29ae1988a P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:30.083196 10775 leader_election.cc:302] T a44d9f2f7e3e48aeb0f31fa29ae1988a P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:30.084399 10951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3edf29118e048c5a59b4ff2643ed206. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.085110 12067 raft_consensus.cc:2781] T a44d9f2f7e3e48aeb0f31fa29ae1988a P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:30.085628 12067 raft_consensus.cc:686] T a44d9f2f7e3e48aeb0f31fa29ae1988a P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Becoming Leader. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Running, Role: LEADER
I20260925 09:57:30.086607 10389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_6f3d5186b0fc4a2b83567d14adb39641"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:30.086624 12067 consensus_queue.cc:237] T a44d9f2f7e3e48aeb0f31fa29ae1988a P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:30.088464 10391 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_bc338e8ca7174647948301512850ce00" } modify_external_catalogs: true
I20260925 09:57:30.088913 10391 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_bc338e8ca7174647948301512850ce00" } modify_external_catalogs: true
W20260925 09:57:30.071198 11084 ts_tablet_manager.cc:1460] T f62c62715c9b42de826c1a66fd431eab P be97ab5d75934a648d4ab76341193ec0: Tablet startup took 603ms
I20260925 09:57:30.091361 10386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_2a228b9f88074bb897dd2b8e4624913b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:30.092795 10386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2a228b9f88074bb897dd2b8e4624913b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 09:57:30.092803 10389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6f3d5186b0fc4a2b83567d14adb39641 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:30.095956 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f62c62715c9b42de826c1a66fd431eab" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:30.104468 10982 raft_consensus.cc:2454] T fe91b39f91274a7ca61e9618d8cbb475 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:30.103801 11085 tablet_bootstrap.cc:492] T 3ff4cf556eee494d8cd590884466f09f P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:30.105334 10654 tablet_service.cc:1467] Processing CreateTablet for tablet 76b1f79efa784662a0adf032c074ce4a (DEFAULT_TABLE table=default.Table_9d3d93d30cc442bcbabe7edfbf665469 [id=e59d5ae8365d40f08ece3a094978d896]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.105867 11085 ts_tablet_manager.cc:1397] T 3ff4cf556eee494d8cd590884466f09f P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.121s user 0.011s sys 0.000s
I20260925 09:57:30.107752 10653 tablet_service.cc:1467] Processing CreateTablet for tablet 0a018bea4200474a9904f62aee37240b (DEFAULT_TABLE table=default.Table_14a20618504e490289d610b4f74cea84 [id=44f5211ebe4044a1bb2d528a304a1754]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.111469 10384 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: default.Table_44ddd064d56640a0a0e3dd8fd38344fd [id=8307778a24ee442b8f7e89c09652309e] alter complete (version 1)
I20260925 09:57:30.113354 11085 raft_consensus.cc:348] T 3ff4cf556eee494d8cd590884466f09f P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:30.113969 11085 raft_consensus.cc:374] T 3ff4cf556eee494d8cd590884466f09f P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:30.114195 11085 raft_consensus.cc:729] T 3ff4cf556eee494d8cd590884466f09f P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:30.117730 10655 tablet_service.cc:1467] Processing CreateTablet for tablet b025f8ba34694ba09cb01ca959d3629c (DEFAULT_TABLE table=default.Table_dfc8dd8209e3467b872784a505f0d78c [id=65b6e9ec3f114cea85e70f1509e3d1b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.119004 10657 tablet_service.cc:1467] Processing CreateTablet for tablet f46095351ea64812a7fdcfc79cd92c1f (DEFAULT_TABLE table=default.Table_cc9f467e078c448f87392bf324b040a8 [id=4cefdaddfd6143468df4502cd35afac4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.122443 11085 consensus_queue.cc:260] T 3ff4cf556eee494d8cd590884466f09f P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:30.130900 11085 ts_tablet_manager.cc:1428] T 3ff4cf556eee494d8cd590884466f09f P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.024s user 0.006s sys 0.000s
I20260925 09:57:30.132663 11085 tablet_bootstrap.cc:492] T 977a59175e5b43eeb6f23b938afd00a3 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:30.145139 10388 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 61aa6380d30a4b21b70ee2d321247bd8
I20260925 09:57:30.146850 10664 tablet_service.cc:1514] Processing DeleteTablet for tablet 61aa6380d30a4b21b70ee2d321247bd8 with delete_type TABLET_DATA_DELETED (replaced by tablet 77ebb59faf3e4fb187bbce29612c25ea) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:30.147922 10388 catalog_manager.cc:6565] ReplaceTablet: tablet 61aa6380d30a4b21b70ee2d321247bd8 deleted and replaced by tablet 77ebb59faf3e4fb187bbce29612c25ea
I20260925 09:57:30.150975 11085 tablet_bootstrap.cc:654] T 977a59175e5b43eeb6f23b938afd00a3 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:30.150856 10384 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_55e6a408ce794b5fbd2fe39d51e6a2aa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:30.152266 10384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_55e6a408ce794b5fbd2fe39d51e6a2aa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:30.171554 10391 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 815b4b1bd914433798dcb5deec6ec79c
I20260925 09:57:30.173992 10664 tablet_service.cc:1514] Processing DeleteTablet for tablet 815b4b1bd914433798dcb5deec6ec79c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:30 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:30.194136 10668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3edf29118e048c5a59b4ff2643ed206. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.199631 10389 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_dfc8dd8209e3467b872784a505f0d78c" } new_table_name: "default.Table_2702858b51514e7f9a51a2bf19d36b1f" modify_external_catalogs: true
I20260925 09:57:30.204008 10665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 451d1984f9b144b68f0681837f8eacf1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:30.205736 10772 leader_election.cc:341] T a44d9f2f7e3e48aeb0f31fa29ae1988a P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: a44d9f2f7e3e48aeb0f31fa29ae1988a
I20260925 09:57:30.207450 10654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76b1f79efa784662a0adf032c074ce4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.211854 10653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a018bea4200474a9904f62aee37240b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.212486 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7cd1d6a603642afa2c8c737079961b5" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:30.213254 10832 raft_consensus.cc:2379] T c7cd1d6a603642afa2c8c737079961b5 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b8850818e2ee42a082802d0095c08d21 in current term 1: Already voted for candidate c43461c4ee854ae1ac52b873ccada318 in this term.
I20260925 09:57:30.214294 10836 raft_consensus.cc:1226] T 2d087d3e35814c90a03eccd90eeea86c P c43461c4ee854ae1ac52b873ccada318 [term 2 FOLLOWER]: Rejecting Update request from peer b8850818e2ee42a082802d0095c08d21 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260925 09:57:30.214381 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7cd1d6a603642afa2c8c737079961b5" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:30.214998 10839 raft_consensus.cc:2379] T c7cd1d6a603642afa2c8c737079961b5 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b8850818e2ee42a082802d0095c08d21 in current term 1: Already voted for candidate c43461c4ee854ae1ac52b873ccada318 in this term.
I20260925 09:57:30.215809 10655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b025f8ba34694ba09cb01ca959d3629c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.215965 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2358acdaa5104e3eacbab8bf182b8f30" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:30.216429 10839 raft_consensus.cc:2454] T 2358acdaa5104e3eacbab8bf182b8f30 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:30.220036 10657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f46095351ea64812a7fdcfc79cd92c1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.110157 10981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7cd1d6a603642afa2c8c737079961b5" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:30.113464 10955 tablet_service.cc:1467] Processing CreateTablet for tablet 76b1f79efa784662a0adf032c074ce4a (DEFAULT_TABLE table=default.Table_9d3d93d30cc442bcbabe7edfbf665469 [id=e59d5ae8365d40f08ece3a094978d896]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.220887 10981 raft_consensus.cc:2379] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c43461c4ee854ae1ac52b873ccada318 in current term 1: Already voted for candidate b8850818e2ee42a082802d0095c08d21 in this term.
I20260925 09:57:30.114573 10949 tablet_service.cc:1467] Processing CreateTablet for tablet 451d1984f9b144b68f0681837f8eacf1 (DEFAULT_TABLE table=default.Table_7a34d8a020ed46dc8177c60c3acfb2ec [id=8b7604f5a95c4cebb355038aa0471564]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.114763 10947 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53560 (request call id 288) took 1055 ms. Trace:
I20260925 09:57:30.116431 10950 tablet_service.cc:1467] Processing CreateTablet for tablet b025f8ba34694ba09cb01ca959d3629c (DEFAULT_TABLE table=default.Table_dfc8dd8209e3467b872784a505f0d78c [id=65b6e9ec3f114cea85e70f1509e3d1b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.115733 10956 tablet_service.cc:1467] Processing CreateTablet for tablet 0a018bea4200474a9904f62aee37240b (DEFAULT_TABLE table=default.Table_14a20618504e490289d610b4f74cea84 [id=44f5211ebe4044a1bb2d528a304a1754]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.124593 10952 tablet_service.cc:1514] Processing DeleteTablet for tablet c7cd1d6a603642afa2c8c737079961b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:29 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:30.092335 10973 raft_consensus.cc:2454] T 2358acdaa5104e3eacbab8bf182b8f30 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:30.137004 12057 tablet.cc:1722] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:30.233593 10619 leader_election.cc:302] T fe91b39f91274a7ca61e9618d8cbb475 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:30.234378 12065 raft_consensus.cc:2781] T fe91b39f91274a7ca61e9618d8cbb475 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:30.234810 12065 raft_consensus.cc:686] T fe91b39f91274a7ca61e9618d8cbb475 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:30.148363 10963 tablet_service.cc:1514] Processing DeleteTablet for tablet 61aa6380d30a4b21b70ee2d321247bd8 with delete_type TABLET_DATA_DELETED (replaced by tablet 77ebb59faf3e4fb187bbce29612c25ea) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:30.125391 10957 tablet_service.cc:1467] Processing CreateTablet for tablet f46095351ea64812a7fdcfc79cd92c1f (DEFAULT_TABLE table=default.Table_cc9f467e078c448f87392bf324b040a8 [id=4cefdaddfd6143468df4502cd35afac4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.235606 12065 consensus_queue.cc:237] T fe91b39f91274a7ca61e9618d8cbb475 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:30.187901 10919 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 136924us
I20260925 09:57:30.219769 10946 tablet_service.cc:1514] Processing DeleteTablet for tablet 815b4b1bd914433798dcb5deec6ec79c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:30 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:30.231598 12039 raft_consensus.cc:2741] T fe91b39f91274a7ca61e9618d8cbb475 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260925 09:57:30.232529 10955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76b1f79efa784662a0adf032c074ce4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.232730 10947 rpcz_store.cc:276] 0925 09:57:29.066559 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:29.072400 (+ 5841us) service_pool.cc:227] Handling call
0925 09:57:29.405757 (+333357us) spinlock_profiling.cc:243] Waited 14.8 ms on lock 0x7b5800001898. stack: 00007f0a8f2ab033 00007f0a8f2aaf30 00007f0a8ee2c534 00000000004db2b4 00000000004db249 00007f0a8f121f11 00007f0a8f2a5b7c 00007f0a8f2a573b 00007f0a9813cb09 00007f0a97199730 00007f0a9727bd21 00007f0a972591e3 00007f0a9720a869 00007f0a958fcc44 00007f0a958fcbd2 00007f0a958fcb01
0925 09:57:29.408260 (+ 2503us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0925 09:57:29.443930 (+ 35670us) ts_tablet_manager.cc:676] Creating new metadata...
0925 09:57:29.576151 (+132221us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:29.995498 (+419347us) tablet_replica.cc:205] Creating consensus instance
0925 09:57:30.114689 (+119191us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":34145024}
I20260925 09:57:30.241240 11082 tablet_bootstrap.cc:492] T 6161774f61ae48ab9764a05f2188a80b P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:30.241689 11082 ts_tablet_manager.cc:1397] T 6161774f61ae48ab9764a05f2188a80b P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.500s user 0.011s sys 0.000s
I20260925 09:57:30.244459 11082 raft_consensus.cc:348] T 6161774f61ae48ab9764a05f2188a80b P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:30.235219 10918 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 138593us
I20260925 09:57:30.250574 10391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_3275db3804df4b12ad6e64e4d88bc75b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:30.252100 10391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3275db3804df4b12ad6e64e4d88bc75b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 09:57:30.238359 10919 leader_election.cc:341] T aca5987b14b64152887ac5ee584b257a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:30.255699 10919 leader_election.cc:302] T aca5987b14b64152887ac5ee584b257a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
W20260925 09:57:30.260193 10918 leader_election.cc:341] T f62c62715c9b42de826c1a66fd431eab P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:30.259433 10772 leader_election.cc:302] T c7cd1d6a603642afa2c8c737079961b5 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:30.261456 11961 raft_consensus.cc:2726] T c7cd1d6a603642afa2c8c737079961b5 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260925 09:57:30.261447 10387 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_bcf18c926c6e4a62b31f3bffe9b0037e" } modify_external_catalogs: true
I20260925 09:57:30.262125 10387 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_bcf18c926c6e4a62b31f3bffe9b0037e" } modify_external_catalogs: true
I20260925 09:57:30.266814 12093 raft_consensus.cc:2726] T cf775c9d9b114ddcb97dc6c6eca8e0fc P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:30.268921 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 227) took 5038 ms. Trace:
I20260925 09:57:30.269256 11727 rpcz_store.cc:276] 0925 09:57:25.230870 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:25.231938 (+ 1068us) service_pool.cc:227] Handling call
0925 09:57:28.895475 (+3663537us) ts_tablet_manager.cc:1145] Deleting tablet 67ce2e3325214aa6bdc3347e9e8f82b0
0925 09:57:28.895514 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:29.409032 (+513518us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:29.522187 (+113155us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:30.268866 (+746679us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":174995,"tablet-delete.queue_time_us":3663023}
I20260925 09:57:30.270864 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 67ce2e3325214aa6bdc3347e9e8f82b0 (table default.Table_5157be3699be4fa4ac153a69f6010ecc [id=9cd1024c9d784df9af5f8239e39d1be7]) successfully deleted
W20260925 09:57:30.091964 11084 ts_tablet_manager.cc:1462] T f62c62715c9b42de826c1a66fd431eab P be97ab5d75934a648d4ab76341193ec0: Trace:
0925 09:57:23.452435 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:23.457505 (+ 5070us) service_pool.cc:227] Handling call
0925 09:57:23.528594 (+ 71089us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0925 09:57:23.528737 (+ 143us) ts_tablet_manager.cc:676] Creating new metadata...
0925 09:57:23.583970 (+ 55233us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:23.656112 (+ 72142us) tablet_replica.cc:205] Creating consensus instance
0925 09:57:23.677792 (+ 21680us) inbound_call.cc:173] Queueing success response
0925 09:57:28.821792 (+5144000us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0925 09:57:29.467539 (+645747us) ts_tablet_manager.cc:1429] Starting tablet replica
0925 09:57:29.605648 (+138109us) tablet_replica.cc:259] Starting instrumentation
0925 09:57:29.606173 (+ 525us) tablet_replica.cc:278] Starting consensus
Metrics: {"mutex_wait_us":22818,"raft.queue_time_us":3901,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":157,"tablet-open.queue_time_us":5165249,"thread_start_us":128342,"threads_started":1}
W20260925 09:57:30.284289 10919 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 45964.9us
W20260925 09:57:30.285050 10919 leader_election.cc:341] T 69a96874eea1409a9a5a06b5787fbd99 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:30.285444 10919 leader_election.cc:302] T 69a96874eea1409a9a5a06b5787fbd99 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
W20260925 09:57:30.286386 10919 leader_election.cc:341] T dba14a44a9de492886d629441606767e P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:30.286238 10386 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_7c70dabea5b347c8b156b4b1c72703eb" } new_table_name: "default.Table_bac584c420284db7964cb9262e8a309d" modify_external_catalogs: true
I20260925 09:57:30.286790 10919 leader_election.cc:302] T dba14a44a9de492886d629441606767e P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:30.288321 11084 tablet_bootstrap.cc:492] T 9a73a1f242e7430cbf52229c07b4dc82 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
W20260925 09:57:30.311455 10380 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 5608c5f9b65b4da7b2fbb56417c92d88 (table default.Table_f37bcee5db2044f2bea9c353982b30cd [id=e46d114e5d484b66bb15eb0416b5ce0d], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 5608c5f9b65b4da7b2fbb56417c92d88 consensus state has no leader
W20260925 09:57:17.311981 11477 debug-util.cc:398] Leaking SignalData structure 0x7b0800014860 after lost signal to thread 11435
W20260925 09:57:30.316560 11477 kernel_stack_watchdog.cc:198] Thread 11435 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 424ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
I20260925 09:57:30.316860 11085 tablet_bootstrap.cc:492] T 977a59175e5b43eeb6f23b938afd00a3 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
W20260925 09:57:30.320034 11435 thread.cc:636] GCE (cloud detector) Time spent creating pthread: real 14.443s user 1.008s sys 1.873s
W20260925 09:57:30.320550 11435 thread.cc:602] GCE (cloud detector) Time spent starting thread: real 14.444s user 1.008s sys 1.874s
I20260925 09:57:30.322211 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 230) took 4769 ms. Trace:
I20260925 09:57:30.322613 11727 rpcz_store.cc:276] 0925 09:57:25.552676 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:25.552787 (+ 111us) service_pool.cc:227] Handling call
0925 09:57:30.271698 (+4718911us) ts_tablet_manager.cc:1145] Deleting tablet d9e454f82a4e4c429b58f32926319b7c
0925 09:57:30.287656 (+ 15958us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:30.322151 (+ 34495us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4714658}
W20260925 09:57:30.323953 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet d9e454f82a4e4c429b58f32926319b7c because the tablet was not found. No further retry: Not found: Tablet not found: d9e454f82a4e4c429b58f32926319b7c
I20260925 09:57:30.318249 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe91b39f91274a7ca61e9618d8cbb475" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:30.325819 10839 raft_consensus.cc:3037] T fe91b39f91274a7ca61e9618d8cbb475 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:30.317385 12093 raft_consensus.cc:2726] T 69a96874eea1409a9a5a06b5787fbd99 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:30.329525 12095 raft_consensus.cc:2726] T aca5987b14b64152887ac5ee584b257a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:30.326675 10956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a018bea4200474a9904f62aee37240b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.332741 10839 raft_consensus.cc:2454] T fe91b39f91274a7ca61e9618d8cbb475 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:30.317229 11085 ts_tablet_manager.cc:1397] T 977a59175e5b43eeb6f23b938afd00a3 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.185s user 0.014s sys 0.002s
I20260925 09:57:30.335427 10957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f46095351ea64812a7fdcfc79cd92c1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.336092 11085 raft_consensus.cc:348] T 977a59175e5b43eeb6f23b938afd00a3 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:30.336702 11085 raft_consensus.cc:374] T 977a59175e5b43eeb6f23b938afd00a3 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:30.336935 11085 raft_consensus.cc:729] T 977a59175e5b43eeb6f23b938afd00a3 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
W20260925 09:57:30.337458 10389 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b025f8ba34694ba09cb01ca959d3629c consensus state has no leader
I20260925 09:57:30.339723 10950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b025f8ba34694ba09cb01ca959d3629c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.343653 10949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 451d1984f9b144b68f0681837f8eacf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.343876 11085 consensus_queue.cc:260] T 977a59175e5b43eeb6f23b938afd00a3 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:30.346186 11085 ts_tablet_manager.cc:1428] T 977a59175e5b43eeb6f23b938afd00a3 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.012s user 0.004s sys 0.000s
I20260925 09:57:30.352521 11085 tablet_bootstrap.cc:492] T 5246648245e04e26be2abf1dbf706688 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:30.372109 11084 tablet_bootstrap.cc:654] T 9a73a1f242e7430cbf52229c07b4dc82 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:30.374367 12111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:57:30.375378 11085 tablet_bootstrap.cc:654] T 5246648245e04e26be2abf1dbf706688 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:30.375803 10391 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_1299538965b945cf859dbbd0cb7003f1" } modify_external_catalogs: true
I20260925 09:57:30.376274 10391 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_1299538965b945cf859dbbd0cb7003f1" } modify_external_catalogs: true
I20260925 09:57:30.378790 11435 server_base.cc:1034] running on GCE node
W20260925 09:57:30.381680 10386 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bc1d7b25d6ba42bc91130927c32d4ae6 consensus state has no leader
I20260925 09:57:30.399093 10386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_dc0b8cf3c1fb457985e9d19f5ef2ceb7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:30.401033 10386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dc0b8cf3c1fb457985e9d19f5ef2ceb7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:30.372567 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 231) took 4819 ms. Trace:
I20260925 09:57:30.416404 11727 rpcz_store.cc:276] 0925 09:57:25.553803 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:25.553917 (+ 114us) service_pool.cc:227] Handling call
0925 09:57:30.324858 (+4770941us) ts_tablet_manager.cc:1145] Deleting tablet 60fdc299865446c89d3e5374d573029a
0925 09:57:30.324927 (+ 69us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:30.372507 (+ 47580us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4767531}
I20260925 09:57:30.417608 10819 tablet_service.cc:1514] Processing DeleteTablet for tablet d9e454f82a4e4c429b58f32926319b7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:22 UTC) from {username='slave'} at 127.0.0.1:33344
W20260925 09:57:30.415904 10919 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 129528us
I20260925 09:57:30.419154 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet b555eeba505740988c704df5c734e7c0 (table default.Table_48f368d811494119be0dc534e2fdcfbd [id=3e32026d175b4b92a3ee036f17e54b9d]) successfully deleted
W20260925 09:57:30.419606 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 60fdc299865446c89d3e5374d573029a because the tablet was not found. No further retry: Not found: Tablet not found: 60fdc299865446c89d3e5374d573029a
W20260925 09:57:30.420635 10919 leader_election.cc:341] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:30.421049 10919 leader_election.cc:302] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:30.264672 10619 leader_election.cc:302] T 2358acdaa5104e3eacbab8bf182b8f30 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
W20260925 09:57:30.424865 10619 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 160261us
W20260925 09:57:30.413416 10621 consensus_peers.cc:487] T 3ff4cf556eee494d8cd590884466f09f P b8850818e2ee42a082802d0095c08d21 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:30.427546 12065 raft_consensus.cc:2781] T 2358acdaa5104e3eacbab8bf182b8f30 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:30.419163 10666 tablet_service.cc:1467] Processing CreateTablet for tablet dd153002e3954e98ac518bb6b5b38864 (DEFAULT_TABLE table=default.Table_7dfed08625c8462dbccb81cd1ceed336 [id=d98592e936684260a4edf3dc136160de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.427866 12065 raft_consensus.cc:482] T 2358acdaa5104e3eacbab8bf182b8f30 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:30.428150 12065 raft_consensus.cc:3037] T 2358acdaa5104e3eacbab8bf182b8f30 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:30.428491 10666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd153002e3954e98ac518bb6b5b38864. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.431811 10388 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 3d652a76d96f4dbeafde15f3c16534a6 (table default.Table_be0f658aa41f4188bc16db37e96f4ad1 [id=73083c1560cb4db5a12964387cb49b73]). Expected version 1 got 0
I20260925 09:57:30.433970 10657 tablet_service.cc:1467] Processing CreateTablet for tablet e9af6d625f0e45dd9d804ea8494b21fc (DEFAULT_TABLE table=default.Table_5a48ae73ce6b4d6e8eb352d18dd62ad5 [id=1772b4b82801445690d6ffd537663982]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.435006 10657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9af6d625f0e45dd9d804ea8494b21fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.435123 12065 raft_consensus.cc:504] T 2358acdaa5104e3eacbab8bf182b8f30 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:30.440979 10655 tablet_service.cc:1467] Processing CreateTablet for tablet d822e8ac13d9479aa9ce4f74ace55dd5 (DEFAULT_TABLE table=default.Table_574ab2069f754ba398e8d4a2e98817cf [id=3c77d857be90452696086151f02e8e77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.441645 11082 raft_consensus.cc:374] T 6161774f61ae48ab9764a05f2188a80b P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:30.441877 10655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d822e8ac13d9479aa9ce4f74ace55dd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.445214 11085 tablet_bootstrap.cc:492] T 5246648245e04e26be2abf1dbf706688 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:30.445708 11085 ts_tablet_manager.cc:1397] T 5246648245e04e26be2abf1dbf706688 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.093s user 0.010s sys 0.011s
I20260925 09:57:30.441918 11082 raft_consensus.cc:729] T 6161774f61ae48ab9764a05f2188a80b P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:30.443562 11709 raft_consensus.cc:2229] T 977a59175e5b43eeb6f23b938afd00a3 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:30.447960 10388 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 42ec275c905343c4be1b72dc2bd0f6e3 (table default.Table_7a8e20521b384bfa8f659eaaf46bae54 [id=cf07ed8756904011ab3dac900ea6705a]). Expected version 2 got 1
I20260925 09:57:30.448099 11709 raft_consensus.cc:2258] T 977a59175e5b43eeb6f23b938afd00a3 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:30.449826 11709 ts_tablet_manager.cc:1905] T 977a59175e5b43eeb6f23b938afd00a3 P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:30.449991 10388 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet c588b0c05a1a4f3ea545066183e76502 (table default.Table_bd6b1d4faeb94310be9fdecc693e5b96 [id=a2086b5e7b334d63972538ed53810c55]). Expected version 1 got 0
W20260925 09:57:15.928702 11503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:57:30.442900 12065 leader_election.cc:288] T 2358acdaa5104e3eacbab8bf182b8f30 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:30.456156 10673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61aa6380d30a4b21b70ee2d321247bd8" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:30.456764 11435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:57:30.458127 10387 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 2d087d3e35814c90a03eccd90eeea86c
I20260925 09:57:30.454227 11082 consensus_queue.cc:260] T 6161774f61ae48ab9764a05f2188a80b P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:30.459110 10673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5246648245e04e26be2abf1dbf706688" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:30.444147 10653 tablet_service.cc:1467] Processing CreateTablet for tablet 7a7389f4fe5e4d3d9ad42a79c5dd2e69 (DEFAULT_TABLE table=default.Table_d0a68d52a7f445538f3308aabc6591db [id=361c1d826f5d4a64abf5280f2baa1407]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.455895 11085 raft_consensus.cc:348] T 5246648245e04e26be2abf1dbf706688 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:30.461391 11085 raft_consensus.cc:374] T 5246648245e04e26be2abf1dbf706688 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:30.463637 10665 tablet_service.cc:1467] Processing CreateTablet for tablet 71fe7382bcb340d58a09f0b9ef255369 (DEFAULT_TABLE table=default.Table_7d5661d0c59f4ac2912f9cb12f58886a [id=8d895c1c06c043cf8c0b1f78d37ded61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.463948 10654 tablet_service.cc:1467] Processing CreateTablet for tablet cdc80b43c43e4d3a9da4fb053d7939e5 (DEFAULT_TABLE table=default.Table_5bdcdd02d06948d6b95e9c2ea4c22471 [id=7bcc19c9c8ae4de6b4df9bdbbecd0787]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.465210 10664 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d087d3e35814c90a03eccd90eeea86c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:30 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:30.461658 11085 raft_consensus.cc:729] T 5246648245e04e26be2abf1dbf706688 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:30.467473 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 232) took 4912 ms. Trace:
I20260925 09:57:30.467782 11727 rpcz_store.cc:276] 0925 09:57:25.554617 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:25.554938 (+ 321us) service_pool.cc:227] Handling call
0925 09:57:30.417236 (+4862298us) ts_tablet_manager.cc:1145] Deleting tablet 6161774f61ae48ab9764a05f2188a80b
0925 09:57:30.417277 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:30.467419 (+ 50142us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4859596}
I20260925 09:57:30.444880 10668 tablet_service.cc:1467] Processing CreateTablet for tablet 4c31466d0b8d474a8fd19fbf1b2f70cf (DEFAULT_TABLE table=default.Table_47277d736fdb436a9d4e8c03f5ec80a9 [id=ad9889ba99b446ab9a9d8165c5c729b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.468691 11727 tablet_replica.cc:330] T 3be0eccc65564bfba3d6e5c398c61fb6 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:30.468288 11085 consensus_queue.cc:260] T 5246648245e04e26be2abf1dbf706688 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:30.470064 10379 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.62:34455, user_credentials={real_user=slave}} blocked reactor thread for 45331.9us
I20260925 09:57:30.470100 10959 tablet_service.cc:1467] Processing CreateTablet for tablet d822e8ac13d9479aa9ce4f74ace55dd5 (DEFAULT_TABLE table=default.Table_574ab2069f754ba398e8d4a2e98817cf [id=3c77d857be90452696086151f02e8e77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.471056 10959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d822e8ac13d9479aa9ce4f74ace55dd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.472294 11085 ts_tablet_manager.cc:1428] T 5246648245e04e26be2abf1dbf706688 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.026s user 0.007s sys 0.000s
I20260925 09:57:30.473083 11085 tablet_bootstrap.cc:492] T 417058493c6547c69e72347cef373292 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:30.474781 10946 tablet_service.cc:1467] Processing CreateTablet for tablet e9af6d625f0e45dd9d804ea8494b21fc (DEFAULT_TABLE table=default.Table_5a48ae73ce6b4d6e8eb352d18dd62ad5 [id=1772b4b82801445690d6ffd537663982]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.480810 10808 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ff4cf556eee494d8cd590884466f09f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:29 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:30.482067 10658 tablet_service.cc:1467] Processing CreateTablet for tablet 8a1e9536f9af4ed299d1cc5e391f2500 (DEFAULT_TABLE table=default.Table_7c6bc1b63b0a45c5b67bc1fe7021a101 [id=3392fbc239084dde9fb99d793999cca1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.482298 10808 tablet_service.cc:1467] Processing CreateTablet for tablet a3edf29118e048c5a59b4ff2643ed206 (DEFAULT_TABLE table=default.Table_c91ec477f2e541c394b6cc12607a0d3e [id=49e1579274a540f58414076daa019f31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.483333 10808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3edf29118e048c5a59b4ff2643ed206. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.484328 10667 tablet_service.cc:1467] Processing CreateTablet for tablet 038c9962a12a48089300b655c56f368d (DEFAULT_TABLE table=default.Table_56638c187b5547f1b35fe75c2454fc97 [id=1d00807f5264462ca61db1d06de7e323]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.485798 10947 tablet_service.cc:1467] Processing CreateTablet for tablet dd153002e3954e98ac518bb6b5b38864 (DEFAULT_TABLE table=default.Table_7dfed08625c8462dbccb81cd1ceed336 [id=d98592e936684260a4edf3dc136160de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.486351 10387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_29f3a4e3540c4755bea01e0ff2182e83"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:30.467063 11901 raft_consensus.cc:2726] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:30.465823 12093 raft_consensus.cc:2726] T dba14a44a9de492886d629441606767e P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:30.487715 10387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_29f3a4e3540c4755bea01e0ff2182e83 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:30.487728 10664 tablet_service.cc:1467] Processing CreateTablet for tablet 4c3e66f6ad23475f9fb831afc1d579c4 (DEFAULT_TABLE table=default.Table_8f56e9a1b3504017bfabda8fd9b359ba [id=6b9c75f4db1c4d15b23284a5a6987cfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.494103 11085 tablet_bootstrap.cc:654] T 417058493c6547c69e72347cef373292 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:30.503706 11082 ts_tablet_manager.cc:1428] T 6161774f61ae48ab9764a05f2188a80b P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.262s user 0.003s sys 0.003s
I20260925 09:57:30.504743 11082 tablet_bootstrap.cc:492] T e684798155d445bfb325a1b7c5c48753 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:30.506486 10386 master_service.cc:946] ReplaceTablet: received request to replace tablet a58cf1dc81454710a9ad1a8baf155c9f from {username='slave'} at 127.0.0.1:35140
I20260925 09:57:30.506558 12080 raft_consensus.cc:482] T 6161774f61ae48ab9764a05f2188a80b P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:30.507138 12080 raft_consensus.cc:504] T 6161774f61ae48ab9764a05f2188a80b P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:30.509833 12080 leader_election.cc:288] T 6161774f61ae48ab9764a05f2188a80b P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:30.511924 10680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f62c62715c9b42de826c1a66fd431eab" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:30.513602 10390 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: default.Table_bd6b1d4faeb94310be9fdecc693e5b96 [id=a2086b5e7b334d63972538ed53810c55] alter complete (version 1)
I20260925 09:57:30.513885 10391 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 197f61525b0c4f9da441d98701d231a3
I20260925 09:57:30.513826 10646 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:37864 (request call id 23) took 1389 ms. Trace:
I20260925 09:57:30.514161 10646 rpcz_store.cc:276] 0925 09:57:29.124037 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:29.158460 (+ 34423us) service_pool.cc:227] Handling call
0925 09:57:30.164489 (+1006029us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b5400162e54. stack: 00007fb965334033 00007fb965333f30 00007fb964eb5534 00000000004db2b4 00000000004db249 00007fb96c240471 00007fb96c2b5268 00007fb96d2a16cd 00007fb96bb49e04 00007fb96bb49d92 00007fb96bb49cc1 00007fb96bb49c3c 00007fb96bb48ef2 00007fb96a06a94c 00007fb96a069d86 00007fb96a06972f
0925 09:57:30.513743 (+349254us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":272715648}
I20260925 09:57:30.516767 10658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a1e9536f9af4ed299d1cc5e391f2500. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.520296 10664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c3e66f6ad23475f9fb831afc1d579c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.520817 10665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71fe7382bcb340d58a09f0b9ef255369. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.521363 10654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdc80b43c43e4d3a9da4fb053d7939e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.522769 10667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 038c9962a12a48089300b655c56f368d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.523944 10668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c31466d0b8d474a8fd19fbf1b2f70cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.525012 10653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a7389f4fe5e4d3d9ad42a79c5dd2e69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.525714 10663 tablet_service.cc:1514] Processing DeleteTablet for tablet 197f61525b0c4f9da441d98701d231a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:30 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:30.530483 11727 raft_consensus.cc:2229] T 3be0eccc65564bfba3d6e5c398c61fb6 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:30.531713 11082 tablet_bootstrap.cc:654] T e684798155d445bfb325a1b7c5c48753 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:30.415198 12027 ts_tablet_manager.cc:1918] T 60fdc299865446c89d3e5374d573029a P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:30.532295 12027 log.cc:1196] T 60fdc299865446c89d3e5374d573029a P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/60fdc299865446c89d3e5374d573029a
I20260925 09:57:30.534740 10952 tablet_service.cc:1467] Processing CreateTablet for tablet 7a7389f4fe5e4d3d9ad42a79c5dd2e69 (DEFAULT_TABLE table=default.Table_d0a68d52a7f445538f3308aabc6591db [id=361c1d826f5d4a64abf5280f2baa1407]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.536810 10945 tablet_service.cc:1467] Processing CreateTablet for tablet cdc80b43c43e4d3a9da4fb053d7939e5 (DEFAULT_TABLE table=default.Table_5bdcdd02d06948d6b95e9c2ea4c22471 [id=7bcc19c9c8ae4de6b4df9bdbbecd0787]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.538614 10963 tablet_service.cc:1467] Processing CreateTablet for tablet 4c31466d0b8d474a8fd19fbf1b2f70cf (DEFAULT_TABLE table=default.Table_47277d736fdb436a9d4e8c03f5ec80a9 [id=ad9889ba99b446ab9a9d8165c5c729b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.539470 10819 tablet_service.cc:1467] Processing CreateTablet for tablet 451d1984f9b144b68f0681837f8eacf1 (DEFAULT_TABLE table=default.Table_7a34d8a020ed46dc8177c60c3acfb2ec [id=8b7604f5a95c4cebb355038aa0471564]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.539952 10948 tablet_service.cc:1467] Processing CreateTablet for tablet 71fe7382bcb340d58a09f0b9ef255369 (DEFAULT_TABLE table=default.Table_7d5661d0c59f4ac2912f9cb12f58886a [id=8d895c1c06c043cf8c0b1f78d37ded61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.540462 10819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 451d1984f9b144b68f0681837f8eacf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.529744 11901 raft_consensus.cc:482] T 6540aba5854a44419addc5d454e473b7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:30.544267 11901 raft_consensus.cc:504] T 6540aba5854a44419addc5d454e473b7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:30.546028 10809 tablet_service.cc:1514] Processing DeleteTablet for tablet c7cd1d6a603642afa2c8c737079961b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:29 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:30.546052 11901 leader_election.cc:288] T 6540aba5854a44419addc5d454e473b7 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:30.547796 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6540aba5854a44419addc5d454e473b7" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:30.548264 10389 master_service.cc:946] ReplaceTablet: received request to replace tablet abfc8f390d434c47a8c4990f8406c270 from {username='slave'} at 127.0.0.1:35140
I20260925 09:57:30.549283 10946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9af6d625f0e45dd9d804ea8494b21fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.545840 11082 tablet_bootstrap.cc:492] T e684798155d445bfb325a1b7c5c48753 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:30.549877 11082 ts_tablet_manager.cc:1397] T e684798155d445bfb325a1b7c5c48753 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.045s user 0.003s sys 0.012s
W20260925 09:57:30.549619 10379 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.62:34455, user_credentials={real_user=slave}} blocked reactor thread for 44906.2us
I20260925 09:57:30.552031 10948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71fe7382bcb340d58a09f0b9ef255369. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.555992 10963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c31466d0b8d474a8fd19fbf1b2f70cf. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:30.559723 10918 leader_election.cc:341] T 6540aba5854a44419addc5d454e473b7 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:30.560566 10947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd153002e3954e98ac518bb6b5b38864. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.560724 11082 raft_consensus.cc:348] T e684798155d445bfb325a1b7c5c48753 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:30.561357 11082 raft_consensus.cc:374] T e684798155d445bfb325a1b7c5c48753 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:30.561614 11082 raft_consensus.cc:729] T e684798155d445bfb325a1b7c5c48753 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:30.562390 11082 consensus_queue.cc:260] T e684798155d445bfb325a1b7c5c48753 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:30.563858 10952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a7389f4fe5e4d3d9ad42a79c5dd2e69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.566555 11082 ts_tablet_manager.cc:1428] T e684798155d445bfb325a1b7c5c48753 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.016s user 0.005s sys 0.000s
I20260925 09:57:30.567123 11727 raft_consensus.cc:2258] T 3be0eccc65564bfba3d6e5c398c61fb6 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:30.569486 11082 tablet_bootstrap.cc:492] T 5608c5f9b65b4da7b2fbb56417c92d88 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:30.572050 10814 tablet_service.cc:1467] Processing CreateTablet for tablet 76b1f79efa784662a0adf032c074ce4a (DEFAULT_TABLE table=default.Table_9d3d93d30cc442bcbabe7edfbf665469 [id=e59d5ae8365d40f08ece3a094978d896]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.572999 10814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76b1f79efa784662a0adf032c074ce4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.572896 10805 tablet_service.cc:1467] Processing CreateTablet for tablet b025f8ba34694ba09cb01ca959d3629c (DEFAULT_TABLE table=default.Table_dfc8dd8209e3467b872784a505f0d78c [id=65b6e9ec3f114cea85e70f1509e3d1b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.573635 10809 tablet_service.cc:1467] Processing CreateTablet for tablet 0a018bea4200474a9904f62aee37240b (DEFAULT_TABLE table=default.Table_14a20618504e490289d610b4f74cea84 [id=44f5211ebe4044a1bb2d528a304a1754]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.573958 10805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b025f8ba34694ba09cb01ca959d3629c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.574595 10809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a018bea4200474a9904f62aee37240b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.588456 11082 tablet_bootstrap.cc:654] T 5608c5f9b65b4da7b2fbb56417c92d88 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:30.592569 10806 tablet_service.cc:1514] Processing DeleteTablet for tablet a44d9f2f7e3e48aeb0f31fa29ae1988a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:18 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:30.592741 10812 tablet_service.cc:1514] Processing DeleteTablet for tablet 6161774f61ae48ab9764a05f2188a80b with delete_type TABLET_DATA_DELETED (replaced by tablet 10657a81058241fc988d37e5ea7ff8cf) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:30.593474 10812 tablet_service.cc:1514] Processing DeleteTablet for tablet 61aa6380d30a4b21b70ee2d321247bd8 with delete_type TABLET_DATA_DELETED (replaced by tablet 77ebb59faf3e4fb187bbce29612c25ea) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:30.593979 10815 tablet_service.cc:1467] Processing CreateTablet for tablet f46095351ea64812a7fdcfc79cd92c1f (DEFAULT_TABLE table=default.Table_cc9f467e078c448f87392bf324b040a8 [id=4cefdaddfd6143468df4502cd35afac4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.594267 10945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdc80b43c43e4d3a9da4fb053d7939e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.594924 10815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f46095351ea64812a7fdcfc79cd92c1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.596637 10807 tablet_service.cc:1467] Processing CreateTablet for tablet 4c31466d0b8d474a8fd19fbf1b2f70cf (DEFAULT_TABLE table=default.Table_47277d736fdb436a9d4e8c03f5ec80a9 [id=ad9889ba99b446ab9a9d8165c5c729b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.596632 10812 tablet_service.cc:1467] Processing CreateTablet for tablet e9af6d625f0e45dd9d804ea8494b21fc (DEFAULT_TABLE table=default.Table_5a48ae73ce6b4d6e8eb352d18dd62ad5 [id=1772b4b82801445690d6ffd537663982]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.597579 10807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c31466d0b8d474a8fd19fbf1b2f70cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.598098 10812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9af6d625f0e45dd9d804ea8494b21fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.600903 10816 tablet_service.cc:1467] Processing CreateTablet for tablet dd153002e3954e98ac518bb6b5b38864 (DEFAULT_TABLE table=default.Table_7dfed08625c8462dbccb81cd1ceed336 [id=d98592e936684260a4edf3dc136160de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.601823 10816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd153002e3954e98ac518bb6b5b38864. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.603724 10806 tablet_service.cc:1514] Processing DeleteTablet for tablet 815b4b1bd914433798dcb5deec6ec79c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:30 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:30.604265 10813 tablet_service.cc:1467] Processing CreateTablet for tablet d822e8ac13d9479aa9ce4f74ace55dd5 (DEFAULT_TABLE table=default.Table_574ab2069f754ba398e8d4a2e98817cf [id=3c77d857be90452696086151f02e8e77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.605217 10813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d822e8ac13d9479aa9ce4f74ace55dd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.608011 10818 tablet_service.cc:1467] Processing CreateTablet for tablet 7a7389f4fe5e4d3d9ad42a79c5dd2e69 (DEFAULT_TABLE table=default.Table_d0a68d52a7f445538f3308aabc6591db [id=361c1d826f5d4a64abf5280f2baa1407]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.608846 10818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a7389f4fe5e4d3d9ad42a79c5dd2e69. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:30.619339 10919 leader_election.cc:341] T 61aa6380d30a4b21b70ee2d321247bd8 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:30.619781 10919 leader_election.cc:302] T 61aa6380d30a4b21b70ee2d321247bd8 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:30.620910 10680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6540aba5854a44419addc5d454e473b7" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:30.622352 11709 ts_tablet_manager.cc:1918] T 977a59175e5b43eeb6f23b938afd00a3 P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:30.623234 11709 log.cc:1196] T 977a59175e5b43eeb6f23b938afd00a3 P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/977a59175e5b43eeb6f23b938afd00a3
I20260925 09:57:30.623673 10958 tablet_service.cc:1467] Processing CreateTablet for tablet 8a1e9536f9af4ed299d1cc5e391f2500 (DEFAULT_TABLE table=default.Table_7c6bc1b63b0a45c5b67bc1fe7021a101 [id=3392fbc239084dde9fb99d793999cca1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.624764 10958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a1e9536f9af4ed299d1cc5e391f2500. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.627789 10944 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d087d3e35814c90a03eccd90eeea86c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:30 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:30.628223 11709 ts_tablet_manager.cc:1939] T 977a59175e5b43eeb6f23b938afd00a3 P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:30.640414 10817 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d087d3e35814c90a03eccd90eeea86c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:30 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:30.640715 10810 tablet_service.cc:1467] Processing CreateTablet for tablet cdc80b43c43e4d3a9da4fb053d7939e5 (DEFAULT_TABLE table=default.Table_5bdcdd02d06948d6b95e9c2ea4c22471 [id=7bcc19c9c8ae4de6b4df9bdbbecd0787]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.641880 10810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdc80b43c43e4d3a9da4fb053d7939e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.644045 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 977a59175e5b43eeb6f23b938afd00a3 (table default.Table_ff4e7021d36b43f8b51f19960e707ac8 [id=f17a11d223204599afb24e2e9d1d9d9d]) successfully deleted
I20260925 09:57:30.646772 10960 tablet_service.cc:1467] Processing CreateTablet for tablet 038c9962a12a48089300b655c56f368d (DEFAULT_TABLE table=default.Table_56638c187b5547f1b35fe75c2454fc97 [id=1d00807f5264462ca61db1d06de7e323]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.647982 10960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 038c9962a12a48089300b655c56f368d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.651705 10944 tablet_service.cc:1467] Processing CreateTablet for tablet 4c3e66f6ad23475f9fb831afc1d579c4 (DEFAULT_TABLE table=default.Table_8f56e9a1b3504017bfabda8fd9b359ba [id=6b9c75f4db1c4d15b23284a5a6987cfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.652984 10944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c3e66f6ad23475f9fb831afc1d579c4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:30.664090 10379 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.62:34455, user_credentials={real_user=slave}} blocked reactor thread for 89907us
I20260925 09:57:30.694370 11435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:57:30.696622 10390 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_6f3d5186b0fc4a2b83567d14adb39641" } new_table_name: "default.Table_00cfe005d2584fc19576d064db0f5fd1" modify_external_catalogs: true
I20260925 09:57:30.697353 10385 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_0478cdcfe9d64b66ba0e3fbc638f9a3b" } modify_external_catalogs: true
I20260925 09:57:30.697817 10385 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_0478cdcfe9d64b66ba0e3fbc638f9a3b" } modify_external_catalogs: true
I20260925 09:57:30.700045 10384 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_9ec3b7a983bf4da4b28cb07c3cb5b6c1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:30.701002 10392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_7a410b26ea224bea88b0ea886e0fc427"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:30.701421 10384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9ec3b7a983bf4da4b28cb07c3cb5b6c1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 09:57:30.702282 10392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7a410b26ea224bea88b0ea886e0fc427 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:30.703440 11727 ts_tablet_manager.cc:1905] T 3be0eccc65564bfba3d6e5c398c61fb6 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:30.703593 10393 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_0f7925265a1741aa97617300030475a1" } new_table_name: "default.Table_8180842f664042b0a9073b7008438ff7" modify_external_catalogs: true
I20260925 09:57:30.704161 10391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_e056caa00883493e8714ca2c15ff464e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:30.705389 10391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e056caa00883493e8714ca2c15ff464e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:30.714119 10961 tablet_service.cc:1514] Processing DeleteTablet for tablet 197f61525b0c4f9da441d98701d231a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:30 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:30.631136 11709 tablet_replica.cc:330] T 60fdc299865446c89d3e5374d573029a P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:30.731613 12041 raft_consensus.cc:482] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:30.732045 12041 raft_consensus.cc:504] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:30.739199 10978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2358acdaa5104e3eacbab8bf182b8f30" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:30.739789 10978 raft_consensus.cc:3037] T 2358acdaa5104e3eacbab8bf182b8f30 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:30.740536 11082 tablet_bootstrap.cc:492] T 5608c5f9b65b4da7b2fbb56417c92d88 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:30.740911 11082 ts_tablet_manager.cc:1397] T 5608c5f9b65b4da7b2fbb56417c92d88 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.172s user 0.028s sys 0.017s
I20260925 09:57:30.742890 11709 raft_consensus.cc:2229] T 60fdc299865446c89d3e5374d573029a P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:30.743093 11082 raft_consensus.cc:348] T 5608c5f9b65b4da7b2fbb56417c92d88 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:30.743762 11709 pending_rounds.cc:62] T 60fdc299865446c89d3e5374d573029a P b8850818e2ee42a082802d0095c08d21: Trying to abort 1 pending ops.
I20260925 09:57:30.743816 11082 raft_consensus.cc:374] T 5608c5f9b65b4da7b2fbb56417c92d88 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:30.744146 11082 raft_consensus.cc:729] T 5608c5f9b65b4da7b2fbb56417c92d88 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:30.744119 11709 pending_rounds.cc:68] T 60fdc299865446c89d3e5374d573029a P b8850818e2ee42a082802d0095c08d21: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192703072145408 op_type: NO_OP noop_request { }
I20260925 09:57:30.744635 11709 raft_consensus.cc:2866] T 60fdc299865446c89d3e5374d573029a P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:57:30.744870 12041 leader_election.cc:288] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:30.744956 11709 raft_consensus.cc:2258] T 60fdc299865446c89d3e5374d573029a P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:30.744776 11082 consensus_queue.cc:260] T 5608c5f9b65b4da7b2fbb56417c92d88 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:30.746763 11082 ts_tablet_manager.cc:1428] T 5608c5f9b65b4da7b2fbb56417c92d88 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260925 09:57:30.752182 10978 raft_consensus.cc:2454] T 2358acdaa5104e3eacbab8bf182b8f30 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:30.753242 11082 tablet_bootstrap.cc:492] T 5246648245e04e26be2abf1dbf706688 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:30.753767 10389 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_40181c2d583b42b89dbb7e1f589f5317" } modify_external_catalogs: true
I20260925 09:57:30.754262 10389 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_40181c2d583b42b89dbb7e1f589f5317" } modify_external_catalogs: true
I20260925 09:57:30.758194 11082 tablet_bootstrap.cc:654] T 5246648245e04e26be2abf1dbf706688 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:30.765853 10619 leader_election.cc:341] T 6161774f61ae48ab9764a05f2188a80b P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 6161774f61ae48ab9764a05f2188a80b
W20260925 09:57:30.773134 10951 consensus_meta.cc:287] T a3edf29118e048c5a59b4ff2643ed206 P be97ab5d75934a648d4ab76341193ec0: Time spent flushing consensus metadata: real 0.623s user 0.009s sys 0.000s
I20260925 09:57:30.774758 11082 tablet_bootstrap.cc:492] T 5246648245e04e26be2abf1dbf706688 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:30.775282 11082 ts_tablet_manager.cc:1397] T 5246648245e04e26be2abf1dbf706688 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.022s user 0.004s sys 0.006s
I20260925 09:57:30.777451 11082 raft_consensus.cc:348] T 5246648245e04e26be2abf1dbf706688 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:30.778167 11082 raft_consensus.cc:374] T 5246648245e04e26be2abf1dbf706688 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:30.778452 11082 raft_consensus.cc:729] T 5246648245e04e26be2abf1dbf706688 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:30.779120 11082 consensus_queue.cc:260] T 5246648245e04e26be2abf1dbf706688 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:30.792896 11082 ts_tablet_manager.cc:1428] T 5246648245e04e26be2abf1dbf706688 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.017s user 0.004s sys 0.000s
I20260925 09:57:30.800063 11082 tablet_bootstrap.cc:492] T 417058493c6547c69e72347cef373292 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
W20260925 09:57:30.804939 10919 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 185620us
W20260925 09:57:30.805608 10919 leader_election.cc:341] T 3be0eccc65564bfba3d6e5c398c61fb6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Not found: Tablet not found: 3be0eccc65564bfba3d6e5c398c61fb6
I20260925 09:57:30.806011 10919 leader_election.cc:302] T 3be0eccc65564bfba3d6e5c398c61fb6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
W20260925 09:57:30.807940 10772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 43702.2us
I20260925 09:57:30.809652 11435 hybrid_clock.cc:648] HybridClock initialized: now 1790330250809580 us; error 67 us; skew 500 ppm
I20260925 09:57:30.811239 11435 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:57:30.815760 12041 raft_consensus.cc:2726] T 61aa6380d30a4b21b70ee2d321247bd8 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:30.833001 10386 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet a58cf1dc81454710a9ad1a8baf155c9f
I20260925 09:57:30.834494 10386 catalog_manager.cc:6565] ReplaceTablet: tablet a58cf1dc81454710a9ad1a8baf155c9f deleted and replaced by tablet ddaa47d7287b4f1b94f999f81e1583f2
I20260925 09:57:30.842007 10388 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: default.Table_0011387b45014de4b24b466f4cb907f7 [id=e3f7aa5e988c4b6c98be5cd3d465245d] alter complete (version 1)
W20260925 09:57:30.844543 10955 consensus_meta.cc:287] T 76b1f79efa784662a0adf032c074ce4a P be97ab5d75934a648d4ab76341193ec0: Time spent flushing consensus metadata: real 0.519s user 0.006s sys 0.002s
W20260925 09:57:30.844686 10393 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6540aba5854a44419addc5d454e473b7 consensus state has no leader
I20260925 09:57:30.846032 11082 tablet_bootstrap.cc:654] T 417058493c6547c69e72347cef373292 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:30.809055 10805 tablet_service.cc:1467] Processing CreateTablet for tablet 038c9962a12a48089300b655c56f368d (DEFAULT_TABLE table=default.Table_56638c187b5547f1b35fe75c2454fc97 [id=1d00807f5264462ca61db1d06de7e323]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.849543 10817 tablet_service.cc:1467] Processing CreateTablet for tablet 4c3e66f6ad23475f9fb831afc1d579c4 (DEFAULT_TABLE table=default.Table_8f56e9a1b3504017bfabda8fd9b359ba [id=6b9c75f4db1c4d15b23284a5a6987cfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.850270 10805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 038c9962a12a48089300b655c56f368d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:30.851982 10390 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8f3112845dc744d083c0f5e442f0aef7 has no consensus state
I20260925 09:57:30.807291 10814 tablet_service.cc:1467] Processing CreateTablet for tablet 8a1e9536f9af4ed299d1cc5e391f2500 (DEFAULT_TABLE table=default.Table_7c6bc1b63b0a45c5b67bc1fe7021a101 [id=3392fbc239084dde9fb99d793999cca1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:30.853674 10814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a1e9536f9af4ed299d1cc5e391f2500. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.860771 10657 tablet_service.cc:1514] Processing DeleteTablet for tablet a58cf1dc81454710a9ad1a8baf155c9f with delete_type TABLET_DATA_DELETED (replaced by tablet ddaa47d7287b4f1b94f999f81e1583f2) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:30.864738 10811 tablet_service.cc:1514] Processing DeleteTablet for tablet 197f61525b0c4f9da441d98701d231a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:30 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:30.867547 10817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c3e66f6ad23475f9fb831afc1d579c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.868196 10619 leader_election.cc:302] T 2358acdaa5104e3eacbab8bf182b8f30 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:30.869107 12080 raft_consensus.cc:2781] T 2358acdaa5104e3eacbab8bf182b8f30 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:30.869979 12080 raft_consensus.cc:686] T 2358acdaa5104e3eacbab8bf182b8f30 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:30.870869 12080 consensus_queue.cc:237] T 2358acdaa5104e3eacbab8bf182b8f30 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:30.879114 11709 ts_tablet_manager.cc:1905] T 60fdc299865446c89d3e5374d573029a P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:30.885270 12027 ts_tablet_manager.cc:1939] T 60fdc299865446c89d3e5374d573029a P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:30.880927 12113 raft_consensus.cc:482] T 5246648245e04e26be2abf1dbf706688 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:30.901608 12113 raft_consensus.cc:504] T 5246648245e04e26be2abf1dbf706688 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:30.903935 12113 leader_election.cc:288] T 5246648245e04e26be2abf1dbf706688 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:30.912812 10981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5246648245e04e26be2abf1dbf706688" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:30.913619 10981 raft_consensus.cc:2454] T 5246648245e04e26be2abf1dbf706688 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:30.917022 10619 leader_election.cc:302] T 5246648245e04e26be2abf1dbf706688 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:30.917953 12113 raft_consensus.cc:2781] T 5246648245e04e26be2abf1dbf706688 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:30.918310 12113 raft_consensus.cc:482] T 5246648245e04e26be2abf1dbf706688 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:30.918642 12113 raft_consensus.cc:3037] T 5246648245e04e26be2abf1dbf706688 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:30.905462 10806 tablet_service.cc:1467] Processing CreateTablet for tablet 71fe7382bcb340d58a09f0b9ef255369 (DEFAULT_TABLE table=default.Table_7d5661d0c59f4ac2912f9cb12f58886a [id=8d895c1c06c043cf8c0b1f78d37ded61]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 09:57:30.921147 10772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 112652us
I20260925 09:57:30.885104 10834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aba253f22afa4200926f31a1b43e3970" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:30.930449 11961 raft_consensus.cc:482] T 5246648245e04e26be2abf1dbf706688 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:30.930863 11961 raft_consensus.cc:504] T 5246648245e04e26be2abf1dbf706688 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:30.932621 11961 leader_election.cc:288] T 5246648245e04e26be2abf1dbf706688 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
W20260925 09:57:30.864257 10919 raft_consensus.cc:2670] T 3be0eccc65564bfba3d6e5c398c61fb6 P be97ab5d75934a648d4ab76341193ec0: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260925 09:57:30.935817 10919 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 130227us
W20260925 09:57:30.936875 10919 leader_election.cc:341] T 5246648245e04e26be2abf1dbf706688 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:30.937264 10919 leader_election.cc:302] T 5246648245e04e26be2abf1dbf706688 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:30.941277 12113 raft_consensus.cc:504] T 5246648245e04e26be2abf1dbf706688 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:30.946107 10919 leader_election.cc:341] T f62c62715c9b42de826c1a66fd431eab P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:30.946514 10919 leader_election.cc:302] T f62c62715c9b42de826c1a66fd431eab P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:30.950443 10806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71fe7382bcb340d58a09f0b9ef255369. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:30.948423 11901 raft_consensus.cc:2726] T 5246648245e04e26be2abf1dbf706688 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:30.954123 10834 raft_consensus.cc:2454] T aba253f22afa4200926f31a1b43e3970 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be97ab5d75934a648d4ab76341193ec0 in term 1.
I20260925 09:57:30.973407 10918 leader_election.cc:302] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:30.974320 12041 raft_consensus.cc:2781] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 09:57:30.974632 12041 raft_consensus.cc:482] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:30.974916 12041 raft_consensus.cc:3037] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:57:30.985385 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2358acdaa5104e3eacbab8bf182b8f30" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:30.985899 10835 raft_consensus.cc:3037] T 2358acdaa5104e3eacbab8bf182b8f30 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:30.986259 12041 raft_consensus.cc:504] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:30.988138 10390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_219dc5152bc1486a948a0e678b1eb15e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:30.989859 10390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_219dc5152bc1486a948a0e678b1eb15e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 09:57:30.990543 10919 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 44461.9us
W20260925 09:57:30.991430 10919 leader_election.cc:341] T 6540aba5854a44419addc5d454e473b7 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:30.991840 10919 leader_election.cc:302] T 6540aba5854a44419addc5d454e473b7 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:31.044592 11435 webserver.cc:460] Webserver started at http://127.10.9.60:41643/ using document root <none> and password file <none>
I20260925 09:57:31.052661 11435 fs_manager.cc:359] Metadata directory not provided
I20260925 09:57:31.053036 11435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:57:31.060448 10391 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_bd0dc294c2c049e9b9c9eddce6772a44" } new_table_name: "default.Table_baec870bf67344cab7091ef1e210eb9a" modify_external_catalogs: true
I20260925 09:57:31.060956 10835 raft_consensus.cc:2454] T 2358acdaa5104e3eacbab8bf182b8f30 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:31.062781 10393 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_5bb8cffc4b434151b44aa2a11097fa19" } new_table_name: "default.Table_dfb73f4afe7c464d8dc98356500054e2" modify_external_catalogs: true
I20260925 09:57:31.065771 10386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_b6e9ba234e6140479745055b3237faeb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:31.067131 10386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b6e9ba234e6140479745055b3237faeb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:31.010632 11901 raft_consensus.cc:2726] T f62c62715c9b42de826c1a66fd431eab P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:30.987594 10956 tablet_service.cc:1514] Processing DeleteTablet for tablet a58cf1dc81454710a9ad1a8baf155c9f with delete_type TABLET_DATA_DELETED (replaced by tablet ddaa47d7287b4f1b94f999f81e1583f2) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:31.031908 11084 tablet_bootstrap.cc:492] T 9a73a1f242e7430cbf52229c07b4dc82 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:31.079280 11084 ts_tablet_manager.cc:1397] T 9a73a1f242e7430cbf52229c07b4dc82 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.791s user 0.010s sys 0.004s
I20260925 09:57:31.081415 11084 raft_consensus.cc:348] T 9a73a1f242e7430cbf52229c07b4dc82 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:31.092127 11085 thread.cc:636] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.542s user 0.038s sys 0.085s
W20260925 09:57:31.092417 11085 thread.cc:602] log-alloc [worker] (thread pool) Time spent starting thread: real 0.542s user 0.038s sys 0.085s
I20260925 09:57:31.009749 11082 tablet_bootstrap.cc:492] T 417058493c6547c69e72347cef373292 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:31.099365 11082 ts_tablet_manager.cc:1397] T 417058493c6547c69e72347cef373292 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.299s user 0.014s sys 0.012s
I20260925 09:57:31.101568 11082 raft_consensus.cc:348] T 417058493c6547c69e72347cef373292 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:31.102213 11082 raft_consensus.cc:374] T 417058493c6547c69e72347cef373292 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:31.102434 11082 raft_consensus.cc:729] T 417058493c6547c69e72347cef373292 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:31.102962 11082 consensus_queue.cc:260] T 417058493c6547c69e72347cef373292 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:31.105352 11082 ts_tablet_manager.cc:1428] T 417058493c6547c69e72347cef373292 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260925 09:57:31.106230 11082 tablet_bootstrap.cc:492] T 69a96874eea1409a9a5a06b5787fbd99 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:31.113260 12123 raft_consensus.cc:482] T 977a59175e5b43eeb6f23b938afd00a3 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:31.113704 12123 raft_consensus.cc:504] T 977a59175e5b43eeb6f23b938afd00a3 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:31.114755 11082 tablet_bootstrap.cc:654] T 69a96874eea1409a9a5a06b5787fbd99 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:31.066634 10619 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 72967.5us
I20260925 09:57:31.072201 10680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5246648245e04e26be2abf1dbf706688" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:31.118597 12113 leader_election.cc:288] T 5246648245e04e26be2abf1dbf706688 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:31.118520 10833 raft_consensus.cc:3037] T 3ff4cf556eee494d8cd590884466f09f P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:31.118571 12123 leader_election.cc:288] T 977a59175e5b43eeb6f23b938afd00a3 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
W20260925 09:57:31.119778 10775 leader_election.cc:341] T 977a59175e5b43eeb6f23b938afd00a3 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Not found: Tablet not found: 977a59175e5b43eeb6f23b938afd00a3
I20260925 09:57:31.123925 10680 raft_consensus.cc:2379] T 5246648245e04e26be2abf1dbf706688 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c43461c4ee854ae1ac52b873ccada318 in current term 1: Already voted for candidate b8850818e2ee42a082802d0095c08d21 in this term.
I20260925 09:57:31.044138 12095 raft_consensus.cc:482] T a436fe287a0d48a1821191adcd031d74 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:31.127000 12095 raft_consensus.cc:504] T a436fe287a0d48a1821191adcd031d74 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:31.128934 12095 leader_election.cc:288] T a436fe287a0d48a1821191adcd031d74 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:31.067639 10978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5246648245e04e26be2abf1dbf706688" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:31.129727 10978 raft_consensus.cc:2454] T 5246648245e04e26be2abf1dbf706688 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
W20260925 09:57:31.039521 10950 consensus_meta.cc:287] T b025f8ba34694ba09cb01ca959d3629c P be97ab5d75934a648d4ab76341193ec0: Time spent flushing consensus metadata: real 0.649s user 0.009s sys 0.000s
I20260925 09:57:31.131209 10950 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53560 (request call id 310) took 1015 ms. Trace:
I20260925 09:57:31.131649 10950 rpcz_store.cc:276] 0925 09:57:30.115361 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:30.115870 (+ 509us) service_pool.cc:227] Handling call
0925 09:57:30.234045 (+118175us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0925 09:57:30.234115 (+ 70us) ts_tablet_manager.cc:676] Creating new metadata...
0925 09:57:30.389669 (+155554us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:31.130772 (+741103us) tablet_replica.cc:205] Creating consensus instance
0925 09:57:31.131160 (+ 388us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260925 09:57:31.082162 12041 leader_election.cc:288] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:31.132983 11084 raft_consensus.cc:374] T 9a73a1f242e7430cbf52229c07b4dc82 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:31.135519 12065 raft_consensus.cc:482] T e684798155d445bfb325a1b7c5c48753 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:31.135968 12065 raft_consensus.cc:504] T e684798155d445bfb325a1b7c5c48753 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:31.136943 11084 raft_consensus.cc:729] T 9a73a1f242e7430cbf52229c07b4dc82 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:31.137639 11084 consensus_queue.cc:260] T 9a73a1f242e7430cbf52229c07b4dc82 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:31.138187 12065 leader_election.cc:288] T e684798155d445bfb325a1b7c5c48753 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:31.138602 10830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aba253f22afa4200926f31a1b43e3970" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:31.139199 10830 raft_consensus.cc:3037] T aba253f22afa4200926f31a1b43e3970 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:57:31.086062 11901 raft_consensus.cc:2726] T 6540aba5854a44419addc5d454e473b7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:31.086776 10919 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 95374us
I20260925 09:57:31.085661 11727 ts_tablet_manager.cc:1918] T 3be0eccc65564bfba3d6e5c398c61fb6 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:31.144675 11727 log.cc:1196] T 3be0eccc65564bfba3d6e5c398c61fb6 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/3be0eccc65564bfba3d6e5c398c61fb6
I20260925 09:57:31.145419 11727 ts_tablet_manager.cc:1939] T 3be0eccc65564bfba3d6e5c398c61fb6 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:31.147660 12125 raft_consensus.cc:482] T d9e454f82a4e4c429b58f32926319b7c P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:31.148118 12125 raft_consensus.cc:504] T d9e454f82a4e4c429b58f32926319b7c P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:31.150020 12125 leader_election.cc:288] T d9e454f82a4e4c429b58f32926319b7c P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:31.151149 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6161774f61ae48ab9764a05f2188a80b" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:31.151546 12039 raft_consensus.cc:482] T ab6bdba5ca5d41ae921a90c1f77006c5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:31.151780 10832 raft_consensus.cc:2454] T 6161774f61ae48ab9764a05f2188a80b P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:31.151978 12039 raft_consensus.cc:504] T ab6bdba5ca5d41ae921a90c1f77006c5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:31.153726 12039 leader_election.cc:288] T ab6bdba5ca5d41ae921a90c1f77006c5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
W20260925 09:57:31.156102 10775 leader_election.cc:341] T d9e454f82a4e4c429b58f32926319b7c P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Not found: Tablet not found: d9e454f82a4e4c429b58f32926319b7c
I20260925 09:57:31.171041 11085 tablet_bootstrap.cc:492] T 417058493c6547c69e72347cef373292 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:31.171550 11085 ts_tablet_manager.cc:1397] T 417058493c6547c69e72347cef373292 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.699s user 0.049s sys 0.086s
I20260925 09:57:31.177057 10978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5246648245e04e26be2abf1dbf706688" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:31.177567 10978 raft_consensus.cc:3037] T 5246648245e04e26be2abf1dbf706688 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:31.182966 10830 raft_consensus.cc:2454] T aba253f22afa4200926f31a1b43e3970 P c43461c4ee854ae1ac52b873ccada318 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be97ab5d75934a648d4ab76341193ec0 in term 2.
I20260925 09:57:31.193980 10388 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: default.Table_01f37e9d4f5b4dc5a66a24eac603810d [id=64032d286bf94c4cbdddde18adfb739e] alter complete (version 2)
I20260925 09:57:31.198082 10978 raft_consensus.cc:2454] T 5246648245e04e26be2abf1dbf706688 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:31.199759 10653 tablet_service.cc:1514] Processing DeleteTablet for tablet 977a59175e5b43eeb6f23b938afd00a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:18 UTC) from {username='slave'} at 127.0.0.1:37876
W20260925 09:57:31.199517 10388 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.1:56563 (request call id 38) took 996 ms (client timeout 1000 ms). Trace:
I20260925 09:57:31.204511 10981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e684798155d445bfb325a1b7c5c48753" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:31.205156 10981 raft_consensus.cc:2454] T e684798155d445bfb325a1b7c5c48753 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:31.232143 11082 tablet_bootstrap.cc:492] T 69a96874eea1409a9a5a06b5787fbd99 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:31.232609 11082 ts_tablet_manager.cc:1397] T 69a96874eea1409a9a5a06b5787fbd99 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.127s user 0.006s sys 0.003s
I20260925 09:57:31.235800 11082 raft_consensus.cc:348] T 69a96874eea1409a9a5a06b5787fbd99 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:31.200104 10388 rpcz_store.cc:269] 0925 09:57:30.203070 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:30.203169 (+ 99us) service_pool.cc:227] Handling call
0925 09:57:30.255658 (+ 52489us) spinlock_profiling.cc:243] Waited 48.3 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2ad9c3e 00007fabf0610fb4 00007fabf0610f42 00007fabf0610e71 00007fabf0610dec 00007fabf06100a2 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:31.199458 (+943800us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:30.709256 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:30.709613 (+ 357us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:30.709635 (+ 22us) write_op.cc:435] Acquired schema lock
0925 09:57:30.709646 (+ 11us) tablet.cc:660] Decoding operations
0925 09:57:30.709869 (+ 223us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:30.709902 (+ 33us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:30.709923 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:30.710048 (+ 125us) tablet.cc:699] Row locks acquired
0925 09:57:30.710060 (+ 12us) write_op.cc:260] PREPARE: finished
0925 09:57:30.710157 (+ 97us) write_op.cc:270] Start()
0925 09:57:30.710230 (+ 73us) write_op.cc:276] Timestamp: P: 1790330250710123 usec, L: 0
0925 09:57:30.710245 (+ 15us) op_driver.cc:331] REPLICATION: starting
0925 09:57:30.710728 (+ 483us) log.cc:842] Serialized 356 byte log entry
0925 09:57:30.839741 (+129013us) op_driver.cc:444] REPLICATION: finished
0925 09:57:30.839980 (+ 239us) write_op.cc:301] APPLY: starting
0925 09:57:30.840038 (+ 58us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:30.840155 (+ 117us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:30.840259 (+ 104us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:30.840451 (+ 192us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:30.840614 (+ 163us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:30.840674 (+ 60us) write_op.cc:312] APPLY: finished
0925 09:57:30.841406 (+ 732us) log.cc:842] Serialized 23 byte log entry
0925 09:57:30.841575 (+ 169us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:30.841647 (+ 72us) write_op.cc:454] Released schema lock
0925 09:57:30.841686 (+ 39us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:30.841857 (+ 171us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:30.841879 (+ 22us) write_op.cc:454] Released schema lock
Related trace 'op':
0925 09:57:30.999775 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:31.000250 (+ 475us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:31.000285 (+ 35us) write_op.cc:435] Acquired schema lock
0925 09:57:31.000300 (+ 15us) tablet.cc:660] Decoding operations
0925 09:57:31.000608 (+ 308us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:31.000656 (+ 48us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:31.000688 (+ 32us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:31.000871 (+ 183us) tablet.cc:699] Row locks acquired
0925 09:57:31.000889 (+ 18us) write_op.cc:260] PREPARE: finished
0925 09:57:31.001018 (+ 129us) write_op.cc:270] Start()
0925 09:57:31.001117 (+ 99us) write_op.cc:276] Timestamp: P: 1790330251000953 usec, L: 0
0925 09:57:31.001139 (+ 22us) op_driver.cc:331] REPLICATION: starting
0925 09:57:31.001629 (+ 490us) log.cc:842] Serialized 356 byte log entry
0925 09:57:31.192200 (+190571us) op_driver.cc:444] REPLICATION: finished
0925 09:57:31.192664 (+ 464us) write_op.cc:301] APPLY: starting
0925 09:57:31.192716 (+ 52us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:31.192821 (+ 105us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:31.192910 (+ 89us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:31.193068 (+ 158us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:31.193190 (+ 122us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:31.193228 (+ 38us) write_op.cc:312] APPLY: finished
0925 09:57:31.193392 (+ 164us) log.cc:842] Serialized 23 byte log entry
0925 09:57:31.193532 (+ 140us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:31.193593 (+ 61us) write_op.cc:454] Released schema lock
0925 09:57:31.193621 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:31.193791 (+ 170us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:31.193812 (+ 21us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":28,"spinlock_wait_cycles":111111424,"child_traces":[["op",{"apply.queue_time_us":80,"apply.run_cpu_time_us":1533,"apply.run_wall_time_us":2048,"num_ops":1,"prepare.queue_time_us":184677,"prepare.run_cpu_time_us":2229,"prepare.run_wall_time_us":2501,"raft.queue_time_us":200,"raft.run_cpu_time_us":11,"raft.run_wall_time_us":11,"replication_time_us":129416}],["op",{"apply.queue_time_us":329,"apply.run_cpu_time_us":1284,"apply.run_wall_time_us":1284,"num_ops":1,"prepare.queue_time_us":152328,"prepare.run_cpu_time_us":2118,"prepare.run_wall_time_us":3155,"replication_time_us":190985}]]}
I20260925 09:57:31.241389 10388 master_service.cc:946] ReplaceTablet: received request to replace tablet 0a018bea4200474a9904f62aee37240b from {username='slave'} at 127.0.0.1:35140
W20260925 09:57:31.242350 10919 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 97859.4us
I20260925 09:57:31.243472 11082 raft_consensus.cc:374] T 69a96874eea1409a9a5a06b5787fbd99 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:31.186249 10824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e684798155d445bfb325a1b7c5c48753" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:31.188117 10825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5246648245e04e26be2abf1dbf706688" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:31.252957 10825 raft_consensus.cc:3037] T 5246648245e04e26be2abf1dbf706688 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:31.260667 10825 raft_consensus.cc:2454] T 5246648245e04e26be2abf1dbf706688 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:31.206792 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab6bdba5ca5d41ae921a90c1f77006c5" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:31.205644 10829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a436fe287a0d48a1821191adcd031d74" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:31.210181 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 290) took 1565 ms. Trace:
I20260925 09:57:31.262166 12027 rpcz_store.cc:276] 0925 09:57:29.645094 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:29.645194 (+ 100us) service_pool.cc:227] Handling call
0925 09:57:29.797367 (+152173us) ts_tablet_manager.cc:1145] Deleting tablet 60fdc299865446c89d3e5374d573029a
0925 09:57:29.797408 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:30.408157 (+610749us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:30.414977 (+ 6820us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:31.114903 (+699926us) spinlock_profiling.cc:243] Waited 13.8 ms on lock 0x7b5800001898. stack: 00007f0270186033 00007f0270185f30 00007f026fd07534 00000000004db2b4 00000000004db249 00007f026fffcf11 00007f0270180851 00007f0270180fee 00007f02790408f9 00007f02780746a1 00007f0278138ef6 00007f0278168e10 00007f027814fe0d 00007f027814fdb9 00007f027814fd49 00007f027814fd11
0925 09:57:31.210120 (+ 95217us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":31645952,"tablet-delete.queue_time_us":151558}
I20260925 09:57:31.263865 12027 tablet_replica.cc:330] T aba253f22afa4200926f31a1b43e3970 P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:31.212677 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5246648245e04e26be2abf1dbf706688" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:31.265125 10832 raft_consensus.cc:2362] T 5246648245e04e26be2abf1dbf706688 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1. Re-sending same reply.
I20260925 09:57:31.266010 11082 raft_consensus.cc:729] T 69a96874eea1409a9a5a06b5787fbd99 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:31.227330 11085 raft_consensus.cc:348] T 417058493c6547c69e72347cef373292 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:31.229888 10772 leader_election.cc:302] T 5246648245e04e26be2abf1dbf706688 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21
I20260925 09:57:31.267320 11082 consensus_queue.cc:260] T 69a96874eea1409a9a5a06b5787fbd99 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:31.268405 10772 consensus_peers.cc:487] T a44d9f2f7e3e48aeb0f31fa29ae1988a P c43461c4ee854ae1ac52b873ccada318 -> Peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Couldn't send request to peer be97ab5d75934a648d4ab76341193ec0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a44d9f2f7e3e48aeb0f31fa29ae1988a. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:31.268921 12067 raft_consensus.cc:2741] T 5246648245e04e26be2abf1dbf706688 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260925 09:57:31.269778 11082 ts_tablet_manager.cc:1428] T 69a96874eea1409a9a5a06b5787fbd99 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.037s user 0.005s sys 0.000s
I20260925 09:57:31.243115 11709 ts_tablet_manager.cc:1918] T 60fdc299865446c89d3e5374d573029a P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:31.251093 12113 consensus_queue.cc:1035] T c7cd1d6a603642afa2c8c737079961b5 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:31.252652 12134 consensus_queue.cc:1035] T fe91b39f91274a7ca61e9618d8cbb475 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:31.254340 10619 leader_election.cc:302] T 5246648245e04e26be2abf1dbf706688 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:31.248669 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab6bdba5ca5d41ae921a90c1f77006c5" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:31.272143 10619 leader_election.cc:302] T e684798155d445bfb325a1b7c5c48753 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
W20260925 09:57:31.272812 10918 leader_election.cc:341] T a436fe287a0d48a1821191adcd031d74 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:31.273329 10918 leader_election.cc:341] T ab6bdba5ca5d41ae921a90c1f77006c5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:31.274179 12133 raft_consensus.cc:2781] T 5246648245e04e26be2abf1dbf706688 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:31.274878 12080 raft_consensus.cc:2781] T e684798155d445bfb325a1b7c5c48753 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:31.275162 12080 raft_consensus.cc:482] T e684798155d445bfb325a1b7c5c48753 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:31.275460 12080 raft_consensus.cc:3037] T e684798155d445bfb325a1b7c5c48753 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:31.234380 12137 tablet.cc:1722] T 59f0c6489a33433899bb6e62088121b6 P c43461c4ee854ae1ac52b873ccada318: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 09:57:31.246577 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a436fe287a0d48a1821191adcd031d74" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:31.286697 11901 raft_consensus.cc:482] T defaef4d6b754ca5a2dfc21b7a596a5b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:31.287115 11901 raft_consensus.cc:504] T defaef4d6b754ca5a2dfc21b7a596a5b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:31.288972 12141 raft_consensus.cc:482] T 10657a81058241fc988d37e5ea7ff8cf P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:31.289479 12141 raft_consensus.cc:504] T 10657a81058241fc988d37e5ea7ff8cf P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:31.291620 12141 leader_election.cc:288] T 10657a81058241fc988d37e5ea7ff8cf P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:31.293118 11901 leader_election.cc:288] T defaef4d6b754ca5a2dfc21b7a596a5b P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
W20260925 09:57:31.295471 10772 leader_election.cc:341] T 977a59175e5b43eeb6f23b938afd00a3 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 977a59175e5b43eeb6f23b938afd00a3
I20260925 09:57:31.295738 11084 ts_tablet_manager.cc:1428] T 9a73a1f242e7430cbf52229c07b4dc82 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.216s user 0.006s sys 0.000s
I20260925 09:57:31.295918 10772 leader_election.cc:302] T 977a59175e5b43eeb6f23b938afd00a3 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:31.296690 11084 tablet_bootstrap.cc:492] T 9744b9814b69470396ce039348fe5e45 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
W20260925 09:57:31.297354 10772 leader_election.cc:341] T d9e454f82a4e4c429b58f32926319b7c P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: d9e454f82a4e4c429b58f32926319b7c
I20260925 09:57:31.297775 10772 leader_election.cc:302] T d9e454f82a4e4c429b58f32926319b7c P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:31.299326 12096 tablet.cc:1722] T 42ec275c905343c4be1b72dc2bd0f6e3 P be97ab5d75934a648d4ab76341193ec0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 09:57:31.300261 12123 raft_consensus.cc:2726] T 977a59175e5b43eeb6f23b938afd00a3 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:31.300746 12123 raft_consensus.cc:2726] T d9e454f82a4e4c429b58f32926319b7c P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:31.301491 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10657a81058241fc988d37e5ea7ff8cf" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:31.304021 10918 leader_election.cc:341] T 10657a81058241fc988d37e5ea7ff8cf P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:31.307668 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "defaef4d6b754ca5a2dfc21b7a596a5b" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:31.293066 12080 raft_consensus.cc:504] T e684798155d445bfb325a1b7c5c48753 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:31.312053 10918 leader_election.cc:341] T defaef4d6b754ca5a2dfc21b7a596a5b P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:31.236155 12138 tablet.cc:1722] T 26fae3fbfe374f348a1729c21ec751b9 P c43461c4ee854ae1ac52b873ccada318: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:31.312737 11709 log.cc:1196] T 60fdc299865446c89d3e5374d573029a P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/60fdc299865446c89d3e5374d573029a
I20260925 09:57:31.313616 11709 ts_tablet_manager.cc:1939] T 60fdc299865446c89d3e5374d573029a P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:31.318179 11085 raft_consensus.cc:374] T 417058493c6547c69e72347cef373292 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:31.323707 10673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10657a81058241fc988d37e5ea7ff8cf" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:31.323741 10681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "defaef4d6b754ca5a2dfc21b7a596a5b" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:31.325714 11082 tablet_bootstrap.cc:492] T d5956be0da6441c79723e72f12dfe54a P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:31.326143 11085 raft_consensus.cc:729] T 417058493c6547c69e72347cef373292 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
W20260925 09:57:31.314961 10919 leader_election.cc:341] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Not found: Tablet not found: aba253f22afa4200926f31a1b43e3970
I20260925 09:57:31.316483 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 233) took 5758 ms. Trace:
I20260925 09:57:31.328508 11727 rpcz_store.cc:276] 0925 09:57:25.558009 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:25.558099 (+ 90us) service_pool.cc:227] Handling call
0925 09:57:30.468552 (+4910453us) ts_tablet_manager.cc:1145] Deleting tablet 3be0eccc65564bfba3d6e5c398c61fb6
0925 09:57:30.468596 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:30.807857 (+339261us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:31.085467 (+277610us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:31.316424 (+230957us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":230021,"tablet-delete.queue_time_us":4909842}
I20260925 09:57:31.330569 11085 consensus_queue.cc:260] T 417058493c6547c69e72347cef373292 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:31.331684 11709 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 297) took 1637 ms. Trace:
I20260925 09:57:31.332115 11709 rpcz_store.cc:276] 0925 09:57:29.694325 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:29.705867 (+ 11542us) service_pool.cc:227] Handling call
0925 09:57:30.630916 (+925049us) ts_tablet_manager.cc:1145] Deleting tablet 60fdc299865446c89d3e5374d573029a
0925 09:57:30.630978 (+ 62us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:31.143329 (+512351us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:31.242906 (+ 99577us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:31.331590 (+ 88684us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":14395,"tablet-delete.queue_time_us":891396}
I20260925 09:57:31.333709 11709 tablet_replica.cc:330] T a44d9f2f7e3e48aeb0f31fa29ae1988a P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:31.336285 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 235) took 5733 ms. Trace:
I20260925 09:57:31.337352 11727 rpcz_store.cc:276] 0925 09:57:25.602907 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:25.602995 (+ 88us) service_pool.cc:227] Handling call
0925 09:57:31.335986 (+5732991us) ts_tablet_manager.cc:1145] Deleting tablet cf35f39bcd6a41128acb2a4936fcc316
0925 09:57:31.336040 (+ 54us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:31.336219 (+ 179us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5731790}
I20260925 09:57:31.340117 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 236) took 5736 ms. Trace:
I20260925 09:57:31.346289 11085 ts_tablet_manager.cc:1428] T 417058493c6547c69e72347cef373292 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.174s user 0.002s sys 0.002s
I20260925 09:57:31.344206 12027 raft_consensus.cc:2229] T aba253f22afa4200926f31a1b43e3970 P c43461c4ee854ae1ac52b873ccada318 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:31.348636 12027 raft_consensus.cc:2258] T aba253f22afa4200926f31a1b43e3970 P c43461c4ee854ae1ac52b873ccada318 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:31.350677 12027 ts_tablet_manager.cc:1905] T aba253f22afa4200926f31a1b43e3970 P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:31.350637 12133 raft_consensus.cc:686] T 5246648245e04e26be2abf1dbf706688 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
W20260925 09:57:31.320240 10378 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet b3d9a3cf5bda4cf6bc1fa3fae1cb9288 (table default.Table_cc9f467e078c448f87392bf324b040a8 [id=4cefdaddfd6143468df4502cd35afac4], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet b3d9a3cf5bda4cf6bc1fa3fae1cb9288 consensus state has no leader
I20260925 09:57:31.333504 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 60fdc299865446c89d3e5374d573029a (table default.Table_32e7bdf7eaf147ea90740b6b3cfb2fd3 [id=be059d72d5db4915ba767959c6a1c7b2]) successfully deleted
I20260925 09:57:31.335609 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 3be0eccc65564bfba3d6e5c398c61fb6 (table default.Table_0af2360d761342cf9f057be5f265aa41 [id=99ba81c4d0dd497398e048a05759419a]) successfully deleted
W20260925 09:57:31.352962 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet cf35f39bcd6a41128acb2a4936fcc316 because the tablet was not found. No further retry: Not found: Tablet not found: cf35f39bcd6a41128acb2a4936fcc316
I20260925 09:57:31.343971 11709 raft_consensus.cc:2229] T a44d9f2f7e3e48aeb0f31fa29ae1988a P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:31.354038 11709 raft_consensus.cc:2258] T a44d9f2f7e3e48aeb0f31fa29ae1988a P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:31.356573 11709 ts_tablet_manager.cc:1905] T a44d9f2f7e3e48aeb0f31fa29ae1988a P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:31.361982 12132 raft_consensus.cc:482] T 417058493c6547c69e72347cef373292 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:31.362491 12132 raft_consensus.cc:504] T 417058493c6547c69e72347cef373292 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:31.354745 11082 tablet_bootstrap.cc:654] T d5956be0da6441c79723e72f12dfe54a P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:31.365100 12132 leader_election.cc:288] T 417058493c6547c69e72347cef373292 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:31.365392 10673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "417058493c6547c69e72347cef373292" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:31.366117 10673 raft_consensus.cc:2454] T 417058493c6547c69e72347cef373292 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
I20260925 09:57:31.367286 10385 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet ddaa47d7287b4f1b94f999f81e1583f2
I20260925 09:57:31.360116 12133 consensus_queue.cc:237] T 5246648245e04e26be2abf1dbf706688 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:31.368462 10385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_50ec368977774d53a7d8643459e4aa10"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:31.370205 10385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_50ec368977774d53a7d8643459e4aa10 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:31.367069 10775 leader_election.cc:302] T 417058493c6547c69e72347cef373292 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:31.375255 12123 raft_consensus.cc:2781] T 417058493c6547c69e72347cef373292 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:31.375705 12123 raft_consensus.cc:482] T 417058493c6547c69e72347cef373292 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:31.376068 12123 raft_consensus.cc:3037] T 417058493c6547c69e72347cef373292 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:31.379189 12080 leader_election.cc:288] T e684798155d445bfb325a1b7c5c48753 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:31.381369 12146 tablet.cc:1722] T 42ec275c905343c4be1b72dc2bd0f6e3 P c43461c4ee854ae1ac52b873ccada318: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 09:57:31.384567 12123 raft_consensus.cc:504] T 417058493c6547c69e72347cef373292 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:31.388904 12147 consensus_queue.cc:1035] T c7cd1d6a603642afa2c8c737079961b5 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:31.390111 12151 tablet.cc:1722] T c588b0c05a1a4f3ea545066183e76502 P b8850818e2ee42a082802d0095c08d21: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:31.391122 11085 tablet_bootstrap.cc:492] T 5608c5f9b65b4da7b2fbb56417c92d88 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:31.391649 11727 rpcz_store.cc:276] 0925 09:57:25.603991 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:25.604074 (+ 83us) service_pool.cc:227] Handling call
0925 09:57:31.339845 (+5735771us) ts_tablet_manager.cc:1145] Deleting tablet ec1b78e61e8d46cb8b87f60786059658
0925 09:57:31.339892 (+ 47us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:31.340065 (+ 173us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5735153}
I20260925 09:57:31.342087 11084 tablet_bootstrap.cc:654] T 9744b9814b69470396ce039348fe5e45 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:31.392380 12152 tablet.cc:1722] T 42ec275c905343c4be1b72dc2bd0f6e3 P b8850818e2ee42a082802d0095c08d21: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 09:57:31.395457 12080 raft_consensus.cc:482] T 69a96874eea1409a9a5a06b5787fbd99 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:31.395941 12080 raft_consensus.cc:504] T 69a96874eea1409a9a5a06b5787fbd99 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:31.398947 12080 leader_election.cc:288] T 69a96874eea1409a9a5a06b5787fbd99 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:31.353004 10918 leader_election.cc:302] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21
I20260925 09:57:31.366389 10981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "417058493c6547c69e72347cef373292" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:31.399791 10981 raft_consensus.cc:2454] T 417058493c6547c69e72347cef373292 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
I20260925 09:57:31.378553 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e684798155d445bfb325a1b7c5c48753" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:31.401413 10976 raft_consensus.cc:3037] T e684798155d445bfb325a1b7c5c48753 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:31.414728 10976 raft_consensus.cc:2454] T e684798155d445bfb325a1b7c5c48753 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:31.399168 10982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69a96874eea1409a9a5a06b5787fbd99" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:31.416271 10982 raft_consensus.cc:2454] T 69a96874eea1409a9a5a06b5787fbd99 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
W20260925 09:57:31.329530 10919 leader_election.cc:341] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Not found: Tablet not found: aba253f22afa4200926f31a1b43e3970
I20260925 09:57:31.416824 10619 leader_election.cc:302] T e684798155d445bfb325a1b7c5c48753 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
W20260925 09:57:31.417200 10919 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 87699.6us
W20260925 09:57:31.417873 10919 leader_election.cc:341] T ab6bdba5ca5d41ae921a90c1f77006c5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:31.418078 10619 leader_election.cc:302] T 69a96874eea1409a9a5a06b5787fbd99 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:31.418283 10919 leader_election.cc:302] T ab6bdba5ca5d41ae921a90c1f77006c5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:31.418815 12147 raft_consensus.cc:2781] T e684798155d445bfb325a1b7c5c48753 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:31.418967 12041 raft_consensus.cc:2726] T ab6bdba5ca5d41ae921a90c1f77006c5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:31.419283 12147 raft_consensus.cc:686] T e684798155d445bfb325a1b7c5c48753 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:31.419409 12134 raft_consensus.cc:2781] T 69a96874eea1409a9a5a06b5787fbd99 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:31.419762 12134 raft_consensus.cc:482] T 69a96874eea1409a9a5a06b5787fbd99 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:31.420049 12134 raft_consensus.cc:3037] T 69a96874eea1409a9a5a06b5787fbd99 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:31.420089 12147 consensus_queue.cc:237] T e684798155d445bfb325a1b7c5c48753 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:31.421999 12123 leader_election.cc:288] T 417058493c6547c69e72347cef373292 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:31.428066 12041 raft_consensus.cc:2781] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 09:57:31.428851 12041 raft_consensus.cc:686] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [term 2 LEADER]: Becoming Leader. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Running, Role: LEADER
I20260925 09:57:31.430536 12134 raft_consensus.cc:504] T 69a96874eea1409a9a5a06b5787fbd99 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:31.432461 12134 leader_election.cc:288] T 69a96874eea1409a9a5a06b5787fbd99 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:31.422173 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "417058493c6547c69e72347cef373292" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:31.433709 11082 tablet_bootstrap.cc:492] T d5956be0da6441c79723e72f12dfe54a P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
W20260925 09:57:31.433670 10918 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 80761.8us
I20260925 09:57:31.434091 11082 ts_tablet_manager.cc:1397] T d5956be0da6441c79723e72f12dfe54a P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.109s user 0.009s sys 0.000s
I20260925 09:57:31.434259 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "417058493c6547c69e72347cef373292" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:31.436650 10687 raft_consensus.cc:3037] T 417058493c6547c69e72347cef373292 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:31.436236 11082 raft_consensus.cc:348] T d5956be0da6441c79723e72f12dfe54a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:31.443104 11082 raft_consensus.cc:374] T d5956be0da6441c79723e72f12dfe54a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:31.443403 11082 raft_consensus.cc:729] T d5956be0da6441c79723e72f12dfe54a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:31.444067 11082 consensus_queue.cc:260] T d5956be0da6441c79723e72f12dfe54a P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:31.446197 11082 ts_tablet_manager.cc:1428] T d5956be0da6441c79723e72f12dfe54a P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.012s user 0.005s sys 0.000s
I20260925 09:57:31.447012 11082 tablet_bootstrap.cc:492] T 7a31b999f0fd447eacff13de71aa3fff P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:31.448287 10982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69a96874eea1409a9a5a06b5787fbd99" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:31.448807 10982 raft_consensus.cc:3037] T 69a96874eea1409a9a5a06b5787fbd99 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:31.452596 12080 raft_consensus.cc:482] T 5608c5f9b65b4da7b2fbb56417c92d88 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:31.453249 12080 raft_consensus.cc:504] T 5608c5f9b65b4da7b2fbb56417c92d88 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:31.455530 10982 raft_consensus.cc:2454] T 69a96874eea1409a9a5a06b5787fbd99 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:31.457140 10619 leader_election.cc:302] T 69a96874eea1409a9a5a06b5787fbd99 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:31.457803 12134 raft_consensus.cc:2781] T 69a96874eea1409a9a5a06b5787fbd99 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:31.458547 10976 raft_consensus.cc:3037] T 417058493c6547c69e72347cef373292 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:31.458717 12134 raft_consensus.cc:686] T 69a96874eea1409a9a5a06b5787fbd99 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:31.459995 10384 master_service.cc:946] ReplaceTablet: received request to replace tablet dba14a44a9de492886d629441606767e from {username='slave'} at 127.0.0.1:35140
W20260925 09:57:31.464021 10919 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 46174.9us
I20260925 09:57:31.463562 12134 consensus_queue.cc:237] T 69a96874eea1409a9a5a06b5787fbd99 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:31.464805 10919 leader_election.cc:341] T a436fe287a0d48a1821191adcd031d74 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:31.465179 10919 leader_election.cc:302] T a436fe287a0d48a1821191adcd031d74 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:31.465875 12039 raft_consensus.cc:2726] T a436fe287a0d48a1821191adcd031d74 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:31.466964 10919 leader_election.cc:341] T defaef4d6b754ca5a2dfc21b7a596a5b P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:31.467352 10919 leader_election.cc:302] T defaef4d6b754ca5a2dfc21b7a596a5b P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:31.468098 12039 raft_consensus.cc:2726] T defaef4d6b754ca5a2dfc21b7a596a5b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:31.457367 12080 leader_election.cc:288] T 5608c5f9b65b4da7b2fbb56417c92d88 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
W20260925 09:57:31.470036 10919 leader_election.cc:341] T 10657a81058241fc988d37e5ea7ff8cf P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:31.470425 10919 leader_election.cc:302] T 10657a81058241fc988d37e5ea7ff8cf P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
W20260925 09:57:31.473665 10393 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e684798155d445bfb325a1b7c5c48753 consensus state has no leader
W20260925 09:57:31.475855 10391 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 25ac35fb43d34c9b85b8e61ad6e386a6 consensus state has no leader
I20260925 09:57:31.489120 10687 raft_consensus.cc:2454] T 417058493c6547c69e72347cef373292 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:31.490706 11082 tablet_bootstrap.cc:654] T 7a31b999f0fd447eacff13de71aa3fff P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:31.493327 10775 leader_election.cc:302] T 417058493c6547c69e72347cef373292 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:31.494065 12067 raft_consensus.cc:2781] T 417058493c6547c69e72347cef373292 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:31.494570 12067 raft_consensus.cc:686] T 417058493c6547c69e72347cef373292 P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Becoming Leader. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Running, Role: LEADER
I20260925 09:57:31.474114 10982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5608c5f9b65b4da7b2fbb56417c92d88" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:31.495421 10982 raft_consensus.cc:2454] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:31.429898 12041 consensus_queue.cc:237] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:31.496773 10619 leader_election.cc:302] T 5608c5f9b65b4da7b2fbb56417c92d88 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:31.497382 12095 raft_consensus.cc:2726] T 10657a81058241fc988d37e5ea7ff8cf P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:31.498339 11709 ts_tablet_manager.cc:1918] T a44d9f2f7e3e48aeb0f31fa29ae1988a P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:31.495461 12067 consensus_queue.cc:237] T 417058493c6547c69e72347cef373292 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:31.500347 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet ec1b78e61e8d46cb8b87f60786059658 because the tablet was not found. No further retry: Not found: Tablet not found: ec1b78e61e8d46cb8b87f60786059658
I20260925 09:57:31.504029 11709 log.cc:1196] T a44d9f2f7e3e48aeb0f31fa29ae1988a P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/a44d9f2f7e3e48aeb0f31fa29ae1988a
I20260925 09:57:31.512385 11709 ts_tablet_manager.cc:1939] T a44d9f2f7e3e48aeb0f31fa29ae1988a P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:31.502625 10621 leader_election.cc:302] T 6161774f61ae48ab9764a05f2188a80b P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters: be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:31.498654 12134 raft_consensus.cc:2781] T 5608c5f9b65b4da7b2fbb56417c92d88 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:31.515724 12134 raft_consensus.cc:482] T 5608c5f9b65b4da7b2fbb56417c92d88 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:31.516129 12134 raft_consensus.cc:3037] T 5608c5f9b65b4da7b2fbb56417c92d88 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:31.516537 12080 raft_consensus.cc:2781] T 6161774f61ae48ab9764a05f2188a80b P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:31.518606 12080 raft_consensus.cc:482] T 6161774f61ae48ab9764a05f2188a80b P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:31.519018 12080 raft_consensus.cc:3037] T 6161774f61ae48ab9764a05f2188a80b P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:31.515158 11709 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 298) took 1820 ms. Trace:
I20260925 09:57:31.528154 11709 rpcz_store.cc:276] 0925 09:57:29.694667 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:29.707857 (+ 13190us) service_pool.cc:227] Handling call
0925 09:57:31.333571 (+1625714us) ts_tablet_manager.cc:1145] Deleting tablet a44d9f2f7e3e48aeb0f31fa29ae1988a
0925 09:57:31.333610 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:31.448148 (+114538us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:31.498110 (+ 49962us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:31.515081 (+ 16971us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1586863}
I20260925 09:57:31.530577 12080 raft_consensus.cc:504] T 6161774f61ae48ab9764a05f2188a80b P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:31.531239 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet a44d9f2f7e3e48aeb0f31fa29ae1988a (table default.Table_eb065d5979c64931abf91d82b8623a0a [id=5b74f0855d2046e3a9463f732ae19e62]) successfully deleted
I20260925 09:57:31.532682 11709 tablet_replica.cc:330] T 3ff4cf556eee494d8cd590884466f09f P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:31.534600 11709 raft_consensus.cc:2229] T 3ff4cf556eee494d8cd590884466f09f P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:31.536777 10805 tablet_service.cc:1514] Processing DeleteTablet for tablet a58cf1dc81454710a9ad1a8baf155c9f with delete_type TABLET_DATA_DELETED (replaced by tablet ddaa47d7287b4f1b94f999f81e1583f2) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:31.537310 12134 raft_consensus.cc:504] T 5608c5f9b65b4da7b2fbb56417c92d88 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:31.537024 11709 raft_consensus.cc:2258] T 3ff4cf556eee494d8cd590884466f09f P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:31.540715 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 237) took 5935 ms. Trace:
W20260925 09:57:31.541790 10775 consensus_peers.cc:487] T a44d9f2f7e3e48aeb0f31fa29ae1988a P c43461c4ee854ae1ac52b873ccada318 -> Peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Couldn't send request to peer b8850818e2ee42a082802d0095c08d21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a44d9f2f7e3e48aeb0f31fa29ae1988a. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:31.541018 11727 rpcz_store.cc:276] 0925 09:57:25.605007 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:25.605089 (+ 82us) service_pool.cc:227] Handling call
0925 09:57:31.500505 (+5895416us) ts_tablet_manager.cc:1145] Deleting tablet 89a143a5623e4ef38027381a56c95227
0925 09:57:31.500546 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:31.540650 (+ 40104us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5894744}
I20260925 09:57:31.543574 11727 tablet_replica.cc:330] T 4e3120347c8e4a3eab57d6562ae40e8d P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
W20260925 09:57:31.543731 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 89a143a5623e4ef38027381a56c95227 because the tablet was not found. No further retry: Not found: Tablet not found: 89a143a5623e4ef38027381a56c95227
I20260925 09:57:31.544545 11709 ts_tablet_manager.cc:1905] T 3ff4cf556eee494d8cd590884466f09f P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:31.548538 10973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5608c5f9b65b4da7b2fbb56417c92d88" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:31.549108 10973 raft_consensus.cc:3037] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:31.551131 11435 fs_manager.cc:709] Time spent opening directory manager: real 0.243s user 0.006s sys 0.001s
I20260925 09:57:31.560370 10389 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef
I20260925 09:57:31.562469 10946 tablet_service.cc:1514] Processing DeleteTablet for tablet 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:30 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:31.565500 10976 raft_consensus.cc:2454] T 417058493c6547c69e72347cef373292 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:31.565842 10389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_7f649eb0207a4f06a9527d2e6dc3a7d3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:31.567529 10389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7f649eb0207a4f06a9527d2e6dc3a7d3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:31.570232 11084 tablet_bootstrap.cc:492] T 9744b9814b69470396ce039348fe5e45 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:31.570628 11084 ts_tablet_manager.cc:1397] T 9744b9814b69470396ce039348fe5e45 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.274s user 0.010s sys 0.001s
W20260925 09:57:31.571435 10919 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 70156.1us
I20260925 09:57:31.576242 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 60fdc299865446c89d3e5374d573029a (table default.Table_32e7bdf7eaf147ea90740b6b3cfb2fd3 [id=be059d72d5db4915ba767959c6a1c7b2]) successfully deleted
I20260925 09:57:31.562372 10653 tablet_service.cc:1514] Processing DeleteTablet for tablet 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:30 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:31.583808 12134 leader_election.cc:288] T 5608c5f9b65b4da7b2fbb56417c92d88 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:31.585062 11082 tablet_bootstrap.cc:492] T 7a31b999f0fd447eacff13de71aa3fff P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:31.585763 10973 raft_consensus.cc:2454] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:31.587664 10385 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_44ff944ba4b84cb7a677f7dd03f8f7dd" } modify_external_catalogs: true
I20260925 09:57:31.588194 10385 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_44ff944ba4b84cb7a677f7dd03f8f7dd" } modify_external_catalogs: true
I20260925 09:57:31.588478 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e684798155d445bfb325a1b7c5c48753" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:31.586982 10619 leader_election.cc:302] T 5608c5f9b65b4da7b2fbb56417c92d88 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:31.591667 11082 ts_tablet_manager.cc:1397] T 7a31b999f0fd447eacff13de71aa3fff P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.145s user 0.012s sys 0.005s
I20260925 09:57:31.592011 12133 raft_consensus.cc:2781] T 5608c5f9b65b4da7b2fbb56417c92d88 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:31.594313 11082 raft_consensus.cc:348] T 7a31b999f0fd447eacff13de71aa3fff P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:31.594918 11082 raft_consensus.cc:374] T 7a31b999f0fd447eacff13de71aa3fff P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:31.595145 11082 raft_consensus.cc:729] T 7a31b999f0fd447eacff13de71aa3fff P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:31.600113 11727 raft_consensus.cc:2229] T 4e3120347c8e4a3eab57d6562ae40e8d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:31.608525 11727 raft_consensus.cc:2258] T 4e3120347c8e4a3eab57d6562ae40e8d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:31.611037 11727 ts_tablet_manager.cc:1905] T 4e3120347c8e4a3eab57d6562ae40e8d P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:31.612169 10825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69a96874eea1409a9a5a06b5787fbd99" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:31.592463 12133 raft_consensus.cc:686] T 5608c5f9b65b4da7b2fbb56417c92d88 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:31.618911 12133 consensus_queue.cc:237] T 5608c5f9b65b4da7b2fbb56417c92d88 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:31.622632 10388 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 0a018bea4200474a9904f62aee37240b
I20260925 09:57:31.622820 11082 consensus_queue.cc:260] T 7a31b999f0fd447eacff13de71aa3fff P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:31.628517 10653 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a018bea4200474a9904f62aee37240b with delete_type TABLET_DATA_DELETED (replaced by tablet adc84ee8b9354322af4f67a4d4f6ccce) from {username='slave'} at 127.0.0.1:37876
W20260925 09:57:31.630853 10387 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.2:40761 (request call id 38) took 965 ms (client timeout 1000 ms). Trace:
W20260925 09:57:31.631233 10387 rpcz_store.cc:269] 0925 09:57:30.665086 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:30.721228 (+ 56142us) service_pool.cc:227] Handling call
0925 09:57:30.749450 (+ 28222us) spinlock_profiling.cc:243] Waited 22.4 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2ad9c3e 00007fabf0610fb4 00007fabf0610f42 00007fabf0610e71 00007fabf0610dec 00007fabf06100a2 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:31.630795 (+881345us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":4,"spinlock_wait_cycles":51450112}
I20260925 09:57:31.633030 11085 tablet_bootstrap.cc:654] T 5608c5f9b65b4da7b2fbb56417c92d88 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:31.609858 10621 leader_election.cc:341] T e684798155d445bfb325a1b7c5c48753 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:31.637809 11082 ts_tablet_manager.cc:1428] T 7a31b999f0fd447eacff13de71aa3fff P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.046s user 0.003s sys 0.003s
I20260925 09:57:31.638650 11082 tablet_bootstrap.cc:492] T 545ac025f2cb469da847979011a0b3ce P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:31.641090 10946 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a018bea4200474a9904f62aee37240b with delete_type TABLET_DATA_DELETED (replaced by tablet adc84ee8b9354322af4f67a4d4f6ccce) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:31.641657 10388 catalog_manager.cc:6565] ReplaceTablet: tablet 0a018bea4200474a9904f62aee37240b deleted and replaced by tablet adc84ee8b9354322af4f67a4d4f6ccce
I20260925 09:57:31.644094 11084 raft_consensus.cc:348] T 9744b9814b69470396ce039348fe5e45 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:31.644254 10653 tablet_service.cc:1467] Processing CreateTablet for tablet 44ec73b45f5443338dc3f36a6e9fcc98 (DEFAULT_TABLE table=default.Table_dc0b8cf3c1fb457985e9d19f5ef2ceb7 [id=f2ae5d5b4acf4ea1ab534e9b122bc211]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:31.610790 12080 leader_election.cc:288] T 6161774f61ae48ab9764a05f2188a80b P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:31.647979 10830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5608c5f9b65b4da7b2fbb56417c92d88" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:31.649220 10833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69a96874eea1409a9a5a06b5787fbd99" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
W20260925 09:57:31.651278 10619 leader_election.cc:341] T 6161774f61ae48ab9764a05f2188a80b P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 6161774f61ae48ab9764a05f2188a80b
I20260925 09:57:31.652511 11084 raft_consensus.cc:374] T 9744b9814b69470396ce039348fe5e45 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:31.653890 11084 raft_consensus.cc:729] T 9744b9814b69470396ce039348fe5e45 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:31.651783 10653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44ec73b45f5443338dc3f36a6e9fcc98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:31.671058 11084 consensus_queue.cc:260] T 9744b9814b69470396ce039348fe5e45 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:31.678493 11082 tablet_bootstrap.cc:654] T 545ac025f2cb469da847979011a0b3ce P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:31.678717 10391 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_5062a73a7614459a8690df32165076dc" } new_table_name: "default.Table_bcbc82519cf043738f61029181f29418" modify_external_catalogs: true
I20260925 09:57:31.677904 10387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_5767a6fb7263485fa31677b488d9f903"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:31.674983 11084 ts_tablet_manager.cc:1428] T 9744b9814b69470396ce039348fe5e45 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.104s user 0.005s sys 0.000s
W20260925 09:57:31.683737 10387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5767a6fb7263485fa31677b488d9f903 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:31.684223 10389 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_7e0155a70cc54485b8fdd4d936460d1c" } new_table_name: "default.Table_2a6203820e0d44aa993080049923de10" modify_external_catalogs: true
I20260925 09:57:31.685894 11084 tablet_bootstrap.cc:492] T 69028aa7908c45b896da4ba8deef07b1 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:31.689791 10393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_e378bf3dbec04ead88c254701fea06fc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:31.691244 10393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e378bf3dbec04ead88c254701fea06fc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:31.691233 10833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5608c5f9b65b4da7b2fbb56417c92d88" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:31.696230 10833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6161774f61ae48ab9764a05f2188a80b" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:31.696386 11709 ts_tablet_manager.cc:1918] T 3ff4cf556eee494d8cd590884466f09f P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:31.696786 10833 raft_consensus.cc:3037] T 6161774f61ae48ab9764a05f2188a80b P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:31.696929 11709 log.cc:1196] T 3ff4cf556eee494d8cd590884466f09f P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/3ff4cf556eee494d8cd590884466f09f
I20260925 09:57:31.695288 10388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_7678491b0b014f7fa63359e06ffbcd3e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:31.700838 10946 tablet_service.cc:1467] Processing CreateTablet for tablet 44ec73b45f5443338dc3f36a6e9fcc98 (DEFAULT_TABLE table=default.Table_dc0b8cf3c1fb457985e9d19f5ef2ceb7 [id=f2ae5d5b4acf4ea1ab534e9b122bc211]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:31.702075 10946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44ec73b45f5443338dc3f36a6e9fcc98. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:31.705489 10388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7678491b0b014f7fa63359e06ffbcd3e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:31.715233 12168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:57:31.716385 12146 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:33344 (request call id 308) took 1156 ms. Trace:
I20260925 09:57:31.717125 10947 tablet_service.cc:1467] Processing CreateTablet for tablet 0ec6a95911ef44f09e08dfc531966a83 (DEFAULT_TABLE table=default.Table_3275db3804df4b12ad6e64e4d88bc75b [id=2f35a7ab50dd44cab6398a77c110e2fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:31.718189 10947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ec6a95911ef44f09e08dfc531966a83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:31.705863 12134 consensus_queue.cc:1035] T fe91b39f91274a7ca61e9618d8cbb475 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:31.720883 10655 tablet_service.cc:1467] Processing CreateTablet for tablet 3ae55832f05743bebaae72ce862191c9 (DEFAULT_TABLE table=default.Table_55e6a408ce794b5fbd2fe39d51e6a2aa [id=a19ac637e43146979e3c299d4e46b180]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:31.707525 10653 tablet_service.cc:1467] Processing CreateTablet for tablet 0ec6a95911ef44f09e08dfc531966a83 (DEFAULT_TABLE table=default.Table_3275db3804df4b12ad6e64e4d88bc75b [id=2f35a7ab50dd44cab6398a77c110e2fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:31.721856 10655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ae55832f05743bebaae72ce862191c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:31.722644 11709 ts_tablet_manager.cc:1939] T 3ff4cf556eee494d8cd590884466f09f P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:31.723673 10653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ec6a95911ef44f09e08dfc531966a83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:31.726542 10667 tablet_service.cc:1467] Processing CreateTablet for tablet 635764b638304b64a523ecd89ee02f80 (DEFAULT_TABLE table=default.Table_2a228b9f88074bb897dd2b8e4624913b [id=2eaeb50bb0744b4db2baad734bf98b29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:31.727432 10667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 635764b638304b64a523ecd89ee02f80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:31.733153 10952 tablet_service.cc:1467] Processing CreateTablet for tablet 3ae55832f05743bebaae72ce862191c9 (DEFAULT_TABLE table=default.Table_55e6a408ce794b5fbd2fe39d51e6a2aa [id=a19ac637e43146979e3c299d4e46b180]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:31.734288 10952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ae55832f05743bebaae72ce862191c9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:31.736089 10621 leader_election.cc:341] T e684798155d445bfb325a1b7c5c48753 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:31.737393 10621 leader_election.cc:341] T 69a96874eea1409a9a5a06b5787fbd99 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:31.737696 10944 tablet_service.cc:1467] Processing CreateTablet for tablet 635764b638304b64a523ecd89ee02f80 (DEFAULT_TABLE table=default.Table_2a228b9f88074bb897dd2b8e4624913b [id=2eaeb50bb0744b4db2baad734bf98b29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:31.738758 10944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 635764b638304b64a523ecd89ee02f80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:31.740557 10668 tablet_service.cc:1467] Processing CreateTablet for tablet 8f3112845dc744d083c0f5e442f0aef7 (DEFAULT_TABLE table=default.Table_00cfe005d2584fc19576d064db0f5fd1 [id=c03fd560c94d49a98899fd1cf5e60717]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:31.741480 10668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f3112845dc744d083c0f5e442f0aef7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:31.744433 11709 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 302) took 1869 ms. Trace:
I20260925 09:57:31.744550 10950 tablet_service.cc:1467] Processing CreateTablet for tablet 8f3112845dc744d083c0f5e442f0aef7 (DEFAULT_TABLE table=default.Table_00cfe005d2584fc19576d064db0f5fd1 [id=c03fd560c94d49a98899fd1cf5e60717]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:31.744740 11709 rpcz_store.cc:276] 0925 09:57:29.874638 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:29.874765 (+ 127us) service_pool.cc:227] Handling call
0925 09:57:31.532535 (+1657770us) ts_tablet_manager.cc:1145] Deleting tablet 3ff4cf556eee494d8cd590884466f09f
0925 09:57:31.532576 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:31.652224 (+119648us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:31.696176 (+ 43952us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:31.744363 (+ 48187us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1657196}
I20260925 09:57:31.745548 10950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f3112845dc744d083c0f5e442f0aef7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:31.745699 11709 tablet_replica.cc:330] T c7cd1d6a603642afa2c8c737079961b5 P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:31.746376 11709 raft_consensus.cc:2229] T c7cd1d6a603642afa2c8c737079961b5 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:31.747056 11709 raft_consensus.cc:2258] T c7cd1d6a603642afa2c8c737079961b5 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:31.747820 10805 tablet_service.cc:1514] Processing DeleteTablet for tablet 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:30 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:31.748780 10654 tablet_service.cc:1467] Processing CreateTablet for tablet 77ebb59faf3e4fb187bbce29612c25ea (DEFAULT_TABLE table=default.Table_32e7bdf7eaf147ea90740b6b3cfb2fd3 [id=be059d72d5db4915ba767959c6a1c7b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:31.749712 11727 ts_tablet_manager.cc:1918] T 4e3120347c8e4a3eab57d6562ae40e8d P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:31.749804 10654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77ebb59faf3e4fb187bbce29612c25ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:31.750042 11727 log.cc:1196] T 4e3120347c8e4a3eab57d6562ae40e8d P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/4e3120347c8e4a3eab57d6562ae40e8d
I20260925 09:57:31.750648 11727 ts_tablet_manager.cc:1939] T 4e3120347c8e4a3eab57d6562ae40e8d P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
W20260925 09:57:31.750584 10621 leader_election.cc:341] T 5608c5f9b65b4da7b2fbb56417c92d88 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:31.751454 10621 leader_election.cc:341] T 69a96874eea1409a9a5a06b5787fbd99 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:31.751703 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 3ff4cf556eee494d8cd590884466f09f (table default.Table_66b8a45b14bf441d80ab32ee756de608 [id=2aeb0fb9ff8c416aba7b24d74354fcb0]) successfully deleted
I20260925 09:57:31.753121 11082 tablet_bootstrap.cc:492] T 545ac025f2cb469da847979011a0b3ce P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:31.753352 10960 tablet_service.cc:1467] Processing CreateTablet for tablet 77ebb59faf3e4fb187bbce29612c25ea (DEFAULT_TABLE table=default.Table_32e7bdf7eaf147ea90740b6b3cfb2fd3 [id=be059d72d5db4915ba767959c6a1c7b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:31.753585 11082 ts_tablet_manager.cc:1397] T 545ac025f2cb469da847979011a0b3ce P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.115s user 0.005s sys 0.005s
I20260925 09:57:31.754297 10960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77ebb59faf3e4fb187bbce29612c25ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:31.757476 11435 fs_manager.cc:725] Time spent opening block manager: real 0.173s user 0.005s sys 0.000s
I20260925 09:57:31.758523 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 238) took 6029 ms. Trace:
I20260925 09:57:31.758882 11727 rpcz_store.cc:276] 0925 09:57:25.728892 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:25.741376 (+ 12484us) service_pool.cc:227] Handling call
0925 09:57:31.543368 (+5801992us) ts_tablet_manager.cc:1145] Deleting tablet 4e3120347c8e4a3eab57d6562ae40e8d
0925 09:57:31.543438 (+ 70us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:31.650495 (+107057us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:31.749482 (+ 98987us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:31.758469 (+ 8987us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5801335}
I20260925 09:57:31.765676 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 4e3120347c8e4a3eab57d6562ae40e8d (table default.Table_8e186a99ccba425b9b588041d2364a40 [id=2d6bc0e2aaee4b1483b01fc678505bb0]) successfully deleted
I20260925 09:57:31.765954 11727 tablet_replica.cc:330] T 10657a81058241fc988d37e5ea7ff8cf P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:31.766718 11727 raft_consensus.cc:2229] T 10657a81058241fc988d37e5ea7ff8cf P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:31.767136 11727 raft_consensus.cc:2258] T 10657a81058241fc988d37e5ea7ff8cf P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:31.768509 11085 tablet_bootstrap.cc:492] T 5608c5f9b65b4da7b2fbb56417c92d88 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:31.768771 11727 ts_tablet_manager.cc:1905] T 10657a81058241fc988d37e5ea7ff8cf P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:31.768899 11085 ts_tablet_manager.cc:1397] T 5608c5f9b65b4da7b2fbb56417c92d88 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.378s user 0.007s sys 0.003s
I20260925 09:57:31.769418 10386 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_8f6c92e47621419db4bc9e3e2dd5ce12" } modify_external_catalogs: true
I20260925 09:57:31.769948 10386 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_8f6c92e47621419db4bc9e3e2dd5ce12" } modify_external_catalogs: true
I20260925 09:57:31.776595 10833 raft_consensus.cc:2454] T 6161774f61ae48ab9764a05f2188a80b P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
W20260925 09:57:31.783192 10621 leader_election.cc:341] T 5608c5f9b65b4da7b2fbb56417c92d88 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:31.794094 11727 ts_tablet_manager.cc:1918] T 10657a81058241fc988d37e5ea7ff8cf P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:31.794513 11727 log.cc:1196] T 10657a81058241fc988d37e5ea7ff8cf P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/10657a81058241fc988d37e5ea7ff8cf
I20260925 09:57:31.800477 11727 ts_tablet_manager.cc:1939] T 10657a81058241fc988d37e5ea7ff8cf P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:31.716990 12146 rpcz_store.cc:276] 0925 09:57:30.559600 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:30.559705 (+ 105us) service_pool.cc:227] Handling call
0925 09:57:31.716326 (+1156621us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:30.571582 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0925 09:57:30.571894 (+ 312us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0925 09:57:30.571933 (+ 39us) alter_schema_op.cc:65] Acquired schema lock
0925 09:57:30.571970 (+ 37us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0925 09:57:30.572076 (+ 106us) alter_schema_op.cc:126] START. Timestamp: 1790330250569904
0925 09:57:30.572100 (+ 24us) op_driver.cc:331] REPLICATION: starting
0925 09:57:30.572575 (+ 475us) log.cc:842] Serialized 173 byte log entry
0925 09:57:31.271021 (+698446us) op_driver.cc:444] REPLICATION: finished
0925 09:57:31.381120 (+110099us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0925 09:57:31.505244 (+124124us) log.cc:842] Serialized 5 byte log entry
0925 09:57:31.618234 (+112990us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:31.619004 (+ 770us) log.cc:842] Serialized 14 byte log entry
0925 09:57:31.716176 (+ 97172us) alter_schema_op.cc:71] Released schema lock
0925 09:57:31.716220 (+ 44us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":109933,"prepare.queue_time_us":182,"prepare.run_cpu_time_us":45839,"prepare.run_wall_time_us":670642,"raft.queue_time_us":38629,"raft.run_cpu_time_us":1582,"raft.run_wall_time_us":1581,"replication_time_us":698832,"thread_start_us":725975,"threads_started":3,"wal-append.queue_time_us":356250}]]}
I20260925 09:57:31.777395 10812 tablet_service.cc:1514] Processing DeleteTablet for tablet 977a59175e5b43eeb6f23b938afd00a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:18 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:31.778270 10805 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a018bea4200474a9904f62aee37240b with delete_type TABLET_DATA_DELETED (replaced by tablet adc84ee8b9354322af4f67a4d4f6ccce) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:31.779522 10806 tablet_service.cc:1467] Processing CreateTablet for tablet 44ec73b45f5443338dc3f36a6e9fcc98 (DEFAULT_TABLE table=default.Table_dc0b8cf3c1fb457985e9d19f5ef2ceb7 [id=f2ae5d5b4acf4ea1ab534e9b122bc211]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:31.803959 10806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44ec73b45f5443338dc3f36a6e9fcc98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:31.805044 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 239) took 6062 ms. Trace:
I20260925 09:57:31.805330 11727 rpcz_store.cc:276] 0925 09:57:25.743001 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:25.761690 (+ 18689us) service_pool.cc:227] Handling call
0925 09:57:31.765802 (+6004112us) ts_tablet_manager.cc:1145] Deleting tablet 10657a81058241fc988d37e5ea7ff8cf
0925 09:57:31.765846 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:31.787455 (+ 21609us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:31.793897 (+ 6442us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:31.804990 (+ 11093us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5984744}
I20260925 09:57:31.806311 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 10657a81058241fc988d37e5ea7ff8cf (table default.Table_64ba123898174e72a6884dbb9ad07de3 [id=7ee526720b2442119e0fc487b7c4fcd4]) successfully deleted
I20260925 09:57:31.757903 11435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-2/data,/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-2/wal
uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2"
format_stamp: "Formatted at 2026-09-25 09:56:56 on dist-test-slave-4xvl"
I20260925 09:57:31.816334 11435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:57:31.822072 12027 ts_tablet_manager.cc:1918] T aba253f22afa4200926f31a1b43e3970 P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:31.826000 10384 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet dba14a44a9de492886d629441606767e
I20260925 09:57:31.827239 10384 catalog_manager.cc:6565] ReplaceTablet: tablet dba14a44a9de492886d629441606767e deleted and replaced by tablet 080c76ce032848c4af26ec5c8566f300
I20260925 09:57:31.828217 10384 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_bb7c3c53b9c44306a98ae97e22140782"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:31.829612 10384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bb7c3c53b9c44306a98ae97e22140782 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:31.834964 10657 tablet_service.cc:1514] Processing DeleteTablet for tablet dba14a44a9de492886d629441606767e with delete_type TABLET_DATA_DELETED (replaced by tablet 080c76ce032848c4af26ec5c8566f300) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:31.835286 10945 tablet_service.cc:1514] Processing DeleteTablet for tablet dba14a44a9de492886d629441606767e with delete_type TABLET_DATA_DELETED (replaced by tablet 080c76ce032848c4af26ec5c8566f300) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:31.839522 11082 raft_consensus.cc:348] T 545ac025f2cb469da847979011a0b3ce P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:31.840164 11082 raft_consensus.cc:374] T 545ac025f2cb469da847979011a0b3ce P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:31.840425 11709 ts_tablet_manager.cc:1905] T c7cd1d6a603642afa2c8c737079961b5 P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:31.840421 11082 raft_consensus.cc:729] T 545ac025f2cb469da847979011a0b3ce P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:31.841056 11082 consensus_queue.cc:260] T 545ac025f2cb469da847979011a0b3ce P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:31.842437 11082 ts_tablet_manager.cc:1428] T 545ac025f2cb469da847979011a0b3ce P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.089s user 0.004s sys 0.000s
I20260925 09:57:31.843232 11082 tablet_bootstrap.cc:492] T 4e3120347c8e4a3eab57d6562ae40e8d P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:31.850445 11082 tablet_bootstrap.cc:654] T 4e3120347c8e4a3eab57d6562ae40e8d P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:31.867233 10392 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 6540aba5854a44419addc5d454e473b7 (table default.Table_8180842f664042b0a9073b7008438ff7 [id=416a4b989570457e919fb7e7b16a017d]). Expected version 1 got 0
W20260925 09:57:31.868675 10391 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f62c62715c9b42de826c1a66fd431eab consensus state has no leader
I20260925 09:57:31.823911 12027 log.cc:1196] T aba253f22afa4200926f31a1b43e3970 P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/aba253f22afa4200926f31a1b43e3970
I20260925 09:57:31.788388 10813 tablet_service.cc:1467] Processing CreateTablet for tablet 0ec6a95911ef44f09e08dfc531966a83 (DEFAULT_TABLE table=default.Table_3275db3804df4b12ad6e64e4d88bc75b [id=2f35a7ab50dd44cab6398a77c110e2fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:31.872171 10813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ec6a95911ef44f09e08dfc531966a83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:31.872377 11727 tablet_replica.cc:330] T 417058493c6547c69e72347cef373292 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:31.873386 11727 raft_consensus.cc:2229] T 417058493c6547c69e72347cef373292 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:31.874115 10392 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 59f0c6489a33433899bb6e62088121b6 (table default.Table_01f37e9d4f5b4dc5a66a24eac603810d [id=64032d286bf94c4cbdddde18adfb739e]). Expected version 2 got 1
I20260925 09:57:31.790097 10810 tablet_service.cc:1467] Processing CreateTablet for tablet 3ae55832f05743bebaae72ce862191c9 (DEFAULT_TABLE table=default.Table_55e6a408ce794b5fbd2fe39d51e6a2aa [id=a19ac637e43146979e3c299d4e46b180]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:31.875701 11727 raft_consensus.cc:2258] T 417058493c6547c69e72347cef373292 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:31.875727 10392 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 3d652a76d96f4dbeafde15f3c16534a6 (table default.Table_be0f658aa41f4188bc16db37e96f4ad1 [id=73083c1560cb4db5a12964387cb49b73]). Expected version 1 got 0
I20260925 09:57:31.875996 10810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ae55832f05743bebaae72ce862191c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:31.876230 10392 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet e684798155d445bfb325a1b7c5c48753 (table default.Table_dfb73f4afe7c464d8dc98356500054e2 [id=ab3c143dc00c43e78531514a114bfd16]). Expected version 1 got 0
I20260925 09:57:31.876837 10392 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 5608c5f9b65b4da7b2fbb56417c92d88 (table default.Table_f37bcee5db2044f2bea9c353982b30cd [id=e46d114e5d484b66bb15eb0416b5ce0d]). Expected version 2 got 0
I20260925 09:57:31.877435 10392 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 26fae3fbfe374f348a1729c21ec751b9 (table default.Table_0011387b45014de4b24b466f4cb907f7 [id=e3f7aa5e988c4b6c98be5cd3d465245d]). Expected version 1 got 0
I20260925 09:57:31.878324 10392 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet c588b0c05a1a4f3ea545066183e76502 (table default.Table_bd6b1d4faeb94310be9fdecc693e5b96 [id=a2086b5e7b334d63972538ed53810c55]). Expected version 1 got 0
I20260925 09:57:31.791946 10809 tablet_service.cc:1467] Processing CreateTablet for tablet 635764b638304b64a523ecd89ee02f80 (DEFAULT_TABLE table=default.Table_2a228b9f88074bb897dd2b8e4624913b [id=2eaeb50bb0744b4db2baad734bf98b29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:31.880319 10809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 635764b638304b64a523ecd89ee02f80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:31.884171 11727 ts_tablet_manager.cc:1905] T 417058493c6547c69e72347cef373292 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:31.786444 11085 raft_consensus.cc:348] T 5608c5f9b65b4da7b2fbb56417c92d88 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:31.903285 11085 raft_consensus.cc:374] T 5608c5f9b65b4da7b2fbb56417c92d88 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:31.905454 11085 raft_consensus.cc:729] T 5608c5f9b65b4da7b2fbb56417c92d88 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:31.911134 11084 tablet_bootstrap.cc:654] T 69028aa7908c45b896da4ba8deef07b1 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:31.920049 12095 raft_consensus.cc:482] T 2f07dc9548364870923165f4ea524310 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:31.924472 12095 raft_consensus.cc:504] T 2f07dc9548364870923165f4ea524310 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:31.926275 12095 leader_election.cc:288] T 2f07dc9548364870923165f4ea524310 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:31.926961 10386 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 476567ad4efa4043b6536f3f28d2ee28
I20260925 09:57:31.929369 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f07dc9548364870923165f4ea524310" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:31.910949 12027 ts_tablet_manager.cc:1939] T aba253f22afa4200926f31a1b43e3970 P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:31.927065 10830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f07dc9548364870923165f4ea524310" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:31.930322 11085 consensus_queue.cc:260] T 5608c5f9b65b4da7b2fbb56417c92d88 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:31.930919 10945 tablet_service.cc:1514] Processing DeleteTablet for tablet 476567ad4efa4043b6536f3f28d2ee28 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:31 UTC) from {username='slave'} at 127.0.0.1:53560
W20260925 09:57:31.930855 10919 leader_election.cc:341] T 2f07dc9548364870923165f4ea524310 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:31.936794 10392 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 2f07dc9548364870923165f4ea524310 (table default.Table_0db927e92a3b4970a7efea3fb765e65e [id=a988264fe43c4eb88a84f8b6359e77a6]). Expected version 1 got 0
I20260925 09:57:31.942664 11085 ts_tablet_manager.cc:1428] T 5608c5f9b65b4da7b2fbb56417c92d88 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.174s user 0.005s sys 0.001s
W20260925 09:57:31.943960 10918 leader_election.cc:341] T 2f07dc9548364870923165f4ea524310 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:31.944397 10918 leader_election.cc:302] T 2f07dc9548364870923165f4ea524310 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:31.945070 12095 raft_consensus.cc:2726] T 2f07dc9548364870923165f4ea524310 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:31.946043 10621 leader_election.cc:302] T 6161774f61ae48ab9764a05f2188a80b P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters: be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:31.946682 12080 raft_consensus.cc:2781] T 6161774f61ae48ab9764a05f2188a80b P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:31.947113 12080 raft_consensus.cc:686] T 6161774f61ae48ab9764a05f2188a80b P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:31.948158 12080 consensus_queue.cc:237] T 6161774f61ae48ab9764a05f2188a80b P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:31.953325 11082 tablet_bootstrap.cc:492] T 4e3120347c8e4a3eab57d6562ae40e8d P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:31.953701 11082 ts_tablet_manager.cc:1397] T 4e3120347c8e4a3eab57d6562ae40e8d P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.111s user 0.011s sys 0.002s
W20260925 09:57:31.954857 10379 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.60:36985, user_credentials={real_user=slave}} blocked reactor thread for 90872.8us
I20260925 09:57:31.955950 11082 raft_consensus.cc:348] T 4e3120347c8e4a3eab57d6562ae40e8d P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:31.956653 11082 raft_consensus.cc:374] T 4e3120347c8e4a3eab57d6562ae40e8d P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:31.956931 11082 raft_consensus.cc:729] T 4e3120347c8e4a3eab57d6562ae40e8d P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:31.957545 11082 consensus_queue.cc:260] T 4e3120347c8e4a3eab57d6562ae40e8d P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:31.959772 11082 ts_tablet_manager.cc:1428] T 4e3120347c8e4a3eab57d6562ae40e8d P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260925 09:57:31.960599 11082 tablet_bootstrap.cc:492] T 2f07dc9548364870923165f4ea524310 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:31.961647 10392 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 42ec275c905343c4be1b72dc2bd0f6e3 (table default.Table_7a8e20521b384bfa8f659eaaf46bae54 [id=cf07ed8756904011ab3dac900ea6705a]). Expected version 2 got 1
I20260925 09:57:31.963342 10385 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet fb9c51f7e7fd4c029be2a710cb9462eb
I20260925 09:57:31.963626 10667 tablet_service.cc:1514] Processing DeleteTablet for tablet 476567ad4efa4043b6536f3f28d2ee28 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:31 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:31.965492 10945 tablet_service.cc:1514] Processing DeleteTablet for tablet fb9c51f7e7fd4c029be2a710cb9462eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:31 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:31.965883 10667 tablet_service.cc:1514] Processing DeleteTablet for tablet fb9c51f7e7fd4c029be2a710cb9462eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:31 UTC) from {username='slave'} at 127.0.0.1:37876
W20260925 09:57:31.968240 10379 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 -> Peer b8be057e0b6d40a3a4b90f2b1f4a0fa2 (127.10.9.60:36985): Couldn't send request to peer b8be057e0b6d40a3a4b90f2b1f4a0fa2. Status: Network error: Client connection negotiation failed: client connection to 127.10.9.60:36985: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260925 09:57:31.988699 10817 tablet_service.cc:1514] Processing DeleteTablet for tablet 476567ad4efa4043b6536f3f28d2ee28 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:31 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:31.988952 10819 tablet_service.cc:1514] Processing DeleteTablet for tablet fb9c51f7e7fd4c029be2a710cb9462eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:31 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:31.989382 10810 tablet_service.cc:1467] Processing CreateTablet for tablet 8f3112845dc744d083c0f5e442f0aef7 (DEFAULT_TABLE table=default.Table_00cfe005d2584fc19576d064db0f5fd1 [id=c03fd560c94d49a98899fd1cf5e60717]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:31.989629 11727 ts_tablet_manager.cc:1918] T 417058493c6547c69e72347cef373292 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:31.991268 10805 tablet_service.cc:1467] Processing CreateTablet for tablet 77ebb59faf3e4fb187bbce29612c25ea (DEFAULT_TABLE table=default.Table_32e7bdf7eaf147ea90740b6b3cfb2fd3 [id=be059d72d5db4915ba767959c6a1c7b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:31.991981 10812 tablet_service.cc:1514] Processing DeleteTablet for tablet dba14a44a9de492886d629441606767e with delete_type TABLET_DATA_DELETED (replaced by tablet 080c76ce032848c4af26ec5c8566f300) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:31.993433 11085 tablet_bootstrap.cc:492] T e684798155d445bfb325a1b7c5c48753 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:31.994088 11727 log.cc:1196] T 417058493c6547c69e72347cef373292 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/417058493c6547c69e72347cef373292
I20260925 09:57:31.996025 11727 ts_tablet_manager.cc:1939] T 417058493c6547c69e72347cef373292 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:31.997210 12134 consensus_queue.cc:1035] T 2358acdaa5104e3eacbab8bf182b8f30 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:32.000437 11082 tablet_bootstrap.cc:654] T 2f07dc9548364870923165f4ea524310 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:32.002460 10805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77ebb59faf3e4fb187bbce29612c25ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:32.016057 11709 ts_tablet_manager.cc:1918] T c7cd1d6a603642afa2c8c737079961b5 P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:32.016511 11709 log.cc:1196] T c7cd1d6a603642afa2c8c737079961b5 P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/c7cd1d6a603642afa2c8c737079961b5
I20260925 09:57:32.017212 11709 ts_tablet_manager.cc:1939] T c7cd1d6a603642afa2c8c737079961b5 P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:32.017990 11709 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 305) took 1963 ms. Trace:
I20260925 09:57:32.018371 11709 rpcz_store.cc:276] 0925 09:57:30.054301 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:30.054430 (+ 129us) service_pool.cc:227] Handling call
0925 09:57:31.745568 (+1691138us) ts_tablet_manager.cc:1145] Deleting tablet c7cd1d6a603642afa2c8c737079961b5
0925 09:57:31.745607 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:31.963106 (+217499us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:32.015861 (+ 52755us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:32.017944 (+ 2083us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1690559}
I20260925 09:57:32.019559 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet c7cd1d6a603642afa2c8c737079961b5 (table default.Table_0507f47042b9445abaf8671c979f8994 [id=55860c5dba9f4c818cfa6d523041ec5f]) successfully deleted
I20260925 09:57:31.947741 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 292) took 2297 ms. Trace:
I20260925 09:57:32.019941 11709 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 310) took 1873 ms. Trace:
I20260925 09:57:32.019263 10810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f3112845dc744d083c0f5e442f0aef7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:32.020318 11709 rpcz_store.cc:276] 0925 09:57:30.146462 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:30.146614 (+ 152us) service_pool.cc:227] Handling call
0925 09:57:32.019641 (+1873027us) ts_tablet_manager.cc:1145] Deleting tablet 61aa6380d30a4b21b70ee2d321247bd8
0925 09:57:32.019679 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:32.019891 (+ 212us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1872416}
W20260925 09:57:32.021560 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 61aa6380d30a4b21b70ee2d321247bd8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 61aa6380d30a4b21b70ee2d321247bd8 already in progress: creating tablet
I20260925 09:57:32.022571 11709 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 311) took 1848 ms. Trace:
I20260925 09:57:32.022924 11709 rpcz_store.cc:276] 0925 09:57:30.173719 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:30.173844 (+ 125us) service_pool.cc:227] Handling call
0925 09:57:32.022273 (+1848429us) ts_tablet_manager.cc:1145] Deleting tablet 815b4b1bd914433798dcb5deec6ec79c
0925 09:57:32.022319 (+ 46us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:32.022529 (+ 210us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1847916}
I20260925 09:57:32.004333 11084 tablet_bootstrap.cc:492] T 69028aa7908c45b896da4ba8deef07b1 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:32.023837 11084 ts_tablet_manager.cc:1397] T 69028aa7908c45b896da4ba8deef07b1 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.338s user 0.010s sys 0.000s
W20260925 09:57:32.024070 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 815b4b1bd914433798dcb5deec6ec79c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 815b4b1bd914433798dcb5deec6ec79c already in progress: creating tablet
I20260925 09:57:32.024941 11709 tablet_replica.cc:330] T 2d087d3e35814c90a03eccd90eeea86c P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:32.025678 11709 raft_consensus.cc:2229] T 2d087d3e35814c90a03eccd90eeea86c P b8850818e2ee42a082802d0095c08d21 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:32.026232 11709 raft_consensus.cc:2258] T 2d087d3e35814c90a03eccd90eeea86c P b8850818e2ee42a082802d0095c08d21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:32.026049 11084 raft_consensus.cc:348] T 69028aa7908c45b896da4ba8deef07b1 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:32.026683 11084 raft_consensus.cc:374] T 69028aa7908c45b896da4ba8deef07b1 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:32.026943 11084 raft_consensus.cc:729] T 69028aa7908c45b896da4ba8deef07b1 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:32.007164 12095 raft_consensus.cc:482] T 9a73a1f242e7430cbf52229c07b4dc82 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:32.027581 12095 raft_consensus.cc:504] T 9a73a1f242e7430cbf52229c07b4dc82 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:32.028399 11709 ts_tablet_manager.cc:1905] T 2d087d3e35814c90a03eccd90eeea86c P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:32.029395 12095 leader_election.cc:288] T 9a73a1f242e7430cbf52229c07b4dc82 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:32.030373 11084 consensus_queue.cc:260] T 69028aa7908c45b896da4ba8deef07b1 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:32.030761 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a73a1f242e7430cbf52229c07b4dc82" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:32.031934 11084 ts_tablet_manager.cc:1428] T 69028aa7908c45b896da4ba8deef07b1 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.008s user 0.004s sys 0.000s
I20260925 09:57:32.032707 11084 tablet_bootstrap.cc:492] T a81530200cf14717bc7cbf67bb73f4e5 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:32.032992 10830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a73a1f242e7430cbf52229c07b4dc82" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:32.034469 10918 leader_election.cc:341] T 9a73a1f242e7430cbf52229c07b4dc82 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:32.035985 10919 leader_election.cc:341] T 9a73a1f242e7430cbf52229c07b4dc82 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:32.036351 10919 leader_election.cc:302] T 9a73a1f242e7430cbf52229c07b4dc82 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:32.038214 11084 tablet_bootstrap.cc:654] T a81530200cf14717bc7cbf67bb73f4e5 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:32.039354 12095 raft_consensus.cc:2726] T 9a73a1f242e7430cbf52229c07b4dc82 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:32.039531 12041 raft_consensus.cc:482] T 9744b9814b69470396ce039348fe5e45 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:32.040055 12041 raft_consensus.cc:504] T 9744b9814b69470396ce039348fe5e45 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:32.042294 12041 leader_election.cc:288] T 9744b9814b69470396ce039348fe5e45 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:32.043776 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9744b9814b69470396ce039348fe5e45" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:32.045674 10919 leader_election.cc:341] T 9744b9814b69470396ce039348fe5e45 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:32.047591 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 246) took 5849 ms. Trace:
I20260925 09:57:32.029946 12027 rpcz_store.cc:276] 0925 09:57:29.650684 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:29.650808 (+ 124us) service_pool.cc:227] Handling call
0925 09:57:31.263712 (+1612904us) ts_tablet_manager.cc:1145] Deleting tablet aba253f22afa4200926f31a1b43e3970
0925 09:57:31.263755 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:31.348388 (+ 84633us) spinlock_profiling.cc:243] Waited 79.4 ms on lock 0x7b54000c3354. stack: 00007f0270186033 00007f0270185f30 00007f026fd07534 00000000004db2b4 00000000004db249 00007f0277092471 00007f0277115352 00007f027752b26c 00007f0278138cfa 00007f0278168e10 00007f027814fe0d 00007f027814fdb9 00007f027814fd49 00007f027814fd11 00007f027814ef3d 0000000000580784
0925 09:57:31.659250 (+310862us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:31.821834 (+162584us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:31.947681 (+125847us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":182634752,"tablet-delete.queue_time_us":1612365}
I20260925 09:57:32.050376 12041 raft_consensus.cc:482] T 350904aa5957496d90a2aa2e637b591c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:32.051466 12041 raft_consensus.cc:504] T 350904aa5957496d90a2aa2e637b591c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:32.052423 10830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9744b9814b69470396ce039348fe5e45" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:32.053486 12041 leader_election.cc:288] T 350904aa5957496d90a2aa2e637b591c P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:32.054849 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 293) took 2403 ms. Trace:
I20260925 09:57:32.055254 12027 rpcz_store.cc:276] 0925 09:57:29.651672 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:29.707640 (+ 55968us) service_pool.cc:227] Handling call
0925 09:57:32.054578 (+2346938us) ts_tablet_manager.cc:1145] Deleting tablet cf35f39bcd6a41128acb2a4936fcc316
0925 09:57:32.054625 (+ 47us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:32.054793 (+ 168us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2346225}
I20260925 09:57:32.057282 10824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "350904aa5957496d90a2aa2e637b591c" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:32.059223 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "350904aa5957496d90a2aa2e637b591c" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:32.060527 10919 leader_election.cc:341] T 350904aa5957496d90a2aa2e637b591c P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:32.062040 10918 leader_election.cc:341] T 9744b9814b69470396ce039348fe5e45 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:32.063262 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet cf35f39bcd6a41128acb2a4936fcc316 because the tablet was not found. No further retry: Not found: Tablet not found: cf35f39bcd6a41128acb2a4936fcc316
I20260925 09:57:32.064268 10918 leader_election.cc:302] T 9744b9814b69470396ce039348fe5e45 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
W20260925 09:57:32.065856 10918 leader_election.cc:341] T 350904aa5957496d90a2aa2e637b591c P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:32.067545 10918 leader_election.cc:302] T 350904aa5957496d90a2aa2e637b591c P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:32.073973 11082 tablet_bootstrap.cc:492] T 2f07dc9548364870923165f4ea524310 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:32.074071 10385 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_baec870bf67344cab7091ef1e210eb9a" } new_table_name: "default.Table_2672598ccbce42eb8f11019f47a6fdda" modify_external_catalogs: true
I20260925 09:57:32.074440 11082 ts_tablet_manager.cc:1397] T 2f07dc9548364870923165f4ea524310 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.114s user 0.006s sys 0.002s
I20260925 09:57:32.076112 12095 raft_consensus.cc:2726] T 350904aa5957496d90a2aa2e637b591c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:32.066285 12041 raft_consensus.cc:2726] T 9744b9814b69470396ce039348fe5e45 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:32.062836 11727 rpcz_store.cc:276] 0925 09:57:26.197899 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:26.204186 (+ 6287us) service_pool.cc:227] Handling call
0925 09:57:31.871720 (+5667534us) ts_tablet_manager.cc:1145] Deleting tablet 417058493c6547c69e72347cef373292
0925 09:57:31.872258 (+ 538us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:31.927048 (+ 54790us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:31.989395 (+ 62347us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:32.047535 (+ 58140us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5666704}
I20260925 09:57:32.079207 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 417058493c6547c69e72347cef373292 (table default.Table_d1b168dcbc624b88980354edd6eff43a [id=8165aba086104ad4ab318b0128fdf3a2]) successfully deleted
I20260925 09:57:32.078812 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 249) took 5821 ms. Trace:
I20260925 09:57:32.081552 11727 rpcz_store.cc:276] 0925 09:57:26.257339 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:26.271142 (+ 13803us) service_pool.cc:227] Handling call
0925 09:57:32.078530 (+5807388us) ts_tablet_manager.cc:1145] Deleting tablet 545ac025f2cb469da847979011a0b3ce
0925 09:57:32.078571 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:32.078750 (+ 179us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5805859}
I20260925 09:57:32.082157 12133 consensus_queue.cc:1035] T 2358acdaa5104e3eacbab8bf182b8f30 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 09:57:32.086768 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 545ac025f2cb469da847979011a0b3ce because the tablet was not found. No further retry: Not found: Tablet not found: 545ac025f2cb469da847979011a0b3ce
I20260925 09:57:32.086992 11727 tablet_replica.cc:330] T cf775c9d9b114ddcb97dc6c6eca8e0fc P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:32.087541 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 294) took 2378 ms. Trace:
I20260925 09:57:32.087773 11727 raft_consensus.cc:2229] T cf775c9d9b114ddcb97dc6c6eca8e0fc P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:32.088263 11727 raft_consensus.cc:2258] T cf775c9d9b114ddcb97dc6c6eca8e0fc P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:32.087883 12027 rpcz_store.cc:276] 0925 09:57:29.708686 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:29.711534 (+ 2848us) service_pool.cc:227] Handling call
0925 09:57:32.056321 (+2344787us) ts_tablet_manager.cc:1145] Deleting tablet ec1b78e61e8d46cb8b87f60786059658
0925 09:57:32.083178 (+ 26857us) spinlock_profiling.cc:243] Waited 26.1 ms on lock 0x7b5800001898. stack: 00007f0270186033 00007f0270185f30 00007f026fd07534 00000000004db2b4 00000000004db249 00007f026fffcf11 00007f0270180851 00007f0270180e82 00007f02790408b9 00007f02780745d8 00007f0278138389 00007f0278138983 00007f0278168e10 00007f027814fe0d 00007f027814fdb9 00007f027814fd49
0925 09:57:32.083233 (+ 55us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:32.087475 (+ 4242us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":60000128,"tablet-delete.queue_time_us":2344025}
I20260925 09:57:32.089928 11727 ts_tablet_manager.cc:1905] T cf775c9d9b114ddcb97dc6c6eca8e0fc P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:32.092152 10387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_2c997080e80a44fba286d0881ef12788"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:32.093597 10387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2c997080e80a44fba286d0881ef12788 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:32.097224 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 295) took 2377 ms. Trace:
I20260925 09:57:32.098346 11709 ts_tablet_manager.cc:1918] T 2d087d3e35814c90a03eccd90eeea86c P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:32.100740 12027 rpcz_store.cc:276] 0925 09:57:29.719778 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:29.720021 (+ 243us) service_pool.cc:227] Handling call
0925 09:57:32.096975 (+2376954us) ts_tablet_manager.cc:1145] Deleting tablet 3be0eccc65564bfba3d6e5c398c61fb6
0925 09:57:32.097019 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:32.097174 (+ 155us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2303043}
I20260925 09:57:32.102006 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 301) took 2316 ms. Trace:
I20260925 09:57:32.102290 12027 rpcz_store.cc:276] 0925 09:57:29.785675 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:29.788139 (+ 2464us) service_pool.cc:227] Handling call
0925 09:57:32.101615 (+2313476us) ts_tablet_manager.cc:1145] Deleting tablet 92814437d53143a79f99199f8ee3a742
0925 09:57:32.101659 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:32.101957 (+ 298us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2303427}
I20260925 09:57:32.103268 12027 tablet_replica.cc:330] T d9e454f82a4e4c429b58f32926319b7c P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:32.104151 12027 raft_consensus.cc:2229] T d9e454f82a4e4c429b58f32926319b7c P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:32.091493 11085 tablet_bootstrap.cc:654] T e684798155d445bfb325a1b7c5c48753 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:32.112313 11709 log.cc:1196] T 2d087d3e35814c90a03eccd90eeea86c P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/2d087d3e35814c90a03eccd90eeea86c
I20260925 09:57:32.113090 11709 ts_tablet_manager.cc:1939] T 2d087d3e35814c90a03eccd90eeea86c P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:32.114221 11709 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 319) took 1649 ms. Trace:
I20260925 09:57:32.114997 11709 rpcz_store.cc:276] 0925 09:57:30.464947 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:30.465063 (+ 116us) service_pool.cc:227] Handling call
0925 09:57:32.024814 (+1559751us) ts_tablet_manager.cc:1145] Deleting tablet 2d087d3e35814c90a03eccd90eeea86c
0925 09:57:32.024850 (+ 36us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:32.046681 (+ 21831us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:32.098144 (+ 51463us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:32.114173 (+ 16029us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1559222}
I20260925 09:57:32.116110 11709 tablet_replica.cc:330] T 197f61525b0c4f9da441d98701d231a3 P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:32.116343 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 2d087d3e35814c90a03eccd90eeea86c (table default.Table_bcf18c926c6e4a62b31f3bffe9b0037e [id=53df4f0ffd794afab1aa17fcfb4d56bc]) successfully deleted
I20260925 09:57:32.119112 11709 raft_consensus.cc:2229] T 197f61525b0c4f9da441d98701d231a3 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:32.120568 11709 raft_consensus.cc:2258] T 197f61525b0c4f9da441d98701d231a3 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:32.122928 11709 ts_tablet_manager.cc:1905] T 197f61525b0c4f9da441d98701d231a3 P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:32.124310 10386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_ccbf8d9f16cb4fc5a5cca3d3f809b1e7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:32.125741 10386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ccbf8d9f16cb4fc5a5cca3d3f809b1e7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:32.137459 10384 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_48f8cb39156845c8bffbb2f9f17a900b" } modify_external_catalogs: true
I20260925 09:57:32.137959 10384 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_48f8cb39156845c8bffbb2f9f17a900b" } modify_external_catalogs: true
I20260925 09:57:32.116760 11082 raft_consensus.cc:348] T 2f07dc9548364870923165f4ea524310 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:32.143646 11082 raft_consensus.cc:374] T 2f07dc9548364870923165f4ea524310 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:32.144001 11082 raft_consensus.cc:729] T 2f07dc9548364870923165f4ea524310 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:32.144636 11082 consensus_queue.cc:260] T 2f07dc9548364870923165f4ea524310 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:32.145946 10393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_acebd27bd1c54fcd8668a2fac337ebdd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:32.146996 11082 ts_tablet_manager.cc:1428] T 2f07dc9548364870923165f4ea524310 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.072s user 0.006s sys 0.001s
W20260925 09:57:32.147423 10393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_acebd27bd1c54fcd8668a2fac337ebdd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:32.147879 11082 tablet_bootstrap.cc:492] T a5d872c3620646e5ac33ceb95de2cbd4 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:32.109193 11084 tablet_bootstrap.cc:492] T a81530200cf14717bc7cbf67bb73f4e5 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:32.153570 11084 ts_tablet_manager.cc:1397] T a81530200cf14717bc7cbf67bb73f4e5 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.121s user 0.005s sys 0.004s
I20260925 09:57:32.157998 11084 raft_consensus.cc:348] T a81530200cf14717bc7cbf67bb73f4e5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:32.158738 11084 raft_consensus.cc:374] T a81530200cf14717bc7cbf67bb73f4e5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:32.159034 11084 raft_consensus.cc:729] T a81530200cf14717bc7cbf67bb73f4e5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:32.159727 11084 consensus_queue.cc:260] T a81530200cf14717bc7cbf67bb73f4e5 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:32.163681 12123 raft_consensus.cc:482] T 5608c5f9b65b4da7b2fbb56417c92d88 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:32.164114 12123 raft_consensus.cc:504] T 5608c5f9b65b4da7b2fbb56417c92d88 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:32.165966 12123 leader_election.cc:288] T 5608c5f9b65b4da7b2fbb56417c92d88 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:32.175184 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5608c5f9b65b4da7b2fbb56417c92d88" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:32.180236 10385 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 25ac35fb43d34c9b85b8e61ad6e386a6 consensus state has no leader
I20260925 09:57:32.180900 10385 master_service.cc:946] ReplaceTablet: received request to replace tablet 1e76f9e0ba554e1cad0cf18863393608 from {username='slave'} at 127.0.0.1:35140
I20260925 09:57:32.181793 10387 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_f8b067e4e4cf498bbf44d7bf24c29363" } modify_external_catalogs: true
I20260925 09:57:32.182410 10387 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_f8b067e4e4cf498bbf44d7bf24c29363" } modify_external_catalogs: true
I20260925 09:57:32.187220 11709 ts_tablet_manager.cc:1918] T 197f61525b0c4f9da441d98701d231a3 P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:32.187808 11709 log.cc:1196] T 197f61525b0c4f9da441d98701d231a3 P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/197f61525b0c4f9da441d98701d231a3
I20260925 09:57:32.188642 11709 ts_tablet_manager.cc:1939] T 197f61525b0c4f9da441d98701d231a3 P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:32.189462 11709 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 323) took 1664 ms. Trace:
I20260925 09:57:32.189862 11709 rpcz_store.cc:276] 0925 09:57:30.525323 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:30.525469 (+ 146us) service_pool.cc:227] Handling call
0925 09:57:32.115979 (+1590510us) ts_tablet_manager.cc:1145] Deleting tablet 197f61525b0c4f9da441d98701d231a3
0925 09:57:32.116015 (+ 36us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:32.158345 (+ 42330us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:32.172380 (+ 14035us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:32.189414 (+ 17034us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1589915}
I20260925 09:57:32.191728 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 197f61525b0c4f9da441d98701d231a3 (table default.Table_1299538965b945cf859dbbd0cb7003f1 [id=cdc838b49cbb40778c9f440e9f83e96d]) successfully deleted
I20260925 09:57:32.191840 12027 raft_consensus.cc:2258] T d9e454f82a4e4c429b58f32926319b7c P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:32.193109 11709 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 324) took 1332 ms. Trace:
I20260925 09:57:32.193697 11709 rpcz_store.cc:276] 0925 09:57:30.860457 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:30.860616 (+ 159us) service_pool.cc:227] Handling call
0925 09:57:32.192620 (+1332004us) ts_tablet_manager.cc:1145] Deleting tablet a58cf1dc81454710a9ad1a8baf155c9f
0925 09:57:32.192658 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:32.193037 (+ 379us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1331341}
W20260925 09:57:32.196717 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 977a59175e5b43eeb6f23b938afd00a3 because the tablet was not found. No further retry: Not found: Tablet not found: 977a59175e5b43eeb6f23b938afd00a3
I20260925 09:57:32.166923 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5608c5f9b65b4da7b2fbb56417c92d88" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:32.198632 10976 raft_consensus.cc:2379] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c43461c4ee854ae1ac52b873ccada318 in current term 1: Already voted for candidate b8850818e2ee42a082802d0095c08d21 in this term.
W20260925 09:57:32.199666 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef already in progress: creating tablet
W20260925 09:57:32.201879 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 0a018bea4200474a9904f62aee37240b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0a018bea4200474a9904f62aee37240b already in progress: creating tablet
I20260925 09:57:32.203027 11084 ts_tablet_manager.cc:1428] T a81530200cf14717bc7cbf67bb73f4e5 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.049s user 0.006s sys 0.000s
I20260925 09:57:32.194334 11085 tablet_bootstrap.cc:492] T e684798155d445bfb325a1b7c5c48753 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:32.199770 10772 leader_election.cc:302] T 5608c5f9b65b4da7b2fbb56417c92d88 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:32.203979 11084 tablet_bootstrap.cc:492] T 3039ec3febf848dba7c4e6fd38bb8053 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:32.204527 11085 ts_tablet_manager.cc:1397] T e684798155d445bfb325a1b7c5c48753 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.211s user 0.003s sys 0.007s
W20260925 09:57:32.206110 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet dba14a44a9de492886d629441606767e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dba14a44a9de492886d629441606767e already in progress: creating tablet
W20260925 09:57:32.208109 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 476567ad4efa4043b6536f3f28d2ee28 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 476567ad4efa4043b6536f3f28d2ee28 already in progress: creating tablet
I20260925 09:57:32.209340 12067 raft_consensus.cc:3037] T 5608c5f9b65b4da7b2fbb56417c92d88 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:57:32.209877 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet fb9c51f7e7fd4c029be2a710cb9462eb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fb9c51f7e7fd4c029be2a710cb9462eb already in progress: creating tablet
I20260925 09:57:32.217031 11082 tablet_bootstrap.cc:654] T a5d872c3620646e5ac33ceb95de2cbd4 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:32.220450 12067 raft_consensus.cc:2726] T 5608c5f9b65b4da7b2fbb56417c92d88 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:32.221815 11085 raft_consensus.cc:348] T e684798155d445bfb325a1b7c5c48753 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:32.224606 11085 raft_consensus.cc:374] T e684798155d445bfb325a1b7c5c48753 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:32.229081 11084 tablet_bootstrap.cc:654] T 3039ec3febf848dba7c4e6fd38bb8053 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:32.232385 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet ec1b78e61e8d46cb8b87f60786059658 because the tablet was not found. No further retry: Not found: Tablet not found: ec1b78e61e8d46cb8b87f60786059658
I20260925 09:57:32.233301 11727 ts_tablet_manager.cc:1918] T cf775c9d9b114ddcb97dc6c6eca8e0fc P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260925 09:57:32.234189 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 3be0eccc65564bfba3d6e5c398c61fb6 because the tablet was not found. No further retry: Not found: Tablet not found: 3be0eccc65564bfba3d6e5c398c61fb6
W20260925 09:57:32.235729 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 92814437d53143a79f99199f8ee3a742 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 92814437d53143a79f99199f8ee3a742 already in progress: creating tablet
I20260925 09:57:32.237444 11085 raft_consensus.cc:729] T e684798155d445bfb325a1b7c5c48753 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:32.238194 11085 consensus_queue.cc:260] T e684798155d445bfb325a1b7c5c48753 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:32.248972 11082 tablet_bootstrap.cc:492] T a5d872c3620646e5ac33ceb95de2cbd4 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:32.236318 12027 ts_tablet_manager.cc:1905] T d9e454f82a4e4c429b58f32926319b7c P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:32.249497 11082 ts_tablet_manager.cc:1397] T a5d872c3620646e5ac33ceb95de2cbd4 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.102s user 0.009s sys 0.001s
I20260925 09:57:32.263541 11727 log.cc:1196] T cf775c9d9b114ddcb97dc6c6eca8e0fc P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/cf775c9d9b114ddcb97dc6c6eca8e0fc
I20260925 09:57:32.264289 11727 ts_tablet_manager.cc:1939] T cf775c9d9b114ddcb97dc6c6eca8e0fc P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:32.265132 11082 raft_consensus.cc:348] T a5d872c3620646e5ac33ceb95de2cbd4 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:32.268656 10833 raft_consensus.cc:3037] T e684798155d445bfb325a1b7c5c48753 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:32.271507 11082 raft_consensus.cc:374] T a5d872c3620646e5ac33ceb95de2cbd4 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:32.271828 11082 raft_consensus.cc:729] T a5d872c3620646e5ac33ceb95de2cbd4 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:32.271983 11085 ts_tablet_manager.cc:1428] T e684798155d445bfb325a1b7c5c48753 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.067s user 0.003s sys 0.002s
I20260925 09:57:32.273430 12080 raft_consensus.cc:482] T d5956be0da6441c79723e72f12dfe54a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:32.273838 12080 raft_consensus.cc:504] T d5956be0da6441c79723e72f12dfe54a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:32.274662 11084 tablet_bootstrap.cc:492] T 3039ec3febf848dba7c4e6fd38bb8053 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:32.275060 11084 ts_tablet_manager.cc:1397] T 3039ec3febf848dba7c4e6fd38bb8053 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.071s user 0.004s sys 0.004s
I20260925 09:57:32.275710 12080 leader_election.cc:288] T d5956be0da6441c79723e72f12dfe54a P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:32.277446 11084 raft_consensus.cc:348] T 3039ec3febf848dba7c4e6fd38bb8053 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:32.279151 11085 tablet_bootstrap.cc:492] T 545ac025f2cb469da847979011a0b3ce P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:32.272444 11082 consensus_queue.cc:260] T a5d872c3620646e5ac33ceb95de2cbd4 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:32.285243 11082 ts_tablet_manager.cc:1428] T a5d872c3620646e5ac33ceb95de2cbd4 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.035s user 0.004s sys 0.000s
I20260925 09:57:32.286603 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5956be0da6441c79723e72f12dfe54a" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:32.287648 10976 raft_consensus.cc:2454] T d5956be0da6441c79723e72f12dfe54a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:32.289150 11084 raft_consensus.cc:374] T 3039ec3febf848dba7c4e6fd38bb8053 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:32.289453 11084 raft_consensus.cc:729] T 3039ec3febf848dba7c4e6fd38bb8053 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:32.288755 10619 leader_election.cc:302] T d5956be0da6441c79723e72f12dfe54a P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:32.290148 12080 raft_consensus.cc:2781] T d5956be0da6441c79723e72f12dfe54a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:32.290437 12080 raft_consensus.cc:482] T d5956be0da6441c79723e72f12dfe54a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:32.290705 12080 raft_consensus.cc:3037] T d5956be0da6441c79723e72f12dfe54a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:32.298113 12080 raft_consensus.cc:504] T d5956be0da6441c79723e72f12dfe54a P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:32.301621 10830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5956be0da6441c79723e72f12dfe54a" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:32.311470 12080 leader_election.cc:288] T d5956be0da6441c79723e72f12dfe54a P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:32.313037 11085 tablet_bootstrap.cc:654] T 545ac025f2cb469da847979011a0b3ce P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:32.313071 10621 leader_election.cc:341] T d5956be0da6441c79723e72f12dfe54a P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:32.314172 10833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5956be0da6441c79723e72f12dfe54a" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:32.291759 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 250) took 5994 ms. Trace:
I20260925 09:57:32.300860 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5956be0da6441c79723e72f12dfe54a" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:32.319952 11727 rpcz_store.cc:276] 0925 09:57:26.297573 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:26.321786 (+ 24213us) service_pool.cc:227] Handling call
0925 09:57:32.086843 (+5765057us) ts_tablet_manager.cc:1145] Deleting tablet cf775c9d9b114ddcb97dc6c6eca8e0fc
0925 09:57:32.086885 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:32.168228 (+ 81343us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:32.233087 (+ 64859us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:32.291704 (+ 58617us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5764418}
I20260925 09:57:32.320312 12113 consensus_queue.cc:1035] T e684798155d445bfb325a1b7c5c48753 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:32.326326 11727 tablet_replica.cc:330] T 9a73a1f242e7430cbf52229c07b4dc82 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:32.326475 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet cf775c9d9b114ddcb97dc6c6eca8e0fc (table default.Table_a749aff6c09e4425afc67db6a98196bd [id=9236e950b8d24547ba2ae47fabf2de6e]) successfully deleted
I20260925 09:57:32.329319 11082 tablet_bootstrap.cc:492] T 6540aba5854a44419addc5d454e473b7 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
W20260925 09:57:32.333087 10621 consensus_peers.cc:487] T 69a96874eea1409a9a5a06b5787fbd99 P b8850818e2ee42a082802d0095c08d21 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:32.338311 11085 tablet_bootstrap.cc:492] T 545ac025f2cb469da847979011a0b3ce P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:32.338724 11085 ts_tablet_manager.cc:1397] T 545ac025f2cb469da847979011a0b3ce P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.060s user 0.008s sys 0.000s
I20260925 09:57:32.342052 10386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_835c3ec6bfb943ceac4d44f0735bb91c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:32.343525 10386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_835c3ec6bfb943ceac4d44f0735bb91c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:32.345379 11082 tablet_bootstrap.cc:654] T 6540aba5854a44419addc5d454e473b7 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:32.328039 11084 consensus_queue.cc:260] T 3039ec3febf848dba7c4e6fd38bb8053 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:32.347734 12113 raft_consensus.cc:482] T 7a31b999f0fd447eacff13de71aa3fff P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:32.348332 12113 raft_consensus.cc:504] T 7a31b999f0fd447eacff13de71aa3fff P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:32.349822 12027 ts_tablet_manager.cc:1918] T d9e454f82a4e4c429b58f32926319b7c P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:32.356427 12027 log.cc:1196] T d9e454f82a4e4c429b58f32926319b7c P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/d9e454f82a4e4c429b58f32926319b7c
I20260925 09:57:32.357481 12027 ts_tablet_manager.cc:1939] T d9e454f82a4e4c429b58f32926319b7c P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:32.361301 10976 raft_consensus.cc:3037] T d5956be0da6441c79723e72f12dfe54a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:57:32.365485 10919 consensus_peers.cc:487] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 -> Peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Couldn't send request to peer b8850818e2ee42a082802d0095c08d21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: aba253f22afa4200926f31a1b43e3970. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:32.366508 11727 raft_consensus.cc:2229] T 9a73a1f242e7430cbf52229c07b4dc82 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:32.366931 11727 raft_consensus.cc:2258] T 9a73a1f242e7430cbf52229c07b4dc82 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260925 09:57:32.373467 10619 leader_election.cc:341] T 7a31b999f0fd447eacff13de71aa3fff P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 7a31b999f0fd447eacff13de71aa3fff
I20260925 09:57:32.376582 10976 raft_consensus.cc:2454] T d5956be0da6441c79723e72f12dfe54a P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:32.371193 12113 leader_election.cc:288] T 7a31b999f0fd447eacff13de71aa3fff P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:32.375329 11727 ts_tablet_manager.cc:1905] T 9a73a1f242e7430cbf52229c07b4dc82 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:32.384601 10386 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_b6e9ba234e6140479745055b3237faeb" } new_table_name: "default.Table_00b6767f5a634daebb7f179c663cb8b8" modify_external_catalogs: true
W20260925 09:57:32.385748 10378 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 25ac35fb43d34c9b85b8e61ad6e386a6 (table default.Table_2672598ccbce42eb8f11019f47a6fdda [id=887ce3cafef64210bb69d424ed5dae8a], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 25ac35fb43d34c9b85b8e61ad6e386a6 consensus state has no leader
I20260925 09:57:32.396106 12041 raft_consensus.cc:482] T 3039ec3febf848dba7c4e6fd38bb8053 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:32.396555 12041 raft_consensus.cc:504] T 3039ec3febf848dba7c4e6fd38bb8053 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:32.398384 12041 leader_election.cc:288] T 3039ec3febf848dba7c4e6fd38bb8053 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:32.400468 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3039ec3febf848dba7c4e6fd38bb8053" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:32.401739 10919 leader_election.cc:341] T 3039ec3febf848dba7c4e6fd38bb8053 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:32.416190 10386 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9ddb3e3e716d4a3b8e75614b334f8c38 has no consensus state
I20260925 09:57:32.419368 12041 raft_consensus.cc:482] T 69028aa7908c45b896da4ba8deef07b1 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:32.420063 12041 raft_consensus.cc:504] T 69028aa7908c45b896da4ba8deef07b1 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:32.424285 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69028aa7908c45b896da4ba8deef07b1" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:32.425742 10919 leader_election.cc:341] T 69028aa7908c45b896da4ba8deef07b1 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:32.426258 12041 leader_election.cc:288] T 69028aa7908c45b896da4ba8deef07b1 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:32.418773 10386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_014617e901574abeb4c2cc2ef676bea3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:32.438853 10386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_014617e901574abeb4c2cc2ef676bea3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:32.447259 12095 raft_consensus.cc:482] T a81530200cf14717bc7cbf67bb73f4e5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:32.447830 12095 raft_consensus.cc:504] T a81530200cf14717bc7cbf67bb73f4e5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:32.450196 12095 leader_election.cc:288] T a81530200cf14717bc7cbf67bb73f4e5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:32.468214 10653 tablet_service.cc:1467] Processing CreateTablet for tablet db7daf6b7ca7473ab1407292ce3a6347 (DEFAULT_TABLE table=default.Table_7f649eb0207a4f06a9527d2e6dc3a7d3 [id=c1fa3533902f42fe9140a7294e7b1a67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:32.469458 10653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db7daf6b7ca7473ab1407292ce3a6347. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:32.474187 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a81530200cf14717bc7cbf67bb73f4e5" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:32.475721 10619 leader_election.cc:302] T d5956be0da6441c79723e72f12dfe54a P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:32.476438 12080 consensus_queue.cc:1035] T 69a96874eea1409a9a5a06b5787fbd99 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:32.477540 12080 raft_consensus.cc:2781] T d5956be0da6441c79723e72f12dfe54a P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:32.477945 12080 raft_consensus.cc:686] T d5956be0da6441c79723e72f12dfe54a P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:32.478652 12080 consensus_queue.cc:237] T d5956be0da6441c79723e72f12dfe54a P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:32.480975 10667 tablet_service.cc:1467] Processing CreateTablet for tablet 9ddb3e3e716d4a3b8e75614b334f8c38 (DEFAULT_TABLE table=default.Table_00b6767f5a634daebb7f179c663cb8b8 [id=e2ef5652a9564363a87f76281d131448]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:32.481905 10667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ddb3e3e716d4a3b8e75614b334f8c38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:32.481894 10944 tablet_service.cc:1467] Processing CreateTablet for tablet db7daf6b7ca7473ab1407292ce3a6347 (DEFAULT_TABLE table=default.Table_7f649eb0207a4f06a9527d2e6dc3a7d3 [id=c1fa3533902f42fe9140a7294e7b1a67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:32.482946 10944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db7daf6b7ca7473ab1407292ce3a6347. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:32.486552 10668 tablet_service.cc:1467] Processing CreateTablet for tablet 944c643e97684e98848d70c2169e2dff (DEFAULT_TABLE table=default.Table_9ec3b7a983bf4da4b28cb07c3cb5b6c1 [id=a9edb46de5074391ad1ec1556abcc58e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:32.487418 10668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 944c643e97684e98848d70c2169e2dff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:32.488723 10946 tablet_service.cc:1467] Processing CreateTablet for tablet 9ddb3e3e716d4a3b8e75614b334f8c38 (DEFAULT_TABLE table=default.Table_00b6767f5a634daebb7f179c663cb8b8 [id=e2ef5652a9564363a87f76281d131448]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:32.489748 10946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ddb3e3e716d4a3b8e75614b334f8c38. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:32.490929 10919 leader_election.cc:341] T a81530200cf14717bc7cbf67bb73f4e5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:32.492192 10654 tablet_service.cc:1467] Processing CreateTablet for tablet 8b062e9f7d1740bda01a8b5bf3b6b794 (DEFAULT_TABLE table=default.Table_e056caa00883493e8714ca2c15ff464e [id=f8e1ddb3fd7548fa876e6d6652453c9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:32.493183 10654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b062e9f7d1740bda01a8b5bf3b6b794. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:32.495573 10950 tablet_service.cc:1467] Processing CreateTablet for tablet 944c643e97684e98848d70c2169e2dff (DEFAULT_TABLE table=default.Table_9ec3b7a983bf4da4b28cb07c3cb5b6c1 [id=a9edb46de5074391ad1ec1556abcc58e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:32.496636 10950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 944c643e97684e98848d70c2169e2dff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:32.496574 10960 tablet_service.cc:1467] Processing CreateTablet for tablet 8b062e9f7d1740bda01a8b5bf3b6b794 (DEFAULT_TABLE table=default.Table_e056caa00883493e8714ca2c15ff464e [id=f8e1ddb3fd7548fa876e6d6652453c9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:32.497557 10960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b062e9f7d1740bda01a8b5bf3b6b794. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:32.498883 10655 tablet_service.cc:1467] Processing CreateTablet for tablet 102bd0e21d9e420f870b87e1e22a68f4 (DEFAULT_TABLE table=default.Table_29f3a4e3540c4755bea01e0ff2182e83 [id=a769ea7e188a4bdd84deeaaad141146a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:32.500435 10657 tablet_service.cc:1467] Processing CreateTablet for tablet ceed51b64e9c412ca7f576197832c58d (DEFAULT_TABLE table=default.Table_219dc5152bc1486a948a0e678b1eb15e [id=a5603636e52a4b0eb88d023deca64b69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:32.507117 10952 tablet_service.cc:1467] Processing CreateTablet for tablet 102bd0e21d9e420f870b87e1e22a68f4 (DEFAULT_TABLE table=default.Table_29f3a4e3540c4755bea01e0ff2182e83 [id=a769ea7e188a4bdd84deeaaad141146a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:32.508180 10952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 102bd0e21d9e420f870b87e1e22a68f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:32.530352 10945 tablet_service.cc:1467] Processing CreateTablet for tablet ceed51b64e9c412ca7f576197832c58d (DEFAULT_TABLE table=default.Table_219dc5152bc1486a948a0e678b1eb15e [id=a5603636e52a4b0eb88d023deca64b69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:32.531492 10945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ceed51b64e9c412ca7f576197832c58d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:32.542749 10947 tablet_service.cc:1467] Processing CreateTablet for tablet e0e784a77b3e41eca507ca7a8d1382bf (DEFAULT_TABLE table=default.Table_50ec368977774d53a7d8643459e4aa10 [id=047bea35cf4c4b37a5d503b60bc77097]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:32.543825 10947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0e784a77b3e41eca507ca7a8d1382bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:32.544278 10664 tablet_service.cc:1467] Processing CreateTablet for tablet e0e784a77b3e41eca507ca7a8d1382bf (DEFAULT_TABLE table=default.Table_50ec368977774d53a7d8643459e4aa10 [id=047bea35cf4c4b37a5d503b60bc77097]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:32.548568 10948 tablet_service.cc:1467] Processing CreateTablet for tablet adc84ee8b9354322af4f67a4d4f6ccce (DEFAULT_TABLE table=default.Table_14a20618504e490289d610b4f74cea84 [id=44f5211ebe4044a1bb2d528a304a1754]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:32.549518 10948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adc84ee8b9354322af4f67a4d4f6ccce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:32.552102 10956 tablet_service.cc:1467] Processing CreateTablet for tablet cada9459ea16493e9021eeb3ea5c9adb (DEFAULT_TABLE table=default.Table_7a410b26ea224bea88b0ea886e0fc427 [id=e8f85135c8f7471c9ec891d87f319127]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:32.553202 10956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cada9459ea16493e9021eeb3ea5c9adb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:32.556854 10655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 102bd0e21d9e420f870b87e1e22a68f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:32.557518 10664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0e784a77b3e41eca507ca7a8d1382bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:32.561554 11084 ts_tablet_manager.cc:1428] T 3039ec3febf848dba7c4e6fd38bb8053 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.286s user 0.001s sys 0.007s
I20260925 09:57:32.573866 12041 raft_consensus.cc:482] T 92814437d53143a79f99199f8ee3a742 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:32.575635 10665 tablet_service.cc:1467] Processing CreateTablet for tablet adc84ee8b9354322af4f67a4d4f6ccce (DEFAULT_TABLE table=default.Table_14a20618504e490289d610b4f74cea84 [id=44f5211ebe4044a1bb2d528a304a1754]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:32.575525 12041 raft_consensus.cc:504] T 92814437d53143a79f99199f8ee3a742 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:32.577224 10658 tablet_service.cc:1467] Processing CreateTablet for tablet cada9459ea16493e9021eeb3ea5c9adb (DEFAULT_TABLE table=default.Table_7a410b26ea224bea88b0ea886e0fc427 [id=e8f85135c8f7471c9ec891d87f319127]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:32.582538 10657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ceed51b64e9c412ca7f576197832c58d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:32.585816 10665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adc84ee8b9354322af4f67a4d4f6ccce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:32.591672 10658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cada9459ea16493e9021eeb3ea5c9adb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:32.473629 10391 master_service.cc:946] ReplaceTablet: received request to replace tablet 350904aa5957496d90a2aa2e637b591c from {username='slave'} at 127.0.0.1:35140
I20260925 09:57:32.483217 10390 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 3d652a76d96f4dbeafde15f3c16534a6 (table default.Table_be0f658aa41f4188bc16db37e96f4ad1 [id=73083c1560cb4db5a12964387cb49b73]). Expected version 1 got 0
I20260925 09:57:32.593842 10390 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 42ec275c905343c4be1b72dc2bd0f6e3 (table default.Table_7a8e20521b384bfa8f659eaaf46bae54 [id=cf07ed8756904011ab3dac900ea6705a]). Expected version 2 got 1
I20260925 09:57:32.594915 10390 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet c588b0c05a1a4f3ea545066183e76502 (table default.Table_bd6b1d4faeb94310be9fdecc693e5b96 [id=a2086b5e7b334d63972538ed53810c55]). Expected version 1 got 0
I20260925 09:57:32.524505 10385 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 1e76f9e0ba554e1cad0cf18863393608
I20260925 09:57:32.596489 10385 catalog_manager.cc:6565] ReplaceTablet: tablet 1e76f9e0ba554e1cad0cf18863393608 deleted and replaced by tablet 963e5bf2ac2848b4967074d28acdae52
I20260925 09:57:32.599249 10390 catalog_manager.cc:5495] T fe91b39f91274a7ca61e9618d8cbb475 P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:32.601466 10390 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 5608c5f9b65b4da7b2fbb56417c92d88 (table default.Table_f37bcee5db2044f2bea9c353982b30cd [id=e46d114e5d484b66bb15eb0416b5ce0d]). Expected version 2 got 0
I20260925 09:57:32.602167 10390 catalog_manager.cc:5495] T 2358acdaa5104e3eacbab8bf182b8f30 P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:32.603992 10390 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet e684798155d445bfb325a1b7c5c48753 (table default.Table_dfb73f4afe7c464d8dc98356500054e2 [id=ab3c143dc00c43e78531514a114bfd16]). Expected version 1 got 0
I20260925 09:57:32.607101 10957 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e76f9e0ba554e1cad0cf18863393608 with delete_type TABLET_DATA_DELETED (replaced by tablet 963e5bf2ac2848b4967074d28acdae52) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:32.622397 12133 raft_consensus.cc:482] T 545ac025f2cb469da847979011a0b3ce P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:32.622804 12133 raft_consensus.cc:504] T 545ac025f2cb469da847979011a0b3ce P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:32.624437 12133 leader_election.cc:288] T 545ac025f2cb469da847979011a0b3ce P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:32.629357 11084 tablet_bootstrap.cc:492] T 1e76f9e0ba554e1cad0cf18863393608 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:32.578040 12041 leader_election.cc:288] T 92814437d53143a79f99199f8ee3a742 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:32.663332 10385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_e63b9b18da2949e9bad5432a2ef0ca6f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:32.665084 10385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e63b9b18da2949e9bad5432a2ef0ca6f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:32.675523 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92814437d53143a79f99199f8ee3a742" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:32.676591 10919 leader_election.cc:341] T 92814437d53143a79f99199f8ee3a742 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:32.677389 10654 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e76f9e0ba554e1cad0cf18863393608 with delete_type TABLET_DATA_DELETED (replaced by tablet 963e5bf2ac2848b4967074d28acdae52) from {username='slave'} at 127.0.0.1:37876
W20260925 09:57:32.679042 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 1e76f9e0ba554e1cad0cf18863393608 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1e76f9e0ba554e1cad0cf18863393608 already in progress: creating tablet
I20260925 09:57:32.687528 12041 raft_consensus.cc:482] T 96a80382bb7d4395a14ad9ae972f4a25 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:32.687955 12041 raft_consensus.cc:504] T 96a80382bb7d4395a14ad9ae972f4a25 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:32.689678 12041 leader_election.cc:288] T 96a80382bb7d4395a14ad9ae972f4a25 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:32.690925 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96a80382bb7d4395a14ad9ae972f4a25" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:32.692166 10919 leader_election.cc:341] T 96a80382bb7d4395a14ad9ae972f4a25 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:32.712841 10621 leader_election.cc:341] T d5956be0da6441c79723e72f12dfe54a P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:32.735129 11085 raft_consensus.cc:348] T 545ac025f2cb469da847979011a0b3ce P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:32.736799 11085 raft_consensus.cc:374] T 545ac025f2cb469da847979011a0b3ce P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:32.737128 11085 raft_consensus.cc:729] T 545ac025f2cb469da847979011a0b3ce P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:32.738363 11085 consensus_queue.cc:260] T 545ac025f2cb469da847979011a0b3ce P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:32.744225 11085 ts_tablet_manager.cc:1428] T 545ac025f2cb469da847979011a0b3ce P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.403s user 0.002s sys 0.007s
I20260925 09:57:32.749094 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 302) took 2332 ms. Trace:
I20260925 09:57:32.749636 12027 rpcz_store.cc:276] 0925 09:57:30.416985 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:30.417463 (+ 478us) service_pool.cc:227] Handling call
0925 09:57:32.103124 (+1685661us) ts_tablet_manager.cc:1145] Deleting tablet d9e454f82a4e4c429b58f32926319b7c
0925 09:57:32.103167 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:32.295301 (+192134us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:32.330851 (+ 35550us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:32.748822 (+417971us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":369239,"tablet-delete.queue_time_us":1685124}
I20260925 09:57:32.754184 12123 raft_consensus.cc:482] T e684798155d445bfb325a1b7c5c48753 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:32.754673 12123 raft_consensus.cc:504] T e684798155d445bfb325a1b7c5c48753 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:32.758211 12123 leader_election.cc:288] T e684798155d445bfb325a1b7c5c48753 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:32.765646 12027 tablet_replica.cc:330] T 3ff4cf556eee494d8cd590884466f09f P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:32.766023 10393 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 3d652a76d96f4dbeafde15f3c16534a6 (table default.Table_be0f658aa41f4188bc16db37e96f4ad1 [id=73083c1560cb4db5a12964387cb49b73]). Expected version 1 got 0
I20260925 09:57:32.766847 12027 raft_consensus.cc:2229] T 3ff4cf556eee494d8cd590884466f09f P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:32.767562 12027 raft_consensus.cc:2258] T 3ff4cf556eee494d8cd590884466f09f P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:32.772243 12027 ts_tablet_manager.cc:1905] T 3ff4cf556eee494d8cd590884466f09f P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 09:57:32.776691 10619 leader_election.cc:341] T 545ac025f2cb469da847979011a0b3ce P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 545ac025f2cb469da847979011a0b3ce
I20260925 09:57:32.777107 12204 consensus_queue.cc:1035] T 5246648245e04e26be2abf1dbf706688 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:32.777608 12133 consensus_queue.cc:1035] T e684798155d445bfb325a1b7c5c48753 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:32.786259 10833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3039ec3febf848dba7c4e6fd38bb8053" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:32.787273 10679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e684798155d445bfb325a1b7c5c48753" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:32.772943 11085 tablet_bootstrap.cc:492] T 69a96874eea1409a9a5a06b5787fbd99 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:32.788533 10833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69028aa7908c45b896da4ba8deef07b1" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:32.800890 11082 tablet_bootstrap.cc:492] T 6540aba5854a44419addc5d454e473b7 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:32.801395 11082 ts_tablet_manager.cc:1397] T 6540aba5854a44419addc5d454e473b7 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.472s user 0.002s sys 0.009s
I20260925 09:57:32.803666 11082 raft_consensus.cc:348] T 6540aba5854a44419addc5d454e473b7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:32.815294 10983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e684798155d445bfb325a1b7c5c48753" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:32.827076 10391 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 350904aa5957496d90a2aa2e637b591c
I20260925 09:57:32.828449 10391 catalog_manager.cc:6565] ReplaceTablet: tablet 350904aa5957496d90a2aa2e637b591c deleted and replaced by tablet 154083d32e0f474f9e3a02e6cca940ef
I20260925 09:57:32.832963 10944 tablet_service.cc:1514] Processing DeleteTablet for tablet 350904aa5957496d90a2aa2e637b591c with delete_type TABLET_DATA_DELETED (replaced by tablet 154083d32e0f474f9e3a02e6cca940ef) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:32.850229 11085 tablet_bootstrap.cc:654] T 69a96874eea1409a9a5a06b5787fbd99 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:32.853426 12166 consensus_queue.cc:1035] T 417058493c6547c69e72347cef373292 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:32.884754 11085 tablet_bootstrap.cc:492] T 69a96874eea1409a9a5a06b5787fbd99 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:32.885234 11085 ts_tablet_manager.cc:1397] T 69a96874eea1409a9a5a06b5787fbd99 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.114s user 0.013s sys 0.004s
I20260925 09:57:32.834481 10668 tablet_service.cc:1514] Processing DeleteTablet for tablet 350904aa5957496d90a2aa2e637b591c with delete_type TABLET_DATA_DELETED (replaced by tablet 154083d32e0f474f9e3a02e6cca940ef) from {username='slave'} at 127.0.0.1:37876
W20260925 09:57:32.887833 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 350904aa5957496d90a2aa2e637b591c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 350904aa5957496d90a2aa2e637b591c already in progress: creating tablet
I20260925 09:57:32.888504 11082 raft_consensus.cc:374] T 6540aba5854a44419addc5d454e473b7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:32.888845 11082 raft_consensus.cc:729] T 6540aba5854a44419addc5d454e473b7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:32.893857 10833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a81530200cf14717bc7cbf67bb73f4e5" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:32.893702 10386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_400af62bad0248dfa8846e2b7637b0f7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:32.895241 10386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_400af62bad0248dfa8846e2b7637b0f7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:32.916169 10385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_d05e29f3834046738be803d038f17c88"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:32.917649 10385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d05e29f3834046738be803d038f17c88 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:32.907745 11085 raft_consensus.cc:348] T 69a96874eea1409a9a5a06b5787fbd99 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:32.930960 11085 raft_consensus.cc:374] T 69a96874eea1409a9a5a06b5787fbd99 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:32.931237 11085 raft_consensus.cc:729] T 69a96874eea1409a9a5a06b5787fbd99 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:32.932727 10386 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_0af2360d761342cf9f057be5f265aa41" } new_table_name: "default.Table_168840855e0746f7ab027c1900842751" modify_external_catalogs: true
I20260925 09:57:32.943781 12207 raft_consensus.cc:482] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b8850818e2ee42a082802d0095c08d21)
I20260925 09:57:32.944308 12207 raft_consensus.cc:504] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:32.946328 12207 leader_election.cc:288] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
W20260925 09:57:32.949237 10919 leader_election.cc:341] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Not found: Tablet not found: 2d087d3e35814c90a03eccd90eeea86c
I20260925 09:57:32.956880 10810 tablet_service.cc:1467] Processing CreateTablet for tablet db7daf6b7ca7473ab1407292ce3a6347 (DEFAULT_TABLE table=default.Table_7f649eb0207a4f06a9527d2e6dc3a7d3 [id=c1fa3533902f42fe9140a7294e7b1a67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:32.958196 10810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db7daf6b7ca7473ab1407292ce3a6347. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:32.958904 10819 tablet_service.cc:1467] Processing CreateTablet for tablet ceed51b64e9c412ca7f576197832c58d (DEFAULT_TABLE table=default.Table_219dc5152bc1486a948a0e678b1eb15e [id=a5603636e52a4b0eb88d023deca64b69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:32.959872 10393 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 2f07dc9548364870923165f4ea524310 (table default.Table_0db927e92a3b4970a7efea3fb765e65e [id=a988264fe43c4eb88a84f8b6359e77a6]). Expected version 1 got 0
I20260925 09:57:32.960073 10819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ceed51b64e9c412ca7f576197832c58d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:32.959923 10815 tablet_service.cc:1467] Processing CreateTablet for tablet cada9459ea16493e9021eeb3ea5c9adb (DEFAULT_TABLE table=default.Table_7a410b26ea224bea88b0ea886e0fc427 [id=e8f85135c8f7471c9ec891d87f319127]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:32.959990 10805 tablet_service.cc:1514] Processing DeleteTablet for tablet 417058493c6547c69e72347cef373292 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:25 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:32.960932 10815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cada9459ea16493e9021eeb3ea5c9adb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:32.961339 10813 tablet_service.cc:1467] Processing CreateTablet for tablet 944c643e97684e98848d70c2169e2dff (DEFAULT_TABLE table=default.Table_9ec3b7a983bf4da4b28cb07c3cb5b6c1 [id=a9edb46de5074391ad1ec1556abcc58e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:32.961757 10393 catalog_manager.cc:5495] T 5608c5f9b65b4da7b2fbb56417c92d88 P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:32.962949 10393 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 5608c5f9b65b4da7b2fbb56417c92d88 (table default.Table_f37bcee5db2044f2bea9c353982b30cd [id=e46d114e5d484b66bb15eb0416b5ce0d]). Expected version 2 got 0
I20260925 09:57:32.964779 10812 tablet_service.cc:1467] Processing CreateTablet for tablet 102bd0e21d9e420f870b87e1e22a68f4 (DEFAULT_TABLE table=default.Table_29f3a4e3540c4755bea01e0ff2182e83 [id=a769ea7e188a4bdd84deeaaad141146a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:32.965590 10772 leader_election.cc:302] T e684798155d445bfb325a1b7c5c48753 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:32.966325 10806 tablet_service.cc:1467] Processing CreateTablet for tablet 9ddb3e3e716d4a3b8e75614b334f8c38 (DEFAULT_TABLE table=default.Table_00b6767f5a634daebb7f179c663cb8b8 [id=e2ef5652a9564363a87f76281d131448]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:32.771302 11084 tablet_bootstrap.cc:654] T 1e76f9e0ba554e1cad0cf18863393608 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:32.967890 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d087d3e35814c90a03eccd90eeea86c" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:32.968680 10835 raft_consensus.cc:2379] T 2d087d3e35814c90a03eccd90eeea86c P c43461c4ee854ae1ac52b873ccada318 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be97ab5d75934a648d4ab76341193ec0 in current term 2: Already voted for candidate c43461c4ee854ae1ac52b873ccada318 in this term.
I20260925 09:57:32.968982 10830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92814437d53143a79f99199f8ee3a742" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:32.969168 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96a80382bb7d4395a14ad9ae972f4a25" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:32.970544 10817 tablet_service.cc:1467] Processing CreateTablet for tablet e0e784a77b3e41eca507ca7a8d1382bf (DEFAULT_TABLE table=default.Table_50ec368977774d53a7d8643459e4aa10 [id=047bea35cf4c4b37a5d503b60bc77097]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:32.971230 12113 raft_consensus.cc:482] T 4e3120347c8e4a3eab57d6562ae40e8d P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:32.971804 12166 raft_consensus.cc:2726] T e684798155d445bfb325a1b7c5c48753 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 09:57:32.971714 12113 raft_consensus.cc:504] T 4e3120347c8e4a3eab57d6562ae40e8d P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:32.972785 10814 tablet_service.cc:1467] Processing CreateTablet for tablet adc84ee8b9354322af4f67a4d4f6ccce (DEFAULT_TABLE table=default.Table_14a20618504e490289d610b4f74cea84 [id=44f5211ebe4044a1bb2d528a304a1754]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:32.973568 12113 leader_election.cc:288] T 4e3120347c8e4a3eab57d6562ae40e8d P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:32.974396 10809 tablet_service.cc:1467] Processing CreateTablet for tablet 8b062e9f7d1740bda01a8b5bf3b6b794 (DEFAULT_TABLE table=default.Table_e056caa00883493e8714ca2c15ff464e [id=f8e1ddb3fd7548fa876e6d6652453c9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:32.955034 11085 consensus_queue.cc:260] T 69a96874eea1409a9a5a06b5787fbd99 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:32.976732 11085 ts_tablet_manager.cc:1428] T 69a96874eea1409a9a5a06b5787fbd99 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.091s user 0.007s sys 0.000s
I20260925 09:57:32.977545 11085 tablet_bootstrap.cc:492] T d5956be0da6441c79723e72f12dfe54a P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:32.975613 10805 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e76f9e0ba554e1cad0cf18863393608 with delete_type TABLET_DATA_DELETED (replaced by tablet 963e5bf2ac2848b4967074d28acdae52) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:32.979599 10805 tablet_service.cc:1514] Processing DeleteTablet for tablet 350904aa5957496d90a2aa2e637b591c with delete_type TABLET_DATA_DELETED (replaced by tablet 154083d32e0f474f9e3a02e6cca940ef) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:32.980176 12207 raft_consensus.cc:482] T 8aff53cfd4994c629a408ee8250280c5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:32.980657 10835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a31b999f0fd447eacff13de71aa3fff" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:32.980615 12207 raft_consensus.cc:504] T 8aff53cfd4994c629a408ee8250280c5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:32.980878 12206 consensus_queue.cc:1035] T 5608c5f9b65b4da7b2fbb56417c92d88 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:32.984604 10829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "545ac025f2cb469da847979011a0b3ce" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:32.985213 10829 raft_consensus.cc:2454] T 545ac025f2cb469da847979011a0b3ce P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:32.987018 12027 ts_tablet_manager.cc:1918] T 3ff4cf556eee494d8cd590884466f09f P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260925 09:57:32.988965 10918 leader_election.cc:341] T 3039ec3febf848dba7c4e6fd38bb8053 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:32.989308 10918 leader_election.cc:302] T 3039ec3febf848dba7c4e6fd38bb8053 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:32.889616 11082 consensus_queue.cc:260] T 6540aba5854a44419addc5d454e473b7 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:32.990317 12039 raft_consensus.cc:2726] T 3039ec3febf848dba7c4e6fd38bb8053 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:32.991910 12027 log.cc:1196] T 3ff4cf556eee494d8cd590884466f09f P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/3ff4cf556eee494d8cd590884466f09f
I20260925 09:57:32.995425 11082 ts_tablet_manager.cc:1428] T 6540aba5854a44419addc5d454e473b7 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.194s user 0.004s sys 0.000s
I20260925 09:57:32.996269 11082 tablet_bootstrap.cc:492] T fb9c51f7e7fd4c029be2a710cb9462eb P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
W20260925 09:57:32.996568 10918 leader_election.cc:341] T 69028aa7908c45b896da4ba8deef07b1 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:32.996919 10918 leader_election.cc:302] T 69028aa7908c45b896da4ba8deef07b1 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:32.988886 10813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 944c643e97684e98848d70c2169e2dff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:32.995965 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aff53cfd4994c629a408ee8250280c5" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:32.998062 10918 leader_election.cc:341] T a81530200cf14717bc7cbf67bb73f4e5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:32.998543 10918 leader_election.cc:302] T a81530200cf14717bc7cbf67bb73f4e5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:32.998934 10817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0e784a77b3e41eca507ca7a8d1382bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:32.999459 10814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adc84ee8b9354322af4f67a4d4f6ccce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:32.999583 12095 raft_consensus.cc:2726] T 69028aa7908c45b896da4ba8deef07b1 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:32.999800 12041 raft_consensus.cc:2726] T a81530200cf14717bc7cbf67bb73f4e5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:33.001084 10918 consensus_peers.cc:487] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: aba253f22afa4200926f31a1b43e3970. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:33.002715 10812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 102bd0e21d9e420f870b87e1e22a68f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:33.003820 10806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ddb3e3e716d4a3b8e75614b334f8c38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:33.007364 10918 leader_election.cc:302] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:33.007867 10809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b062e9f7d1740bda01a8b5bf3b6b794. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:33.009617 12039 raft_consensus.cc:3037] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Advancing to term 2
W20260925 09:57:33.014506 10918 leader_election.cc:341] T 92814437d53143a79f99199f8ee3a742 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:33.014868 10918 leader_election.cc:302] T 92814437d53143a79f99199f8ee3a742 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:33.015508 12095 raft_consensus.cc:2726] T 92814437d53143a79f99199f8ee3a742 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:33.015805 10918 leader_election.cc:341] T 96a80382bb7d4395a14ad9ae972f4a25 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:33.016145 10918 leader_election.cc:302] T 96a80382bb7d4395a14ad9ae972f4a25 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:33.017747 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aff53cfd4994c629a408ee8250280c5" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:33.017234 12039 raft_consensus.cc:2726] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 09:57:33.036221 12095 raft_consensus.cc:2726] T 96a80382bb7d4395a14ad9ae972f4a25 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:33.018961 10919 leader_election.cc:341] T 8aff53cfd4994c629a408ee8250280c5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:33.019814 12207 leader_election.cc:288] T 8aff53cfd4994c629a408ee8250280c5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:33.025091 11084 tablet_bootstrap.cc:492] T 1e76f9e0ba554e1cad0cf18863393608 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:33.037495 11084 ts_tablet_manager.cc:1397] T 1e76f9e0ba554e1cad0cf18863393608 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.474s user 0.010s sys 0.000s
W20260925 09:57:33.038487 10918 leader_election.cc:341] T 8aff53cfd4994c629a408ee8250280c5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:33.042788 10385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_47144aad84a8469dbd8f97ce52d04e9f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:33.044440 10385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_47144aad84a8469dbd8f97ce52d04e9f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:33.050062 10918 leader_election.cc:302] T 8aff53cfd4994c629a408ee8250280c5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:33.041627 11084 raft_consensus.cc:348] T 1e76f9e0ba554e1cad0cf18863393608 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:33.050935 11084 raft_consensus.cc:374] T 1e76f9e0ba554e1cad0cf18863393608 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:33.051416 11084 raft_consensus.cc:729] T 1e76f9e0ba554e1cad0cf18863393608 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:33.052325 11084 consensus_queue.cc:260] T 1e76f9e0ba554e1cad0cf18863393608 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:33.055307 11084 ts_tablet_manager.cc:1428] T 1e76f9e0ba554e1cad0cf18863393608 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.017s user 0.006s sys 0.000s
I20260925 09:57:33.056313 12039 raft_consensus.cc:2726] T 8aff53cfd4994c629a408ee8250280c5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:33.059039 11084 tablet_bootstrap.cc:492] T 05e2c7f9820f41679c4d58343ff3f829 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:33.067826 11084 tablet_bootstrap.cc:654] T 05e2c7f9820f41679c4d58343ff3f829 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:33.078719 11085 tablet_bootstrap.cc:654] T d5956be0da6441c79723e72f12dfe54a P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:33.080302 12080 raft_consensus.cc:482] T a5d872c3620646e5ac33ceb95de2cbd4 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:33.081182 12080 raft_consensus.cc:504] T a5d872c3620646e5ac33ceb95de2cbd4 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:33.092468 12206 raft_consensus.cc:482] T 2f07dc9548364870923165f4ea524310 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:33.092985 12206 raft_consensus.cc:504] T 2f07dc9548364870923165f4ea524310 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:33.095186 12206 leader_election.cc:288] T 2f07dc9548364870923165f4ea524310 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:33.096392 12080 leader_election.cc:288] T a5d872c3620646e5ac33ceb95de2cbd4 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
W20260925 09:57:33.105845 10619 leader_election.cc:341] T 4e3120347c8e4a3eab57d6562ae40e8d P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 4e3120347c8e4a3eab57d6562ae40e8d
W20260925 09:57:33.109475 10386 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cc295f7a9169475f9a9235e657be629f consensus state has no leader
W20260925 09:57:33.111121 10619 consensus_peers.cc:487] T 6161774f61ae48ab9764a05f2188a80b P b8850818e2ee42a082802d0095c08d21 -> Peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Couldn't send request to peer be97ab5d75934a648d4ab76341193ec0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6161774f61ae48ab9764a05f2188a80b. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:33.142777 11082 tablet_bootstrap.cc:654] T fb9c51f7e7fd4c029be2a710cb9462eb P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:33.144044 10393 catalog_manager.cc:5495] T e684798155d445bfb325a1b7c5c48753 P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:33.145424 10393 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet e684798155d445bfb325a1b7c5c48753 (table default.Table_dfb73f4afe7c464d8dc98356500054e2 [id=ab3c143dc00c43e78531514a114bfd16]). Expected version 1 got 0
I20260925 09:57:33.025676 11727 ts_tablet_manager.cc:1918] T 9a73a1f242e7430cbf52229c07b4dc82 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:33.147058 11727 log.cc:1196] T 9a73a1f242e7430cbf52229c07b4dc82 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/9a73a1f242e7430cbf52229c07b4dc82
I20260925 09:57:33.147897 11727 ts_tablet_manager.cc:1939] T 9a73a1f242e7430cbf52229c07b4dc82 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:33.149021 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 254) took 6478 ms. Trace:
I20260925 09:57:33.149470 11727 rpcz_store.cc:276] 0925 09:57:26.671016 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:26.671455 (+ 439us) service_pool.cc:227] Handling call
0925 09:57:32.326183 (+5654728us) ts_tablet_manager.cc:1145] Deleting tablet 9a73a1f242e7430cbf52229c07b4dc82
0925 09:57:32.326223 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:32.645396 (+319173us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:33.025408 (+380012us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:33.148952 (+123544us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":38749,"tablet-delete.queue_time_us":5638632}
I20260925 09:57:33.151520 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 9a73a1f242e7430cbf52229c07b4dc82 (table default.Table_c720bf4c23b44a94b73ef67ad45560ce [id=8256a2288a304db39502a0369c9ab58e]) successfully deleted
I20260925 09:57:33.153368 10973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f07dc9548364870923165f4ea524310" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:33.154034 10973 raft_consensus.cc:2454] T 2f07dc9548364870923165f4ea524310 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:33.157124 10973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5d872c3620646e5ac33ceb95de2cbd4" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:33.157727 10973 raft_consensus.cc:2454] T a5d872c3620646e5ac33ceb95de2cbd4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:33.157855 10825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f07dc9548364870923165f4ea524310" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:33.158495 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet d9e454f82a4e4c429b58f32926319b7c (table default.Table_95a4c3e1f6014e63811447ba4a0e34cf [id=626e547271e6456e81ccaa3b1e59bd97]) successfully deleted
I20260925 09:57:33.171612 10386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_9f0d5b7a076747f1aadbb6ac661362b8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:33.173476 10386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9f0d5b7a076747f1aadbb6ac661362b8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:33.176316 12236 tablet.cc:1722] T 3d652a76d96f4dbeafde15f3c16534a6 P be97ab5d75934a648d4ab76341193ec0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:33.153604 12027 ts_tablet_manager.cc:1939] T 3ff4cf556eee494d8cd590884466f09f P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:33.172132 10619 leader_election.cc:302] T 2f07dc9548364870923165f4ea524310 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:33.196334 12205 raft_consensus.cc:2781] T 2f07dc9548364870923165f4ea524310 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:33.178611 12234 tablet.cc:1722] T 3d652a76d96f4dbeafde15f3c16534a6 P b8850818e2ee42a082802d0095c08d21: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260925 09:57:33.223335 10379 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.9.60:36985: connect: Connection refused (error 111) [suppressed 5 similar messages]
I20260925 09:57:33.146366 10393 catalog_manager.cc:5495] T 5246648245e04e26be2abf1dbf706688 P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:33.190013 10830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e3120347c8e4a3eab57d6562ae40e8d" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:33.237840 11727 tablet_replica.cc:330] T 3039ec3febf848dba7c4e6fd38bb8053 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:33.238889 11727 raft_consensus.cc:2229] T 3039ec3febf848dba7c4e6fd38bb8053 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:33.242792 10393 catalog_manager.cc:5495] T 69a96874eea1409a9a5a06b5787fbd99 P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:33.251097 11727 raft_consensus.cc:2258] T 3039ec3febf848dba7c4e6fd38bb8053 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:33.252974 11727 ts_tablet_manager.cc:1905] T 3039ec3febf848dba7c4e6fd38bb8053 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:33.196854 12205 raft_consensus.cc:482] T 2f07dc9548364870923165f4ea524310 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:33.257838 12205 raft_consensus.cc:3037] T 2f07dc9548364870923165f4ea524310 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:33.248548 12080 raft_consensus.cc:482] T 6540aba5854a44419addc5d454e473b7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:33.260577 12080 raft_consensus.cc:504] T 6540aba5854a44419addc5d454e473b7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:33.263293 12080 leader_election.cc:288] T 6540aba5854a44419addc5d454e473b7 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:33.196460 10619 leader_election.cc:302] T a5d872c3620646e5ac33ceb95de2cbd4 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
W20260925 09:57:33.270782 10619 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 74420.2us
I20260925 09:57:33.271529 12080 raft_consensus.cc:2781] T a5d872c3620646e5ac33ceb95de2cbd4 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:33.271926 12080 raft_consensus.cc:482] T a5d872c3620646e5ac33ceb95de2cbd4 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:33.272307 12080 raft_consensus.cc:3037] T a5d872c3620646e5ac33ceb95de2cbd4 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:33.284827 11084 tablet_bootstrap.cc:492] T 05e2c7f9820f41679c4d58343ff3f829 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:33.285315 11084 ts_tablet_manager.cc:1397] T 05e2c7f9820f41679c4d58343ff3f829 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.226s user 0.024s sys 0.039s
I20260925 09:57:33.293136 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6540aba5854a44419addc5d454e473b7" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:33.293823 10976 raft_consensus.cc:2454] T 6540aba5854a44419addc5d454e473b7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
W20260925 09:57:33.300815 10379 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.60:36985, user_credentials={real_user=slave}} blocked reactor thread for 77492.2us
I20260925 09:57:33.198354 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5d872c3620646e5ac33ceb95de2cbd4" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:33.308900 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 304) took 2828 ms. Trace:
I20260925 09:57:33.309286 12027 rpcz_store.cc:276] 0925 09:57:30.480575 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:30.480670 (+ 95us) service_pool.cc:227] Handling call
0925 09:57:32.751590 (+2270920us) ts_tablet_manager.cc:1145] Deleting tablet 3ff4cf556eee494d8cd590884466f09f
0925 09:57:32.765502 (+ 13912us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:32.872451 (+106949us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:32.986801 (+114350us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:33.308838 (+322037us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2270321}
W20260925 09:57:33.309967 10772 consensus_peers.cc:487] T 417058493c6547c69e72347cef373292 P c43461c4ee854ae1ac52b873ccada318 -> Peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Couldn't send request to peer be97ab5d75934a648d4ab76341193ec0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 417058493c6547c69e72347cef373292. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:33.310617 12027 tablet_replica.cc:330] T c7cd1d6a603642afa2c8c737079961b5 P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:33.312718 12027 raft_consensus.cc:2229] T c7cd1d6a603642afa2c8c737079961b5 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:33.313072 11082 tablet_bootstrap.cc:492] T fb9c51f7e7fd4c029be2a710cb9462eb P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:33.313639 11082 ts_tablet_manager.cc:1397] T fb9c51f7e7fd4c029be2a710cb9462eb P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.318s user 0.009s sys 0.002s
W20260925 09:57:33.316206 10621 leader_election.cc:341] T 7a31b999f0fd447eacff13de71aa3fff P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:33.316570 10621 leader_election.cc:302] T 7a31b999f0fd447eacff13de71aa3fff P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8850818e2ee42a082802d0095c08d21; no voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:33.318055 12206 raft_consensus.cc:2726] T 7a31b999f0fd447eacff13de71aa3fff P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:33.320147 12206 consensus_queue.cc:1035] T 5246648245e04e26be2abf1dbf706688 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:33.320845 12205 raft_consensus.cc:504] T 2f07dc9548364870923165f4ea524310 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:33.322750 12205 leader_election.cc:288] T 2f07dc9548364870923165f4ea524310 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:33.326364 10973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f07dc9548364870923165f4ea524310" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:33.326917 10973 raft_consensus.cc:3037] T 2f07dc9548364870923165f4ea524310 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:33.333421 10973 raft_consensus.cc:2454] T 2f07dc9548364870923165f4ea524310 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:33.328542 11084 raft_consensus.cc:348] T 05e2c7f9820f41679c4d58343ff3f829 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:33.345333 10386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_783b675e5dff47d1a8f7d3b0f535efb9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:33.346792 10386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_783b675e5dff47d1a8f7d3b0f535efb9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:33.377643 11084 raft_consensus.cc:374] T 05e2c7f9820f41679c4d58343ff3f829 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:33.337090 10619 leader_election.cc:302] T 6540aba5854a44419addc5d454e473b7 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
W20260925 09:57:33.387055 10619 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 50064.4us
I20260925 09:57:33.387696 12113 raft_consensus.cc:2781] T 6540aba5854a44419addc5d454e473b7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:33.388150 12113 raft_consensus.cc:482] T 6540aba5854a44419addc5d454e473b7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:33.388450 12113 raft_consensus.cc:3037] T 6540aba5854a44419addc5d454e473b7 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:33.390523 10619 leader_election.cc:302] T 2f07dc9548364870923165f4ea524310 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:33.391409 12206 raft_consensus.cc:2781] T 2f07dc9548364870923165f4ea524310 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:33.391952 12206 raft_consensus.cc:686] T 2f07dc9548364870923165f4ea524310 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:33.392913 12206 consensus_queue.cc:237] T 2f07dc9548364870923165f4ea524310 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:33.399012 12113 raft_consensus.cc:504] T 6540aba5854a44419addc5d454e473b7 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:33.401469 12113 leader_election.cc:288] T 6540aba5854a44419addc5d454e473b7 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:33.402506 12027 pending_rounds.cc:62] T c7cd1d6a603642afa2c8c737079961b5 P c43461c4ee854ae1ac52b873ccada318: Trying to abort 1 pending ops.
I20260925 09:57:33.402770 12027 pending_rounds.cc:68] T c7cd1d6a603642afa2c8c737079961b5 P c43461c4ee854ae1ac52b873ccada318: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192704047448064 op_type: NO_OP noop_request { }
I20260925 09:57:33.403261 12027 raft_consensus.cc:2866] T c7cd1d6a603642afa2c8c737079961b5 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:57:33.402612 10386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_065ec6e727eb4cbebcc6acb042d70574"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:33.403707 12027 raft_consensus.cc:2258] T c7cd1d6a603642afa2c8c737079961b5 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 09:57:33.404947 10386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_065ec6e727eb4cbebcc6acb042d70574 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:33.382311 12080 raft_consensus.cc:504] T a5d872c3620646e5ac33ceb95de2cbd4 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:33.419915 12080 leader_election.cc:288] T a5d872c3620646e5ac33ceb95de2cbd4 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:33.420454 11084 raft_consensus.cc:729] T 05e2c7f9820f41679c4d58343ff3f829 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:33.418781 10982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6540aba5854a44419addc5d454e473b7" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:33.425249 10982 raft_consensus.cc:3037] T 6540aba5854a44419addc5d454e473b7 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:33.427172 11084 consensus_queue.cc:260] T 05e2c7f9820f41679c4d58343ff3f829 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:33.438810 10386 master_service.cc:946] ReplaceTablet: received request to replace tablet cfa619b699004882a82976f0ef8204ad from {username='slave'} at 127.0.0.1:35140
I20260925 09:57:33.317159 11082 raft_consensus.cc:348] T fb9c51f7e7fd4c029be2a710cb9462eb P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:33.445806 11082 raft_consensus.cc:374] T fb9c51f7e7fd4c029be2a710cb9462eb P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:33.446120 11082 raft_consensus.cc:729] T fb9c51f7e7fd4c029be2a710cb9462eb P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:33.447211 11082 consensus_queue.cc:260] T fb9c51f7e7fd4c029be2a710cb9462eb P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:33.444104 12206 consensus_queue.cc:1035] T d5956be0da6441c79723e72f12dfe54a P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:33.430060 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5d872c3620646e5ac33ceb95de2cbd4" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:33.451051 10976 raft_consensus.cc:3037] T a5d872c3620646e5ac33ceb95de2cbd4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:33.459770 11082 ts_tablet_manager.cc:1428] T fb9c51f7e7fd4c029be2a710cb9462eb P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.145s user 0.006s sys 0.000s
I20260925 09:57:33.463686 10976 raft_consensus.cc:2454] T a5d872c3620646e5ac33ceb95de2cbd4 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:33.441253 10982 raft_consensus.cc:2454] T 6540aba5854a44419addc5d454e473b7 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:33.452234 11084 ts_tablet_manager.cc:1428] T 05e2c7f9820f41679c4d58343ff3f829 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.167s user 0.006s sys 0.000s
I20260925 09:57:33.466053 11084 tablet_bootstrap.cc:492] T d54ab15dd054489ab50b7f0b26b79a50 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:33.468619 10619 leader_election.cc:302] T a5d872c3620646e5ac33ceb95de2cbd4 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:33.470145 10619 leader_election.cc:302] T 6540aba5854a44419addc5d454e473b7 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:33.476888 12080 raft_consensus.cc:2781] T a5d872c3620646e5ac33ceb95de2cbd4 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:33.477442 12080 raft_consensus.cc:686] T a5d872c3620646e5ac33ceb95de2cbd4 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:33.476899 11082 tablet_bootstrap.cc:492] T 37bab60afe7a4c758a08ef7c9c24aba0 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:33.478507 12206 raft_consensus.cc:2781] T 6540aba5854a44419addc5d454e473b7 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:33.478977 12206 raft_consensus.cc:686] T 6540aba5854a44419addc5d454e473b7 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:33.479771 12206 consensus_queue.cc:237] T 6540aba5854a44419addc5d454e473b7 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:33.495889 10386 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet cfa619b699004882a82976f0ef8204ad
I20260925 09:57:33.498127 10386 catalog_manager.cc:6565] ReplaceTablet: tablet cfa619b699004882a82976f0ef8204ad deleted and replaced by tablet e83d6dc8657d4a1f89ab0893f435abf6
I20260925 09:57:33.498687 10668 tablet_service.cc:1514] Processing DeleteTablet for tablet cfa619b699004882a82976f0ef8204ad with delete_type TABLET_DATA_DELETED (replaced by tablet e83d6dc8657d4a1f89ab0893f435abf6) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:33.499524 12207 raft_consensus.cc:482] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:33.500180 10950 tablet_service.cc:1514] Processing DeleteTablet for tablet cfa619b699004882a82976f0ef8204ad with delete_type TABLET_DATA_DELETED (replaced by tablet e83d6dc8657d4a1f89ab0893f435abf6) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:33.499995 12207 raft_consensus.cc:504] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:33.505645 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb9c51f7e7fd4c029be2a710cb9462eb" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:33.506242 10684 raft_consensus.cc:2454] T fb9c51f7e7fd4c029be2a710cb9462eb P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be97ab5d75934a648d4ab76341193ec0 in term 0.
I20260925 09:57:33.507354 10919 leader_election.cc:302] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:33.508064 12095 raft_consensus.cc:2781] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:33.508368 12095 raft_consensus.cc:482] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:33.508670 12095 raft_consensus.cc:3037] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:33.508893 11727 ts_tablet_manager.cc:1918] T 3039ec3febf848dba7c4e6fd38bb8053 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:33.509303 11727 log.cc:1196] T 3039ec3febf848dba7c4e6fd38bb8053 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/3039ec3febf848dba7c4e6fd38bb8053
W20260925 09:57:33.510409 10379 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet cc295f7a9169475f9a9235e657be629f (table default.Table_168840855e0746f7ab027c1900842751 [id=99ba81c4d0dd497398e048a05759419a], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet cc295f7a9169475f9a9235e657be629f consensus state has no leader
I20260925 09:57:33.512569 11082 tablet_bootstrap.cc:654] T 37bab60afe7a4c758a08ef7c9c24aba0 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:33.515002 11084 tablet_bootstrap.cc:654] T d54ab15dd054489ab50b7f0b26b79a50 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:33.527215 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb9c51f7e7fd4c029be2a710cb9462eb" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:33.530521 10918 leader_election.cc:341] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:33.515832 12207 leader_election.cc:288] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:33.518493 12095 raft_consensus.cc:504] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:33.551218 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb9c51f7e7fd4c029be2a710cb9462eb" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:33.551923 10684 raft_consensus.cc:3037] T fb9c51f7e7fd4c029be2a710cb9462eb P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:33.554373 11084 tablet_bootstrap.cc:492] T d54ab15dd054489ab50b7f0b26b79a50 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:33.554742 11084 ts_tablet_manager.cc:1397] T d54ab15dd054489ab50b7f0b26b79a50 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.089s user 0.009s sys 0.000s
I20260925 09:57:33.558286 11727 ts_tablet_manager.cc:1939] T 3039ec3febf848dba7c4e6fd38bb8053 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:33.560469 10684 raft_consensus.cc:2454] T fb9c51f7e7fd4c029be2a710cb9462eb P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be97ab5d75934a648d4ab76341193ec0 in term 1.
I20260925 09:57:33.562268 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb9c51f7e7fd4c029be2a710cb9462eb" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
W20260925 09:57:33.564123 10918 leader_election.cc:341] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:33.587518 11435 thread.cc:636] rpc reactor (reactor) Time spent creating pthread: real 1.204s user 0.008s sys 0.003s
W20260925 09:57:33.587783 11435 thread.cc:602] rpc reactor (reactor) Time spent starting thread: real 1.204s user 0.008s sys 0.003s
I20260925 09:57:33.596020 12095 leader_election.cc:288] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:33.600222 11084 raft_consensus.cc:348] T d54ab15dd054489ab50b7f0b26b79a50 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:33.608597 12039 raft_consensus.cc:482] T aca5987b14b64152887ac5ee584b257a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:33.608892 11084 raft_consensus.cc:374] T d54ab15dd054489ab50b7f0b26b79a50 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:33.609158 11084 raft_consensus.cc:729] T d54ab15dd054489ab50b7f0b26b79a50 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:33.609189 12039 raft_consensus.cc:504] T aca5987b14b64152887ac5ee584b257a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:33.609779 11084 consensus_queue.cc:260] T d54ab15dd054489ab50b7f0b26b79a50 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:33.611173 12039 leader_election.cc:288] T aca5987b14b64152887ac5ee584b257a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:33.611755 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aca5987b14b64152887ac5ee584b257a" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:33.613044 10918 leader_election.cc:341] T aca5987b14b64152887ac5ee584b257a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:33.614190 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 253) took 6943 ms. Trace:
I20260925 09:57:33.614718 11727 rpcz_store.cc:276] 0925 09:57:26.670728 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:26.690784 (+ 20056us) service_pool.cc:227] Handling call
0925 09:57:33.193585 (+6502801us) ts_tablet_manager.cc:1145] Deleting tablet 3039ec3febf848dba7c4e6fd38bb8053
0925 09:57:33.237714 (+ 44129us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:33.431357 (+193643us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:33.508539 (+ 77182us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:33.614113 (+105574us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":13944,"tablet-delete.queue_time_us":6499633}
I20260925 09:57:33.616102 11727 tablet_replica.cc:330] T 69028aa7908c45b896da4ba8deef07b1 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:33.617039 11727 raft_consensus.cc:2229] T 69028aa7908c45b896da4ba8deef07b1 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:33.617022 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 3039ec3febf848dba7c4e6fd38bb8053 (table default.Table_908fa06a45334bf7a6700063843028b1 [id=5032e01ba3404dec8925a55ea1afed26]) successfully deleted
I20260925 09:57:33.617569 11727 raft_consensus.cc:2258] T 69028aa7908c45b896da4ba8deef07b1 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:33.581008 11082 tablet_bootstrap.cc:492] T 37bab60afe7a4c758a08ef7c9c24aba0 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:33.634670 11082 ts_tablet_manager.cc:1397] T 37bab60afe7a4c758a08ef7c9c24aba0 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.158s user 0.010s sys 0.000s
I20260925 09:57:33.612286 10681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aca5987b14b64152887ac5ee584b257a" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:33.636066 10919 leader_election.cc:341] T aca5987b14b64152887ac5ee584b257a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:33.636467 10919 leader_election.cc:302] T aca5987b14b64152887ac5ee584b257a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:33.636579 11727 ts_tablet_manager.cc:1905] T 69028aa7908c45b896da4ba8deef07b1 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:33.637499 12095 raft_consensus.cc:2726] T aca5987b14b64152887ac5ee584b257a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:33.637267 11082 raft_consensus.cc:348] T 37bab60afe7a4c758a08ef7c9c24aba0 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:33.638015 11082 raft_consensus.cc:374] T 37bab60afe7a4c758a08ef7c9c24aba0 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:33.638363 11082 raft_consensus.cc:729] T 37bab60afe7a4c758a08ef7c9c24aba0 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:33.639055 11082 consensus_queue.cc:260] T 37bab60afe7a4c758a08ef7c9c24aba0 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:33.643045 11082 ts_tablet_manager.cc:1428] T 37bab60afe7a4c758a08ef7c9c24aba0 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.008s user 0.002s sys 0.003s
I20260925 09:57:33.613651 12080 consensus_queue.cc:237] T a5d872c3620646e5ac33ceb95de2cbd4 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:33.647341 10919 leader_election.cc:302] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:33.648640 12095 raft_consensus.cc:2781] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:33.650311 12095 raft_consensus.cc:686] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0 [term 1 LEADER]: Becoming Leader. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Running, Role: LEADER
I20260925 09:57:33.652482 12095 consensus_queue.cc:237] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:33.664911 11085 tablet_bootstrap.cc:492] T d5956be0da6441c79723e72f12dfe54a P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:33.665333 11085 ts_tablet_manager.cc:1397] T d5956be0da6441c79723e72f12dfe54a P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.688s user 0.017s sys 0.014s
I20260925 09:57:33.667788 12027 ts_tablet_manager.cc:1905] T c7cd1d6a603642afa2c8c737079961b5 P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:33.679559 10621 leader_election.cc:302] T 545ac025f2cb469da847979011a0b3ce P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters: be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:33.679844 11727 ts_tablet_manager.cc:1918] T 69028aa7908c45b896da4ba8deef07b1 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:33.680258 11727 log.cc:1196] T 69028aa7908c45b896da4ba8deef07b1 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/69028aa7908c45b896da4ba8deef07b1
I20260925 09:57:33.680773 12113 raft_consensus.cc:2781] T 545ac025f2cb469da847979011a0b3ce P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:33.680990 11727 ts_tablet_manager.cc:1939] T 69028aa7908c45b896da4ba8deef07b1 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:33.681180 12113 raft_consensus.cc:482] T 545ac025f2cb469da847979011a0b3ce P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:33.681558 12113 raft_consensus.cc:3037] T 545ac025f2cb469da847979011a0b3ce P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:33.683143 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 255) took 6885 ms. Trace:
I20260925 09:57:33.683558 11727 rpcz_store.cc:276] 0925 09:57:26.797457 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:26.797602 (+ 145us) service_pool.cc:227] Handling call
0925 09:57:33.615952 (+6818350us) ts_tablet_manager.cc:1145] Deleting tablet 69028aa7908c45b896da4ba8deef07b1
0925 09:57:33.615992 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:33.668723 (+ 52731us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:33.679540 (+ 10817us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:33.683090 (+ 3550us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6817718}
I20260925 09:57:33.685043 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 69028aa7908c45b896da4ba8deef07b1 (table default.Table_fad50b13f9024bf8b4df345241f8c32b [id=351cf96e6f86425cb5077c576bc006db]) successfully deleted
I20260925 09:57:33.685586 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 256) took 6873 ms. Trace:
I20260925 09:57:33.686080 11727 rpcz_store.cc:276] 0925 09:57:26.812267 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:26.812382 (+ 115us) service_pool.cc:227] Handling call
0925 09:57:33.685258 (+6872876us) ts_tablet_manager.cc:1145] Deleting tablet 689ab2b6cf1640129f6db8b113cf2fc6
0925 09:57:33.685314 (+ 56us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:33.685527 (+ 213us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6872250}
W20260925 09:57:33.687425 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 689ab2b6cf1640129f6db8b113cf2fc6 because the tablet was not found. No further retry: Not found: Tablet not found: 689ab2b6cf1640129f6db8b113cf2fc6
I20260925 09:57:33.687841 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 257) took 6869 ms. Trace:
I20260925 09:57:33.688166 11727 rpcz_store.cc:276] 0925 09:57:26.818762 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:26.819536 (+ 774us) service_pool.cc:227] Handling call
0925 09:57:33.687544 (+6868008us) ts_tablet_manager.cc:1145] Deleting tablet a44d9f2f7e3e48aeb0f31fa29ae1988a
0925 09:57:33.687613 (+ 69us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:33.687799 (+ 186us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6867295}
I20260925 09:57:33.688512 12205 consensus_queue.cc:1035] T 5608c5f9b65b4da7b2fbb56417c92d88 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:33.689546 11727 tablet_replica.cc:330] T a81530200cf14717bc7cbf67bb73f4e5 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:33.689910 12207 raft_consensus.cc:482] T b61444c31dbf4882ac748f861ea66818 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:33.690356 12207 raft_consensus.cc:504] T b61444c31dbf4882ac748f861ea66818 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:33.691138 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet a44d9f2f7e3e48aeb0f31fa29ae1988a because the tablet was not found. No further retry: Not found: Tablet not found: a44d9f2f7e3e48aeb0f31fa29ae1988a
I20260925 09:57:33.692258 12113 raft_consensus.cc:504] T 545ac025f2cb469da847979011a0b3ce P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:33.692595 12207 leader_election.cc:288] T b61444c31dbf4882ac748f861ea66818 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:33.693631 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b61444c31dbf4882ac748f861ea66818" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:33.694631 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b61444c31dbf4882ac748f861ea66818" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:33.695011 10919 leader_election.cc:341] T b61444c31dbf4882ac748f861ea66818 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:33.703615 12247 tablet_replica.cc:330] T cfa619b699004882a82976f0ef8204ad P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
W20260925 09:57:33.704526 10918 leader_election.cc:341] T b61444c31dbf4882ac748f861ea66818 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:33.704908 10918 leader_election.cc:302] T b61444c31dbf4882ac748f861ea66818 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
W20260925 09:57:33.707085 10619 leader_election.cc:341] T 545ac025f2cb469da847979011a0b3ce P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 545ac025f2cb469da847979011a0b3ce
I20260925 09:57:33.709654 12207 raft_consensus.cc:2726] T b61444c31dbf4882ac748f861ea66818 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:33.711233 10621 leader_election.cc:341] T 2f07dc9548364870923165f4ea524310 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:33.712071 12205 raft_consensus.cc:482] T 37bab60afe7a4c758a08ef7c9c24aba0 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:33.712481 12205 raft_consensus.cc:504] T 37bab60afe7a4c758a08ef7c9c24aba0 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:33.690361 11727 raft_consensus.cc:2229] T a81530200cf14717bc7cbf67bb73f4e5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:33.713779 11727 raft_consensus.cc:2258] T a81530200cf14717bc7cbf67bb73f4e5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:33.714392 12205 leader_election.cc:288] T 37bab60afe7a4c758a08ef7c9c24aba0 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:33.715551 11727 ts_tablet_manager.cc:1905] T a81530200cf14717bc7cbf67bb73f4e5 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:33.720431 10973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37bab60afe7a4c758a08ef7c9c24aba0" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:33.721021 10973 raft_consensus.cc:2454] T 37bab60afe7a4c758a08ef7c9c24aba0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:33.725132 10619 leader_election.cc:302] T 37bab60afe7a4c758a08ef7c9c24aba0 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:33.727473 12204 raft_consensus.cc:2781] T 37bab60afe7a4c758a08ef7c9c24aba0 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:33.727787 12204 raft_consensus.cc:482] T 37bab60afe7a4c758a08ef7c9c24aba0 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:33.728065 12204 raft_consensus.cc:3037] T 37bab60afe7a4c758a08ef7c9c24aba0 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:33.739179 11082 tablet_bootstrap.cc:492] T 10657a81058241fc988d37e5ea7ff8cf P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:33.744443 11084 ts_tablet_manager.cc:1428] T d54ab15dd054489ab50b7f0b26b79a50 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.189s user 0.003s sys 0.003s
I20260925 09:57:33.752153 12204 raft_consensus.cc:504] T 37bab60afe7a4c758a08ef7c9c24aba0 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:33.755689 12204 leader_election.cc:288] T 37bab60afe7a4c758a08ef7c9c24aba0 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:33.759845 12234 tablet.cc:1722] T e684798155d445bfb325a1b7c5c48753 P b8850818e2ee42a082802d0095c08d21: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:33.761319 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 3ff4cf556eee494d8cd590884466f09f (table default.Table_66b8a45b14bf441d80ab32ee756de608 [id=2aeb0fb9ff8c416aba7b24d74354fcb0]) successfully deleted
I20260925 09:57:33.765618 10813 tablet_service.cc:1514] Processing DeleteTablet for tablet cfa619b699004882a82976f0ef8204ad with delete_type TABLET_DATA_DELETED (replaced by tablet e83d6dc8657d4a1f89ab0893f435abf6) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:33.767433 11085 raft_consensus.cc:348] T d5956be0da6441c79723e72f12dfe54a P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:33.768421 11085 raft_consensus.cc:374] T d5956be0da6441c79723e72f12dfe54a P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:33.768723 11085 raft_consensus.cc:729] T d5956be0da6441c79723e72f12dfe54a P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:33.769575 11085 consensus_queue.cc:260] T d5956be0da6441c79723e72f12dfe54a P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:33.771329 11085 ts_tablet_manager.cc:1428] T d5956be0da6441c79723e72f12dfe54a P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.106s user 0.006s sys 0.000s
W20260925 09:57:33.784416 10621 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 71618.1us
I20260925 09:57:33.784912 11082 tablet_bootstrap.cc:654] T 10657a81058241fc988d37e5ea7ff8cf P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:33.745185 11727 ts_tablet_manager.cc:1918] T a81530200cf14717bc7cbf67bb73f4e5 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:33.759732 12236 tablet.cc:1722] T e684798155d445bfb325a1b7c5c48753 P be97ab5d75934a648d4ab76341193ec0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:33.785744 11727 log.cc:1196] T a81530200cf14717bc7cbf67bb73f4e5 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/a81530200cf14717bc7cbf67bb73f4e5
I20260925 09:57:33.786448 11727 ts_tablet_manager.cc:1939] T a81530200cf14717bc7cbf67bb73f4e5 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:33.788504 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 258) took 6690 ms. Trace:
I20260925 09:57:33.788940 11727 rpcz_store.cc:276] 0925 09:57:27.098467 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:27.098606 (+ 139us) service_pool.cc:227] Handling call
0925 09:57:33.689382 (+6590776us) ts_tablet_manager.cc:1145] Deleting tablet a81530200cf14717bc7cbf67bb73f4e5
0925 09:57:33.689434 (+ 52us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:33.733803 (+ 44369us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:33.744984 (+ 11181us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:33.788443 (+ 43459us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6590160}
I20260925 09:57:33.790525 11727 tablet_replica.cc:330] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:33.793661 11727 raft_consensus.cc:2229] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [term 2 LEADER]: Raft consensus shutting down.
I20260925 09:57:33.794454 11727 pending_rounds.cc:62] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0: Trying to abort 1 pending ops.
I20260925 09:57:33.794770 11727 pending_rounds.cc:68] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0: Aborting op as it isn't in flight: id { term: 2 index: 1 } timestamp: 7333192710178947072 op_type: NO_OP noop_request { }
I20260925 09:57:33.795802 11727 raft_consensus.cc:2866] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:57:33.796219 11727 raft_consensus.cc:2258] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:33.798066 11727 ts_tablet_manager.cc:1905] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:33.694098 12113 leader_election.cc:288] T 545ac025f2cb469da847979011a0b3ce P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
W20260925 09:57:33.800549 10621 leader_election.cc:341] T 4e3120347c8e4a3eab57d6562ae40e8d P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:33.800987 10621 leader_election.cc:302] T 4e3120347c8e4a3eab57d6562ae40e8d P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8850818e2ee42a082802d0095c08d21; no voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:33.801743 12113 raft_consensus.cc:2726] T 4e3120347c8e4a3eab57d6562ae40e8d P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:33.802438 10621 leader_election.cc:341] T a5d872c3620646e5ac33ceb95de2cbd4 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:33.755620 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37bab60afe7a4c758a08ef7c9c24aba0" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:33.806440 10976 raft_consensus.cc:3037] T 37bab60afe7a4c758a08ef7c9c24aba0 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:33.818239 10976 raft_consensus.cc:2454] T 37bab60afe7a4c758a08ef7c9c24aba0 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:33.820986 11082 tablet_bootstrap.cc:492] T 10657a81058241fc988d37e5ea7ff8cf P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:33.821594 11082 ts_tablet_manager.cc:1397] T 10657a81058241fc988d37e5ea7ff8cf P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.083s user 0.005s sys 0.005s
I20260925 09:57:33.821830 10619 leader_election.cc:302] T 37bab60afe7a4c758a08ef7c9c24aba0 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:33.822556 12206 raft_consensus.cc:2781] T 37bab60afe7a4c758a08ef7c9c24aba0 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:33.767160 11084 tablet_bootstrap.cc:492] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:33.822989 12206 raft_consensus.cc:686] T 37bab60afe7a4c758a08ef7c9c24aba0 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:33.823895 11082 raft_consensus.cc:348] T 10657a81058241fc988d37e5ea7ff8cf P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:33.824105 12206 consensus_queue.cc:237] T 37bab60afe7a4c758a08ef7c9c24aba0 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:33.834240 11082 raft_consensus.cc:374] T 10657a81058241fc988d37e5ea7ff8cf P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:33.834856 11082 raft_consensus.cc:729] T 10657a81058241fc988d37e5ea7ff8cf P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:33.835980 11082 consensus_queue.cc:260] T 10657a81058241fc988d37e5ea7ff8cf P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:33.838186 11082 ts_tablet_manager.cc:1428] T 10657a81058241fc988d37e5ea7ff8cf P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.016s user 0.004s sys 0.000s
I20260925 09:57:33.838250 11084 tablet_bootstrap.cc:654] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:33.841625 11435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:57:33.857801 11082 tablet_bootstrap.cc:492] T dba14a44a9de492886d629441606767e P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:33.861140 11435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:57:33.862919 11727 ts_tablet_manager.cc:1918] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.1
I20260925 09:57:33.863651 11727 log.cc:1196] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/aba253f22afa4200926f31a1b43e3970
I20260925 09:57:33.864123 11085 tablet_bootstrap.cc:492] T 7a31b999f0fd447eacff13de71aa3fff P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:33.864756 11435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:57:33.865259 11727 ts_tablet_manager.cc:1939] T aba253f22afa4200926f31a1b43e3970 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:33.868075 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6540aba5854a44419addc5d454e473b7" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:33.871474 11082 tablet_bootstrap.cc:654] T dba14a44a9de492886d629441606767e P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:33.871966 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 259) took 6629 ms. Trace:
I20260925 09:57:33.872344 11727 rpcz_store.cc:276] 0925 09:57:27.242369 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:27.242516 (+ 147us) service_pool.cc:227] Handling call
0925 09:57:33.790369 (+6547853us) ts_tablet_manager.cc:1145] Deleting tablet aba253f22afa4200926f31a1b43e3970
0925 09:57:33.790414 (+ 45us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:33.842149 (+ 51735us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:33.862670 (+ 20521us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:33.871912 (+ 9242us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6547200}
I20260925 09:57:33.873919 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 260) took 6046 ms. Trace:
I20260925 09:57:33.874276 11727 rpcz_store.cc:276] 0925 09:57:27.827706 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:27.827838 (+ 132us) service_pool.cc:227] Handling call
0925 09:57:33.873592 (+6045754us) ts_tablet_manager.cc:1145] Deleting tablet 17bd30416fe14c0e9ab2e0c0bf5f2fb5
0925 09:57:33.873633 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:33.873874 (+ 241us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6045138}
W20260925 09:57:33.875300 10377 catalog_manager.cc:4821] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 17bd30416fe14c0e9ab2e0c0bf5f2fb5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 17bd30416fe14c0e9ab2e0c0bf5f2fb5 already in progress: creating tablet
I20260925 09:57:33.875522 11727 tablet_replica.cc:330] T 37bab60afe7a4c758a08ef7c9c24aba0 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:33.876250 11727 raft_consensus.cc:2229] T 37bab60afe7a4c758a08ef7c9c24aba0 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:33.876740 11727 raft_consensus.cc:2258] T 37bab60afe7a4c758a08ef7c9c24aba0 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:33.879592 11727 ts_tablet_manager.cc:1905] T 37bab60afe7a4c758a08ef7c9c24aba0 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:33.882674 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f07dc9548364870923165f4ea524310" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:33.882926 10838 raft_consensus.cc:3037] T d5956be0da6441c79723e72f12dfe54a P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:33.887493 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6540aba5854a44419addc5d454e473b7" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:33.892931 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5d872c3620646e5ac33ceb95de2cbd4" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:33.894433 10836 raft_consensus.cc:3037] T 69a96874eea1409a9a5a06b5787fbd99 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:33.895171 11084 tablet_bootstrap.cc:492] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:33.895651 11084 ts_tablet_manager.cc:1397] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.129s user 0.011s sys 0.000s
I20260925 09:57:33.898260 11084 raft_consensus.cc:348] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:33.902005 12251 raft_consensus.cc:482] T 05e2c7f9820f41679c4d58343ff3f829 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:33.903007 11084 raft_consensus.cc:374] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:33.903326 11084 raft_consensus.cc:729] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:33.903163 12251 raft_consensus.cc:504] T 05e2c7f9820f41679c4d58343ff3f829 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:33.904099 11084 consensus_queue.cc:260] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:33.908316 10679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05e2c7f9820f41679c4d58343ff3f829" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:33.906157 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05e2c7f9820f41679c4d58343ff3f829" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:33.906270 11084 ts_tablet_manager.cc:1428] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.010s user 0.004s sys 0.000s
W20260925 09:57:33.909736 10919 leader_election.cc:341] T 05e2c7f9820f41679c4d58343ff3f829 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:33.916283 10918 leader_election.cc:341] T 05e2c7f9820f41679c4d58343ff3f829 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:33.916919 10918 leader_election.cc:302] T 05e2c7f9820f41679c4d58343ff3f829 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:33.917124 11082 tablet_bootstrap.cc:492] T dba14a44a9de492886d629441606767e P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:33.917493 11082 ts_tablet_manager.cc:1397] T dba14a44a9de492886d629441606767e P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.060s user 0.008s sys 0.004s
I20260925 09:57:33.917732 12207 raft_consensus.cc:2726] T 05e2c7f9820f41679c4d58343ff3f829 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:33.907948 12251 leader_election.cc:288] T 05e2c7f9820f41679c4d58343ff3f829 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:33.919919 11082 raft_consensus.cc:348] T dba14a44a9de492886d629441606767e P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:33.920593 11082 raft_consensus.cc:374] T dba14a44a9de492886d629441606767e P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:33.920893 11082 raft_consensus.cc:729] T dba14a44a9de492886d629441606767e P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:33.921471 11082 consensus_queue.cc:260] T dba14a44a9de492886d629441606767e P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:33.914774 11727 ts_tablet_manager.cc:1918] T 37bab60afe7a4c758a08ef7c9c24aba0 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:33.922534 11727 log.cc:1196] T 37bab60afe7a4c758a08ef7c9c24aba0 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/37bab60afe7a4c758a08ef7c9c24aba0
I20260925 09:57:33.923593 11727 ts_tablet_manager.cc:1939] T 37bab60afe7a4c758a08ef7c9c24aba0 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:33.924867 11082 ts_tablet_manager.cc:1428] T dba14a44a9de492886d629441606767e P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20260925 09:57:33.946756 12251 raft_consensus.cc:482] T 1e76f9e0ba554e1cad0cf18863393608 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:33.946671 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "545ac025f2cb469da847979011a0b3ce" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:33.947350 10838 raft_consensus.cc:3037] T 545ac025f2cb469da847979011a0b3ce P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:33.953703 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37bab60afe7a4c758a08ef7c9c24aba0" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:33.947345 12251 raft_consensus.cc:504] T 1e76f9e0ba554e1cad0cf18863393608 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:33.957901 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e76f9e0ba554e1cad0cf18863393608" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:33.958190 12251 leader_election.cc:288] T 1e76f9e0ba554e1cad0cf18863393608 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
W20260925 09:57:33.959345 10918 leader_election.cc:341] T 1e76f9e0ba554e1cad0cf18863393608 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:33.959139 10679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e76f9e0ba554e1cad0cf18863393608" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:33.960665 10919 leader_election.cc:341] T 1e76f9e0ba554e1cad0cf18863393608 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:33.961112 10919 leader_election.cc:302] T 1e76f9e0ba554e1cad0cf18863393608 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:33.962546 12251 raft_consensus.cc:2726] T 1e76f9e0ba554e1cad0cf18863393608 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:33.964190 10838 raft_consensus.cc:2454] T 545ac025f2cb469da847979011a0b3ce P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:33.966060 11085 tablet_bootstrap.cc:654] T 7a31b999f0fd447eacff13de71aa3fff P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:33.985196 11082 tablet_bootstrap.cc:492] T 8aff53cfd4994c629a408ee8250280c5 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:33.986264 12247 raft_consensus.cc:2229] T cfa619b699004882a82976f0ef8204ad P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:33.987165 12247 raft_consensus.cc:2258] T cfa619b699004882a82976f0ef8204ad P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:33.990360 12247 ts_tablet_manager.cc:1905] T cfa619b699004882a82976f0ef8204ad P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:33.991684 11082 tablet_bootstrap.cc:654] T 8aff53cfd4994c629a408ee8250280c5 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:33.999244 12273 tablet.cc:1722] T 3d652a76d96f4dbeafde15f3c16534a6 P c43461c4ee854ae1ac52b873ccada318: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:34.008533 11082 tablet_bootstrap.cc:492] T 8aff53cfd4994c629a408ee8250280c5 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:34.011684 11082 ts_tablet_manager.cc:1397] T 8aff53cfd4994c629a408ee8250280c5 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.027s user 0.012s sys 0.002s
I20260925 09:57:34.013823 11082 raft_consensus.cc:348] T 8aff53cfd4994c629a408ee8250280c5 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.017506 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37bab60afe7a4c758a08ef7c9c24aba0" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:34.014920 12206 raft_consensus.cc:482] T 10657a81058241fc988d37e5ea7ff8cf P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:34.020082 12206 raft_consensus.cc:504] T 10657a81058241fc988d37e5ea7ff8cf P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.022542 12206 leader_election.cc:288] T 10657a81058241fc988d37e5ea7ff8cf P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:34.023042 11082 raft_consensus.cc:374] T 8aff53cfd4994c629a408ee8250280c5 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:34.023623 11082 raft_consensus.cc:729] T 8aff53cfd4994c629a408ee8250280c5 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:34.025655 11082 consensus_queue.cc:260] T 8aff53cfd4994c629a408ee8250280c5 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.032313 11082 ts_tablet_manager.cc:1428] T 8aff53cfd4994c629a408ee8250280c5 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.020s user 0.004s sys 0.001s
W20260925 09:57:34.032141 10282 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 60370.9us
I20260925 09:57:34.035597 11082 tablet_bootstrap.cc:492] T b61444c31dbf4882ac748f861ea66818 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:34.041358 12274 tablet.cc:1722] T e684798155d445bfb325a1b7c5c48753 P c43461c4ee854ae1ac52b873ccada318: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:34.043143 12275 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8af3f111e8da4d2dad4e4a98ef0fd5d3)
I20260925 09:57:34.047097 11082 tablet_bootstrap.cc:654] T b61444c31dbf4882ac748f861ea66818 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:34.056912 11082 tablet_bootstrap.cc:492] T b61444c31dbf4882ac748f861ea66818 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:34.058670 11082 ts_tablet_manager.cc:1397] T b61444c31dbf4882ac748f861ea66818 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.023s user 0.009s sys 0.000s
W20260925 09:57:34.059643 10619 leader_election.cc:341] T 10657a81058241fc988d37e5ea7ff8cf P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 10657a81058241fc988d37e5ea7ff8cf
I20260925 09:57:34.063099 11082 raft_consensus.cc:348] T b61444c31dbf4882ac748f861ea66818 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.063779 11082 raft_consensus.cc:374] T b61444c31dbf4882ac748f861ea66818 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:34.063999 11082 raft_consensus.cc:729] T b61444c31dbf4882ac748f861ea66818 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:34.067358 11082 consensus_queue.cc:260] T b61444c31dbf4882ac748f861ea66818 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.073159 11082 ts_tablet_manager.cc:1428] T b61444c31dbf4882ac748f861ea66818 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.014s user 0.005s sys 0.000s
I20260925 09:57:34.073987 11082 tablet_bootstrap.cc:492] T cf775c9d9b114ddcb97dc6c6eca8e0fc P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:34.080394 12257 raft_consensus.cc:482] T b61444c31dbf4882ac748f861ea66818 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:34.080827 12257 raft_consensus.cc:504] T b61444c31dbf4882ac748f861ea66818 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.083937 12257 leader_election.cc:288] T b61444c31dbf4882ac748f861ea66818 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:34.089572 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b61444c31dbf4882ac748f861ea66818" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:34.090044 11082 tablet_bootstrap.cc:654] T cf775c9d9b114ddcb97dc6c6eca8e0fc P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:34.090350 10976 raft_consensus.cc:2454] T b61444c31dbf4882ac748f861ea66818 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:34.093124 10619 leader_election.cc:302] T b61444c31dbf4882ac748f861ea66818 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:34.094070 12257 raft_consensus.cc:2781] T b61444c31dbf4882ac748f861ea66818 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:34.094503 12257 raft_consensus.cc:482] T b61444c31dbf4882ac748f861ea66818 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:34.094924 12257 raft_consensus.cc:3037] T b61444c31dbf4882ac748f861ea66818 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:34.104053 11084 tablet_bootstrap.cc:492] T c9fccada64d1411eb1f75480c8001c49 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:34.104880 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 272) took 5960 ms. Trace:
I20260925 09:57:34.105257 11727 rpcz_store.cc:276] 0925 09:57:28.144533 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:28.167479 (+ 22946us) service_pool.cc:227] Handling call
0925 09:57:33.875371 (+5707892us) ts_tablet_manager.cc:1145] Deleting tablet 37bab60afe7a4c758a08ef7c9c24aba0
0925 09:57:33.875427 (+ 56us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:33.896323 (+ 20896us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:33.914556 (+ 18233us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:34.104816 (+190260us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5707091}
I20260925 09:57:34.109427 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 275) took 5191 ms. Trace:
I20260925 09:57:34.109727 11727 rpcz_store.cc:276] 0925 09:57:28.918472 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:28.918589 (+ 117us) service_pool.cc:227] Handling call
0925 09:57:34.106330 (+5187741us) ts_tablet_manager.cc:1145] Deleting tablet abfc8f390d434c47a8c4990f8406c270
0925 09:57:34.106380 (+ 50us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:34.109373 (+ 2993us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4894525}
I20260925 09:57:34.111660 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 286) took 5166 ms. Trace:
I20260925 09:57:34.111965 11727 rpcz_store.cc:276] 0925 09:57:28.945320 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:28.950000 (+ 4680us) service_pool.cc:227] Handling call
0925 09:57:34.111020 (+5161020us) ts_tablet_manager.cc:1145] Deleting tablet e9faf70be8ac43c79305980d685a28eb
0925 09:57:34.111062 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:34.111608 (+ 546us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4863436}
I20260925 09:57:34.112957 11084 tablet_bootstrap.cc:654] T c9fccada64d1411eb1f75480c8001c49 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:34.113986 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 37bab60afe7a4c758a08ef7c9c24aba0 (table default.Table_8b45818055ec40e386d43e2641ec6ed0 [id=38b800f8e1e948909d40cc976272fdd6]) successfully deleted
W20260925 09:57:34.115525 10377 catalog_manager.cc:4821] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet abfc8f390d434c47a8c4990f8406c270 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet abfc8f390d434c47a8c4990f8406c270 already in progress: creating tablet
W20260925 09:57:34.116400 10377 catalog_manager.cc:4821] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet e9faf70be8ac43c79305980d685a28eb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e9faf70be8ac43c79305980d685a28eb already in progress: creating tablet
I20260925 09:57:34.113023 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 274) took 5194 ms. Trace:
I20260925 09:57:34.126914 11727 rpcz_store.cc:276] 0925 09:57:28.918219 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:28.923683 (+ 5464us) service_pool.cc:227] Handling call
0925 09:57:34.112715 (+5189032us) ts_tablet_manager.cc:1145] Deleting tablet b3d9a3cf5bda4cf6bc1fa3fae1cb9288
0925 09:57:34.112764 (+ 49us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:34.112980 (+ 216us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4864207}
I20260925 09:57:34.128415 11727 tablet_replica.cc:330] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:34.129220 11727 raft_consensus.cc:2229] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:34.129766 12206 consensus_queue.cc:1035] T 6161774f61ae48ab9764a05f2188a80b P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:34.133610 11727 raft_consensus.cc:2258] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 09:57:34.133569 10621 leader_election.cc:341] T 6540aba5854a44419addc5d454e473b7 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:34.134635 10621 leader_election.cc:341] T 2f07dc9548364870923165f4ea524310 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:34.135738 10621 leader_election.cc:341] T 6540aba5854a44419addc5d454e473b7 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:34.137032 10377 catalog_manager.cc:4821] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet b3d9a3cf5bda4cf6bc1fa3fae1cb9288 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b3d9a3cf5bda4cf6bc1fa3fae1cb9288 already in progress: creating tablet
I20260925 09:57:34.043886 12275 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } }
I20260925 09:57:34.058133 10830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10657a81058241fc988d37e5ea7ff8cf" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:34.143092 11084 tablet_bootstrap.cc:492] T c9fccada64d1411eb1f75480c8001c49 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:34.143586 11084 ts_tablet_manager.cc:1397] T c9fccada64d1411eb1f75480c8001c49 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.040s user 0.009s sys 0.001s
I20260925 09:57:34.146514 11082 tablet_bootstrap.cc:492] T cf775c9d9b114ddcb97dc6c6eca8e0fc P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:34.146284 11084 raft_consensus.cc:348] T c9fccada64d1411eb1f75480c8001c49 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.146894 11082 ts_tablet_manager.cc:1397] T cf775c9d9b114ddcb97dc6c6eca8e0fc P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.073s user 0.008s sys 0.000s
I20260925 09:57:34.147146 11084 raft_consensus.cc:374] T c9fccada64d1411eb1f75480c8001c49 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:34.147475 11084 raft_consensus.cc:729] T c9fccada64d1411eb1f75480c8001c49 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:34.148176 11084 consensus_queue.cc:260] T c9fccada64d1411eb1f75480c8001c49 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.148907 11082 raft_consensus.cc:348] T cf775c9d9b114ddcb97dc6c6eca8e0fc P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.149547 11082 raft_consensus.cc:374] T cf775c9d9b114ddcb97dc6c6eca8e0fc P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:34.149796 11082 raft_consensus.cc:729] T cf775c9d9b114ddcb97dc6c6eca8e0fc P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:34.065011 12027 ts_tablet_manager.cc:1918] T c7cd1d6a603642afa2c8c737079961b5 P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:34.150417 12027 log.cc:1196] T c7cd1d6a603642afa2c8c737079961b5 P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/c7cd1d6a603642afa2c8c737079961b5
I20260925 09:57:34.151125 11084 ts_tablet_manager.cc:1428] T c9fccada64d1411eb1f75480c8001c49 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.007s user 0.004s sys 0.001s
I20260925 09:57:34.151355 12027 ts_tablet_manager.cc:1939] T c7cd1d6a603642afa2c8c737079961b5 P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:34.153343 11084 tablet_bootstrap.cc:492] T b39f9e7102ff4a9c96a2cc5d2676f03d P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:34.154146 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 307) took 3645 ms. Trace:
I20260925 09:57:34.156625 12206 consensus_queue.cc:1035] T d5956be0da6441c79723e72f12dfe54a P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 09:57:34.163895 10621 leader_election.cc:341] T a5d872c3620646e5ac33ceb95de2cbd4 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:34.168316 11082 consensus_queue.cc:260] T cf775c9d9b114ddcb97dc6c6eca8e0fc P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.170332 11082 ts_tablet_manager.cc:1428] T cf775c9d9b114ddcb97dc6c6eca8e0fc P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.023s user 0.005s sys 0.000s
I20260925 09:57:34.172227 11727 ts_tablet_manager.cc:1905] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:34.173840 11084 tablet_bootstrap.cc:654] T b39f9e7102ff4a9c96a2cc5d2676f03d P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:34.179418 11082 tablet_bootstrap.cc:492] T 96a80382bb7d4395a14ad9ae972f4a25 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:34.178962 10403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "54c8485a7dcc47a1a75d08ef57647ece" candidate_term: 5 candidate_status { last_received { term: 4 index: 450 } } ignore_live_leader: false dest_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" is_pre_election: true
W20260925 09:57:34.179833 10621 leader_election.cc:341] T 37bab60afe7a4c758a08ef7c9c24aba0 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:34.180600 10621 leader_election.cc:302] T 545ac025f2cb469da847979011a0b3ce P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters: be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:34.181239 12206 raft_consensus.cc:2781] T 545ac025f2cb469da847979011a0b3ce P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:34.181916 12206 raft_consensus.cc:686] T 545ac025f2cb469da847979011a0b3ce P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:34.182793 12206 consensus_queue.cc:237] T 545ac025f2cb469da847979011a0b3ce P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.184404 12275 leader_election.cc:288] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8af3f111e8da4d2dad4e4a98ef0fd5d3 (127.10.9.61:33047), b8be057e0b6d40a3a4b90f2b1f4a0fa2 (127.10.9.60:36985)
W20260925 09:57:34.185434 10304 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.9.60:36985: connect: Connection refused (error 111) [suppressed 13 similar messages]
I20260925 09:57:34.188149 12257 raft_consensus.cc:504] T b61444c31dbf4882ac748f861ea66818 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.191633 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b61444c31dbf4882ac748f861ea66818" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:34.192385 10976 raft_consensus.cc:3037] T b61444c31dbf4882ac748f861ea66818 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:34.198455 11082 tablet_bootstrap.cc:654] T 96a80382bb7d4395a14ad9ae972f4a25 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:34.199848 11084 tablet_bootstrap.cc:492] T b39f9e7102ff4a9c96a2cc5d2676f03d P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:34.204306 11084 ts_tablet_manager.cc:1397] T b39f9e7102ff4a9c96a2cc5d2676f03d P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.051s user 0.003s sys 0.007s
I20260925 09:57:34.200476 10976 raft_consensus.cc:2454] T b61444c31dbf4882ac748f861ea66818 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:34.206661 10619 leader_election.cc:302] T b61444c31dbf4882ac748f861ea66818 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:34.207490 12204 raft_consensus.cc:2781] T b61444c31dbf4882ac748f861ea66818 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:34.207966 12204 raft_consensus.cc:686] T b61444c31dbf4882ac748f861ea66818 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:34.208791 12204 consensus_queue.cc:237] T b61444c31dbf4882ac748f861ea66818 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:34.215896 10621 leader_election.cc:341] T 37bab60afe7a4c758a08ef7c9c24aba0 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:34.220386 10830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b61444c31dbf4882ac748f861ea66818" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:34.154541 12027 rpcz_store.cc:276] 0925 09:57:30.509188 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:30.526546 (+ 17358us) service_pool.cc:227] Handling call
0925 09:57:33.310452 (+2783906us) ts_tablet_manager.cc:1145] Deleting tablet c7cd1d6a603642afa2c8c737079961b5
0925 09:57:33.310511 (+ 59us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:33.853143 (+542632us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:34.064692 (+211549us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:34.153814 (+ 89122us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":33071,"tablet-delete.queue_time_us":2763931}
I20260925 09:57:34.224960 11085 tablet_bootstrap.cc:492] T 7a31b999f0fd447eacff13de71aa3fff P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:34.225401 11085 ts_tablet_manager.cc:1397] T 7a31b999f0fd447eacff13de71aa3fff P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.361s user 0.062s sys 0.055s
I20260925 09:57:34.225955 12027 tablet_replica.cc:330] T a44d9f2f7e3e48aeb0f31fa29ae1988a P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:34.226907 12027 raft_consensus.cc:2229] T a44d9f2f7e3e48aeb0f31fa29ae1988a P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:34.227324 11084 raft_consensus.cc:348] T b39f9e7102ff4a9c96a2cc5d2676f03d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.228076 11084 raft_consensus.cc:374] T b39f9e7102ff4a9c96a2cc5d2676f03d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:34.228343 11084 raft_consensus.cc:729] T b39f9e7102ff4a9c96a2cc5d2676f03d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:34.231546 12257 leader_election.cc:288] T b61444c31dbf4882ac748f861ea66818 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:34.235070 11084 consensus_queue.cc:260] T b39f9e7102ff4a9c96a2cc5d2676f03d P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.237293 11084 ts_tablet_manager.cc:1428] T b39f9e7102ff4a9c96a2cc5d2676f03d P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.033s user 0.004s sys 0.000s
I20260925 09:57:34.238179 11084 tablet_bootstrap.cc:492] T 54157e776efa4c6c8e2bd4c58321fcda P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:34.238624 12251 raft_consensus.cc:482] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b8850818e2ee42a082802d0095c08d21)
I20260925 09:57:34.239241 12251 raft_consensus.cc:504] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.241855 12251 leader_election.cc:288] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
W20260925 09:57:34.243654 10918 leader_election.cc:341] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Not found: Tablet not found: 3ff4cf556eee494d8cd590884466f09f
W20260925 09:57:34.250073 10919 leader_election.cc:341] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Not found: Tablet not found: 3ff4cf556eee494d8cd590884466f09f
I20260925 09:57:34.249904 12205 consensus_queue.cc:1035] T 2f07dc9548364870923165f4ea524310 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:34.250437 10919 leader_election.cc:302] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:34.251669 12251 raft_consensus.cc:2726] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 09:57:34.270898 11084 tablet_bootstrap.cc:654] T 54157e776efa4c6c8e2bd4c58321fcda P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:34.285178 10304 leader_election.cc:334] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b8be057e0b6d40a3a4b90f2b1f4a0fa2 (127.10.9.60:36985): Network error: Client connection negotiation failed: client connection to 127.10.9.60:36985: connect: Connection refused (error 111)
I20260925 09:57:34.286118 10304 leader_election.cc:302] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54c8485a7dcc47a1a75d08ef57647ece; no voters: 8af3f111e8da4d2dad4e4a98ef0fd5d3, b8be057e0b6d40a3a4b90f2b1f4a0fa2
I20260925 09:57:34.292924 12247 ts_tablet_manager.cc:1918] T cfa619b699004882a82976f0ef8204ad P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:34.231940 12123 raft_consensus.cc:482] T 2d087d3e35814c90a03eccd90eeea86c P c43461c4ee854ae1ac52b873ccada318 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:34.302785 12123 raft_consensus.cc:504] T 2d087d3e35814c90a03eccd90eeea86c P c43461c4ee854ae1ac52b873ccada318 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.240190 11085 raft_consensus.cc:348] T 7a31b999f0fd447eacff13de71aa3fff P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.304307 11085 raft_consensus.cc:374] T 7a31b999f0fd447eacff13de71aa3fff P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:34.305042 12123 leader_election.cc:288] T 2d087d3e35814c90a03eccd90eeea86c P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:34.306025 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d087d3e35814c90a03eccd90eeea86c" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:34.306896 10976 raft_consensus.cc:2396] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c43461c4ee854ae1ac52b873ccada318 for term 3 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260925 09:57:34.304586 11085 raft_consensus.cc:729] T 7a31b999f0fd447eacff13de71aa3fff P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:34.309931 11085 consensus_queue.cc:260] T 7a31b999f0fd447eacff13de71aa3fff P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.313117 11085 ts_tablet_manager.cc:1428] T 7a31b999f0fd447eacff13de71aa3fff P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.087s user 0.006s sys 0.001s
I20260925 09:57:34.315896 11085 tablet_bootstrap.cc:492] T 4e3120347c8e4a3eab57d6562ae40e8d P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:34.317440 12027 pending_rounds.cc:62] T a44d9f2f7e3e48aeb0f31fa29ae1988a P c43461c4ee854ae1ac52b873ccada318: Trying to abort 1 pending ops.
I20260925 09:57:34.318853 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b61444c31dbf4882ac748f861ea66818" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:34.321686 12275 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260925 09:57:34.322404 12027 pending_rounds.cc:68] T a44d9f2f7e3e48aeb0f31fa29ae1988a P c43461c4ee854ae1ac52b873ccada318: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192704372707328 op_type: NO_OP noop_request { }
I20260925 09:57:34.327809 12027 raft_consensus.cc:2866] T a44d9f2f7e3e48aeb0f31fa29ae1988a P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:57:34.320076 11082 tablet_bootstrap.cc:492] T 96a80382bb7d4395a14ad9ae972f4a25 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:34.328567 11082 ts_tablet_manager.cc:1397] T 96a80382bb7d4395a14ad9ae972f4a25 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.149s user 0.016s sys 0.012s
I20260925 09:57:34.329697 12247 log.cc:1196] T cfa619b699004882a82976f0ef8204ad P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/cfa619b699004882a82976f0ef8204ad
I20260925 09:57:34.330849 11082 raft_consensus.cc:348] T 96a80382bb7d4395a14ad9ae972f4a25 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.331545 11082 raft_consensus.cc:374] T 96a80382bb7d4395a14ad9ae972f4a25 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:34.331779 11082 raft_consensus.cc:729] T 96a80382bb7d4395a14ad9ae972f4a25 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:34.332213 12247 ts_tablet_manager.cc:1939] T cfa619b699004882a82976f0ef8204ad P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:34.332415 11082 consensus_queue.cc:260] T 96a80382bb7d4395a14ad9ae972f4a25 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.334283 11082 ts_tablet_manager.cc:1428] T 96a80382bb7d4395a14ad9ae972f4a25 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260925 09:57:34.335072 11082 tablet_bootstrap.cc:492] T defaef4d6b754ca5a2dfc21b7a596a5b P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:34.341404 11082 tablet_bootstrap.cc:654] T defaef4d6b754ca5a2dfc21b7a596a5b P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:34.345424 12206 raft_consensus.cc:482] T 8aff53cfd4994c629a408ee8250280c5 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:34.345952 12206 raft_consensus.cc:504] T 8aff53cfd4994c629a408ee8250280c5 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.348233 12206 leader_election.cc:288] T 8aff53cfd4994c629a408ee8250280c5 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:34.349311 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aff53cfd4994c629a408ee8250280c5" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:34.349452 12027 raft_consensus.cc:2258] T a44d9f2f7e3e48aeb0f31fa29ae1988a P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:34.349905 10976 raft_consensus.cc:2454] T 8aff53cfd4994c629a408ee8250280c5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:34.275079 11727 ts_tablet_manager.cc:1918] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:57:34.351369 10619 leader_election.cc:302] T 8aff53cfd4994c629a408ee8250280c5 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:34.351608 11727 log.cc:1196] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/b555eeba505740988c704df5c734e7c0
I20260925 09:57:34.352900 12080 raft_consensus.cc:2781] T 8aff53cfd4994c629a408ee8250280c5 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:34.353221 12080 raft_consensus.cc:482] T 8aff53cfd4994c629a408ee8250280c5 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:34.353557 12080 raft_consensus.cc:3037] T 8aff53cfd4994c629a408ee8250280c5 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:34.356355 11727 ts_tablet_manager.cc:1939] T b555eeba505740988c704df5c734e7c0 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:34.362142 12080 raft_consensus.cc:504] T 8aff53cfd4994c629a408ee8250280c5 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.363222 12251 raft_consensus.cc:482] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:34.363734 12251 raft_consensus.cc:504] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.364509 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 292) took 5289 ms. Trace:
I20260925 09:57:34.365643 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aff53cfd4994c629a408ee8250280c5" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:34.366210 10976 raft_consensus.cc:3037] T 8aff53cfd4994c629a408ee8250280c5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:34.366741 10830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17bd30416fe14c0e9ab2e0c0bf5f2fb5" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:34.367089 10679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17bd30416fe14c0e9ab2e0c0bf5f2fb5" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:34.368036 10918 leader_election.cc:341] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:34.368494 10919 leader_election.cc:341] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:34.368896 10919 leader_election.cc:302] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:34.369643 12207 raft_consensus.cc:2726] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:34.368809 11082 tablet_bootstrap.cc:492] T defaef4d6b754ca5a2dfc21b7a596a5b P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:34.370311 12251 leader_election.cc:288] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:34.371235 11082 ts_tablet_manager.cc:1397] T defaef4d6b754ca5a2dfc21b7a596a5b P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.036s user 0.010s sys 0.001s
I20260925 09:57:34.375252 10976 raft_consensus.cc:2454] T 8aff53cfd4994c629a408ee8250280c5 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:34.364857 11727 rpcz_store.cc:276] 0925 09:57:29.075222 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:29.202096 (+126874us) service_pool.cc:227] Handling call
0925 09:57:34.128266 (+4926170us) ts_tablet_manager.cc:1145] Deleting tablet b555eeba505740988c704df5c734e7c0
0925 09:57:34.128309 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:34.245724 (+117415us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:34.274848 (+ 29124us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:34.364448 (+ 89600us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4787921}
I20260925 09:57:34.376441 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet b555eeba505740988c704df5c734e7c0 (table default.Table_48f368d811494119be0dc534e2fdcfbd [id=3e32026d175b4b92a3ee036f17e54b9d]) successfully deleted
I20260925 09:57:34.376477 10619 leader_election.cc:302] T 8aff53cfd4994c629a408ee8250280c5 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:34.376722 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 297) took 4849 ms. Trace:
I20260925 09:57:34.377205 12080 leader_election.cc:288] T 8aff53cfd4994c629a408ee8250280c5 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:34.377004 11727 rpcz_store.cc:276] 0925 09:57:29.527327 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:29.527481 (+ 154us) service_pool.cc:227] Handling call
0925 09:57:34.376449 (+4848968us) ts_tablet_manager.cc:1145] Deleting tablet 3706253e53d841aeaa43889ead13af32
0925 09:57:34.376499 (+ 50us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:34.376668 (+ 169us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4544020}
W20260925 09:57:34.378090 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 3706253e53d841aeaa43889ead13af32 because the tablet was not found. No further retry: Not found: Tablet not found: 3706253e53d841aeaa43889ead13af32
I20260925 09:57:34.381134 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet c7cd1d6a603642afa2c8c737079961b5 (table default.Table_0507f47042b9445abaf8671c979f8994 [id=55860c5dba9f4c818cfa6d523041ec5f]) successfully deleted
I20260925 09:57:34.381309 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 298) took 4848 ms. Trace:
I20260925 09:57:34.381644 11727 rpcz_store.cc:276] 0925 09:57:29.532564 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:29.532686 (+ 122us) service_pool.cc:227] Handling call
0925 09:57:34.380995 (+4848309us) ts_tablet_manager.cc:1145] Deleting tablet 28d60531970a4ab58ce5ef99cb3a96bf
0925 09:57:34.381039 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:34.381201 (+ 162us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4501277}
W20260925 09:57:34.382680 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 28d60531970a4ab58ce5ef99cb3a96bf because the tablet was not found. No further retry: Not found: Tablet not found: 28d60531970a4ab58ce5ef99cb3a96bf
I20260925 09:57:34.383956 11084 tablet_bootstrap.cc:492] T 54157e776efa4c6c8e2bd4c58321fcda P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:34.384410 11084 ts_tablet_manager.cc:1397] T 54157e776efa4c6c8e2bd4c58321fcda P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.146s user 0.021s sys 0.000s
W20260925 09:57:34.362694 10621 leader_election.cc:341] T 10657a81058241fc988d37e5ea7ff8cf P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:34.387763 10621 leader_election.cc:302] T 10657a81058241fc988d37e5ea7ff8cf P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8850818e2ee42a082802d0095c08d21; no voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:34.387013 11084 raft_consensus.cc:348] T 54157e776efa4c6c8e2bd4c58321fcda P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.389282 11084 raft_consensus.cc:374] T 54157e776efa4c6c8e2bd4c58321fcda P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:34.391579 11084 raft_consensus.cc:729] T 54157e776efa4c6c8e2bd4c58321fcda P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:34.392287 11084 consensus_queue.cc:260] T 54157e776efa4c6c8e2bd4c58321fcda P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.394423 11084 ts_tablet_manager.cc:1428] T 54157e776efa4c6c8e2bd4c58321fcda P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20260925 09:57:34.397109 12257 raft_consensus.cc:2781] T 8aff53cfd4994c629a408ee8250280c5 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:34.402045 11082 raft_consensus.cc:348] T defaef4d6b754ca5a2dfc21b7a596a5b P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.402751 11082 raft_consensus.cc:374] T defaef4d6b754ca5a2dfc21b7a596a5b P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:34.403008 11082 raft_consensus.cc:729] T defaef4d6b754ca5a2dfc21b7a596a5b P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:34.407744 11082 consensus_queue.cc:260] T defaef4d6b754ca5a2dfc21b7a596a5b P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:34.408911 10621 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 46253.4us
I20260925 09:57:34.340566 11085 tablet_bootstrap.cc:654] T 4e3120347c8e4a3eab57d6562ae40e8d P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:34.416522 12257 raft_consensus.cc:686] T 8aff53cfd4994c629a408ee8250280c5 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:34.416415 12205 consensus_queue.cc:1035] T 6540aba5854a44419addc5d454e473b7 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:34.417826 12257 consensus_queue.cc:237] T 8aff53cfd4994c629a408ee8250280c5 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.423216 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet cfa619b699004882a82976f0ef8204ad (table default.Table_7e0155a70cc54485b8fdd4d936460d1c [id=c90c671ed4f043c2b56a1f84ff9a42cd]) successfully deleted
I20260925 09:57:34.430517 12323 raft_consensus.cc:2726] T 10657a81058241fc988d37e5ea7ff8cf P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:34.398865 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 299) took 4864 ms. Trace:
I20260925 09:57:34.435655 11727 rpcz_store.cc:276] 0925 09:57:29.534810 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:29.534931 (+ 121us) service_pool.cc:227] Handling call
0925 09:57:34.388891 (+4853960us) ts_tablet_manager.cc:1145] Deleting tablet 7a31b999f0fd447eacff13de71aa3fff
0925 09:57:34.388928 (+ 37us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:34.398815 (+ 9887us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4508989}
W20260925 09:57:34.436975 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 7a31b999f0fd447eacff13de71aa3fff because the tablet was not found. No further retry: Not found: Tablet not found: 7a31b999f0fd447eacff13de71aa3fff
I20260925 09:57:34.437995 11082 ts_tablet_manager.cc:1428] T defaef4d6b754ca5a2dfc21b7a596a5b P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.056s user 0.005s sys 0.000s
I20260925 09:57:34.439006 11082 tablet_bootstrap.cc:492] T ab6bdba5ca5d41ae921a90c1f77006c5 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
W20260925 09:57:34.438041 10621 leader_election.cc:341] T b61444c31dbf4882ac748f861ea66818 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:34.442184 10621 leader_election.cc:341] T b61444c31dbf4882ac748f861ea66818 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:34.443886 10621 consensus_peers.cc:487] T 2f07dc9548364870923165f4ea524310 P b8850818e2ee42a082802d0095c08d21 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:34.445753 11084 tablet_bootstrap.cc:492] T f9c259867cf44c4c90e50864475a4970 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:34.447152 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 300) took 4844 ms. Trace:
I20260925 09:57:34.447415 12027 ts_tablet_manager.cc:1905] T a44d9f2f7e3e48aeb0f31fa29ae1988a P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:34.447571 11727 rpcz_store.cc:276] 0925 09:57:29.602296 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:29.602407 (+ 111us) service_pool.cc:227] Handling call
0925 09:57:34.437143 (+4834736us) ts_tablet_manager.cc:1145] Deleting tablet 977a59175e5b43eeb6f23b938afd00a3
0925 09:57:34.446846 (+ 9703us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:34.447074 (+ 228us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4557125}
I20260925 09:57:34.448762 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 301) took 4843 ms. Trace:
I20260925 09:57:34.449065 11727 rpcz_store.cc:276] 0925 09:57:29.605232 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:29.605339 (+ 107us) service_pool.cc:227] Handling call
0925 09:57:34.448485 (+4843146us) ts_tablet_manager.cc:1145] Deleting tablet 359090244c4748e8a2d45e3c77e89498
0925 09:57:34.448528 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:34.448698 (+ 170us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4568378}
I20260925 09:57:34.450035 11727 tablet_replica.cc:330] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:34.451012 11727 raft_consensus.cc:2229] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260925 09:57:34.452157 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 977a59175e5b43eeb6f23b938afd00a3 because the tablet was not found. No further retry: Not found: Tablet not found: 977a59175e5b43eeb6f23b938afd00a3
W20260925 09:57:34.452800 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 359090244c4748e8a2d45e3c77e89498 because the tablet was not found. No further retry: Not found: Tablet not found: 359090244c4748e8a2d45e3c77e89498
I20260925 09:57:34.454687 10668 tablet_service.cc:1467] Processing CreateTablet for tablet aab6d06909084ff8b26c7fd8d8ef5d9d (DEFAULT_TABLE table=default.Table_ccbf8d9f16cb4fc5a5cca3d3f809b1e7 [id=c8a336640e37452da4eb17059f613fc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:34.455782 10668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aab6d06909084ff8b26c7fd8d8ef5d9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:34.456640 10950 tablet_service.cc:1467] Processing CreateTablet for tablet aab6d06909084ff8b26c7fd8d8ef5d9d (DEFAULT_TABLE table=default.Table_ccbf8d9f16cb4fc5a5cca3d3f809b1e7 [id=c8a336640e37452da4eb17059f613fc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:34.457676 10950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aab6d06909084ff8b26c7fd8d8ef5d9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:34.459592 11727 raft_consensus.cc:2258] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:34.461526 10667 tablet_service.cc:1467] Processing CreateTablet for tablet ca608f40d4b94f3e8cd2ec501b345186 (DEFAULT_TABLE table=default.Table_bb7c3c53b9c44306a98ae97e22140782 [id=2a1848d48733462bb865d81fae8be6d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:34.462478 10667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca608f40d4b94f3e8cd2ec501b345186. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:34.467797 12207 raft_consensus.cc:482] T d54ab15dd054489ab50b7f0b26b79a50 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:34.468200 12207 raft_consensus.cc:504] T d54ab15dd054489ab50b7f0b26b79a50 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.470095 12207 leader_election.cc:288] T d54ab15dd054489ab50b7f0b26b79a50 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:34.471707 10681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d54ab15dd054489ab50b7f0b26b79a50" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:34.473079 10830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d54ab15dd054489ab50b7f0b26b79a50" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:34.474107 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aff53cfd4994c629a408ee8250280c5" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:34.474365 10919 leader_election.cc:341] T d54ab15dd054489ab50b7f0b26b79a50 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:34.475186 10918 leader_election.cc:341] T d54ab15dd054489ab50b7f0b26b79a50 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:34.475539 10918 leader_election.cc:302] T d54ab15dd054489ab50b7f0b26b79a50 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:34.476142 11727 ts_tablet_manager.cc:1905] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:34.477310 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aff53cfd4994c629a408ee8250280c5" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:34.478637 10655 tablet_service.cc:1467] Processing CreateTablet for tablet 680f0759518047d09e31628356b650a0 (DEFAULT_TABLE table=default.Table_5767a6fb7263485fa31677b488d9f903 [id=42d7af973b9142e5be5a294765928eee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:34.479609 10655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 680f0759518047d09e31628356b650a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:34.480859 10654 tablet_service.cc:1467] Processing CreateTablet for tablet 963e5bf2ac2848b4967074d28acdae52 (DEFAULT_TABLE table=default.Table_a2a96456c8804d1692ffbafab832340b [id=5dc3fd1666fb4431b1c3bbf27b02ba54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:34.481861 10654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 963e5bf2ac2848b4967074d28acdae52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:34.475915 10944 tablet_service.cc:1467] Processing CreateTablet for tablet ca608f40d4b94f3e8cd2ec501b345186 (DEFAULT_TABLE table=default.Table_bb7c3c53b9c44306a98ae97e22140782 [id=2a1848d48733462bb865d81fae8be6d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:34.488296 10944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca608f40d4b94f3e8cd2ec501b345186. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:34.476400 12207 raft_consensus.cc:2726] T d54ab15dd054489ab50b7f0b26b79a50 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:34.478518 10946 tablet_service.cc:1467] Processing CreateTablet for tablet 680f0759518047d09e31628356b650a0 (DEFAULT_TABLE table=default.Table_5767a6fb7263485fa31677b488d9f903 [id=42d7af973b9142e5be5a294765928eee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:34.492434 10946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 680f0759518047d09e31628356b650a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:34.497217 10810 tablet_service.cc:1467] Processing CreateTablet for tablet aab6d06909084ff8b26c7fd8d8ef5d9d (DEFAULT_TABLE table=default.Table_ccbf8d9f16cb4fc5a5cca3d3f809b1e7 [id=c8a336640e37452da4eb17059f613fc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:34.498148 10810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aab6d06909084ff8b26c7fd8d8ef5d9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:34.500828 11084 tablet_bootstrap.cc:654] T f9c259867cf44c4c90e50864475a4970 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:34.479526 12251 raft_consensus.cc:482] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b8850818e2ee42a082802d0095c08d21)
I20260925 09:57:34.482199 10952 tablet_service.cc:1467] Processing CreateTablet for tablet 963e5bf2ac2848b4967074d28acdae52 (DEFAULT_TABLE table=default.Table_a2a96456c8804d1692ffbafab832340b [id=5dc3fd1666fb4431b1c3bbf27b02ba54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:34.503670 12251 raft_consensus.cc:504] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.504227 10952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 963e5bf2ac2848b4967074d28acdae52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:34.507685 12251 leader_election.cc:288] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
W20260925 09:57:34.512553 10918 leader_election.cc:341] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Not found: Tablet not found: c7cd1d6a603642afa2c8c737079961b5
I20260925 09:57:34.521020 11085 tablet_bootstrap.cc:492] T 4e3120347c8e4a3eab57d6562ae40e8d P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:34.528112 11085 ts_tablet_manager.cc:1397] T 4e3120347c8e4a3eab57d6562ae40e8d P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.212s user 0.010s sys 0.000s
W20260925 09:57:34.530589 10919 leader_election.cc:341] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Not found: Tablet not found: c7cd1d6a603642afa2c8c737079961b5
I20260925 09:57:34.530995 10919 leader_election.cc:302] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:34.533581 12251 raft_consensus.cc:2726] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 09:57:34.540375 10809 tablet_service.cc:1467] Processing CreateTablet for tablet ca608f40d4b94f3e8cd2ec501b345186 (DEFAULT_TABLE table=default.Table_bb7c3c53b9c44306a98ae97e22140782 [id=2a1848d48733462bb865d81fae8be6d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:34.541792 10809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca608f40d4b94f3e8cd2ec501b345186. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:34.544719 10956 tablet_service.cc:1467] Processing CreateTablet for tablet 61ffa0bde20a44a1a68b1a37de0650c2 (DEFAULT_TABLE table=default.Table_2c997080e80a44fba286d0881ef12788 [id=ee27d63855fe42e8b9748da6c54b22b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:34.544787 10665 tablet_service.cc:1467] Processing CreateTablet for tablet 080c76ce032848c4af26ec5c8566f300 (DEFAULT_TABLE table=default.Table_bb742c618d96453aaabe897d54fc657b [id=68683893b91c4f0190a0050ecdc6d396]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:34.545151 11082 tablet_bootstrap.cc:654] T ab6bdba5ca5d41ae921a90c1f77006c5 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:34.545825 10956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61ffa0bde20a44a1a68b1a37de0650c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:34.545926 10665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 080c76ce032848c4af26ec5c8566f300. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:34.547325 10813 tablet_service.cc:1467] Processing CreateTablet for tablet 680f0759518047d09e31628356b650a0 (DEFAULT_TABLE table=default.Table_5767a6fb7263485fa31677b488d9f903 [id=42d7af973b9142e5be5a294765928eee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:34.548450 10813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 680f0759518047d09e31628356b650a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:34.552572 10819 tablet_service.cc:1467] Processing CreateTablet for tablet 963e5bf2ac2848b4967074d28acdae52 (DEFAULT_TABLE table=default.Table_a2a96456c8804d1692ffbafab832340b [id=5dc3fd1666fb4431b1c3bbf27b02ba54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:34.553536 10819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 963e5bf2ac2848b4967074d28acdae52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:34.556808 11085 raft_consensus.cc:348] T 4e3120347c8e4a3eab57d6562ae40e8d P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.557466 11085 raft_consensus.cc:374] T 4e3120347c8e4a3eab57d6562ae40e8d P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:34.557705 11085 raft_consensus.cc:729] T 4e3120347c8e4a3eab57d6562ae40e8d P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:34.558328 11085 consensus_queue.cc:260] T 4e3120347c8e4a3eab57d6562ae40e8d P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.572080 10815 tablet_service.cc:1467] Processing CreateTablet for tablet 61ffa0bde20a44a1a68b1a37de0650c2 (DEFAULT_TABLE table=default.Table_2c997080e80a44fba286d0881ef12788 [id=ee27d63855fe42e8b9748da6c54b22b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:34.573817 10815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61ffa0bde20a44a1a68b1a37de0650c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:34.574793 11084 tablet_bootstrap.cc:492] T f9c259867cf44c4c90e50864475a4970 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:34.575205 11084 ts_tablet_manager.cc:1397] T f9c259867cf44c4c90e50864475a4970 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.130s user 0.009s sys 0.005s
I20260925 09:57:34.577312 11084 raft_consensus.cc:348] T f9c259867cf44c4c90e50864475a4970 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.577944 11084 raft_consensus.cc:374] T f9c259867cf44c4c90e50864475a4970 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:34.578179 11084 raft_consensus.cc:729] T f9c259867cf44c4c90e50864475a4970 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:34.578763 11084 consensus_queue.cc:260] T f9c259867cf44c4c90e50864475a4970 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.581337 11085 ts_tablet_manager.cc:1428] T 4e3120347c8e4a3eab57d6562ae40e8d P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.053s user 0.003s sys 0.001s
I20260925 09:57:34.587582 10655 tablet_service.cc:1467] Processing CreateTablet for tablet 420356ce547346b9a85306e4192a498b (DEFAULT_TABLE table=default.Table_e378bf3dbec04ead88c254701fea06fc [id=a86054144084421d8b7f3fbd34842da5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:34.587790 10814 tablet_service.cc:1467] Processing CreateTablet for tablet 080c76ce032848c4af26ec5c8566f300 (DEFAULT_TABLE table=default.Table_bb742c618d96453aaabe897d54fc657b [id=68683893b91c4f0190a0050ecdc6d396]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:34.588588 10655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 420356ce547346b9a85306e4192a498b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:34.588662 10814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 080c76ce032848c4af26ec5c8566f300. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:34.592062 10812 tablet_service.cc:1467] Processing CreateTablet for tablet 420356ce547346b9a85306e4192a498b (DEFAULT_TABLE table=default.Table_e378bf3dbec04ead88c254701fea06fc [id=a86054144084421d8b7f3fbd34842da5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:34.593058 10812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 420356ce547346b9a85306e4192a498b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:34.585834 10377 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet b025f8ba34694ba09cb01ca959d3629c (table default.Table_2702858b51514e7f9a51a2bf19d36b1f [id=65b6e9ec3f114cea85e70f1509e3d1b6], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet b025f8ba34694ba09cb01ca959d3629c consensus state has no leader
I20260925 09:57:34.596645 11085 tablet_bootstrap.cc:492] T 2f07dc9548364870923165f4ea524310 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:34.543377 10388 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 6540aba5854a44419addc5d454e473b7 (table default.Table_8180842f664042b0a9073b7008438ff7 [id=416a4b989570457e919fb7e7b16a017d]). Expected version 1 got 0
I20260925 09:57:34.600795 10388 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 3d652a76d96f4dbeafde15f3c16534a6 (table default.Table_be0f658aa41f4188bc16db37e96f4ad1 [id=73083c1560cb4db5a12964387cb49b73]). Expected version 1 got 0
I20260925 09:57:34.586715 10948 tablet_service.cc:1467] Processing CreateTablet for tablet 080c76ce032848c4af26ec5c8566f300 (DEFAULT_TABLE table=default.Table_bb742c618d96453aaabe897d54fc657b [id=68683893b91c4f0190a0050ecdc6d396]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:34.609025 11084 ts_tablet_manager.cc:1428] T f9c259867cf44c4c90e50864475a4970 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.034s user 0.004s sys 0.000s
I20260925 09:57:34.611093 10388 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet f62c62715c9b42de826c1a66fd431eab (table default.Table_bcbc82519cf043738f61029181f29418 [id=b3f7f21ab2214e4cb637bea27ce62abe]). Expected version 1 got 0
I20260925 09:57:34.610152 10947 tablet_service.cc:1467] Processing CreateTablet for tablet 420356ce547346b9a85306e4192a498b (DEFAULT_TABLE table=default.Table_e378bf3dbec04ead88c254701fea06fc [id=a86054144084421d8b7f3fbd34842da5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:34.612756 10806 tablet_service.cc:1467] Processing CreateTablet for tablet cf7b2d3f3b3949a0b81283bc68e7b131 (DEFAULT_TABLE table=default.Table_835c3ec6bfb943ceac4d44f0735bb91c [id=e499d19b87184375b75fe2fb263d92b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:34.613760 10806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf7b2d3f3b3949a0b81283bc68e7b131. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:34.619810 10388 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 5608c5f9b65b4da7b2fbb56417c92d88 (table default.Table_f37bcee5db2044f2bea9c353982b30cd [id=e46d114e5d484b66bb15eb0416b5ce0d]). Expected version 2 got 0
I20260925 09:57:34.622378 11082 tablet_bootstrap.cc:492] T ab6bdba5ca5d41ae921a90c1f77006c5 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:34.622789 11082 ts_tablet_manager.cc:1397] T ab6bdba5ca5d41ae921a90c1f77006c5 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.184s user 0.016s sys 0.004s
I20260925 09:57:34.623323 10667 tablet_service.cc:1467] Processing CreateTablet for tablet cf7b2d3f3b3949a0b81283bc68e7b131 (DEFAULT_TABLE table=default.Table_835c3ec6bfb943ceac4d44f0735bb91c [id=e499d19b87184375b75fe2fb263d92b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:34.624333 10667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf7b2d3f3b3949a0b81283bc68e7b131. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:34.624223 10388 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 2f07dc9548364870923165f4ea524310 (table default.Table_0db927e92a3b4970a7efea3fb765e65e [id=a988264fe43c4eb88a84f8b6359e77a6]). Expected version 1 got 0
I20260925 09:57:34.625389 10388 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet e684798155d445bfb325a1b7c5c48753 (table default.Table_dfb73f4afe7c464d8dc98356500054e2 [id=ab3c143dc00c43e78531514a114bfd16]). Expected version 1 got 0
I20260925 09:57:34.627381 10948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 080c76ce032848c4af26ec5c8566f300. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:34.627972 10947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 420356ce547346b9a85306e4192a498b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:34.628557 11084 tablet_bootstrap.cc:492] T 8a3149fb2325435699af36338e77d042 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:34.636245 10668 tablet_service.cc:1467] Processing CreateTablet for tablet 61ffa0bde20a44a1a68b1a37de0650c2 (DEFAULT_TABLE table=default.Table_2c997080e80a44fba286d0881ef12788 [id=ee27d63855fe42e8b9748da6c54b22b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:34.637228 10668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61ffa0bde20a44a1a68b1a37de0650c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:34.640534 11084 tablet_bootstrap.cc:654] T 8a3149fb2325435699af36338e77d042 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:34.642690 10775 leader_election.cc:341] T 2d087d3e35814c90a03eccd90eeea86c P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Not found: Tablet not found: 2d087d3e35814c90a03eccd90eeea86c
I20260925 09:57:34.643064 10775 leader_election.cc:302] T 2d087d3e35814c90a03eccd90eeea86c P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:34.644140 12209 raft_consensus.cc:2726] T 2d087d3e35814c90a03eccd90eeea86c P c43461c4ee854ae1ac52b873ccada318 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260925 09:57:34.648031 10810 tablet_service.cc:1467] Processing CreateTablet for tablet eaf431bc73a54898a7f6f411990bfc0a (DEFAULT_TABLE table=default.Table_acebd27bd1c54fcd8668a2fac337ebdd [id=3ac229f940da4d56b43337b114ee3435]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:34.649060 10810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eaf431bc73a54898a7f6f411990bfc0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:34.649686 10391 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet e684798155d445bfb325a1b7c5c48753 (table default.Table_dfb73f4afe7c464d8dc98356500054e2 [id=ab3c143dc00c43e78531514a114bfd16]). Expected version 1 got 0
I20260925 09:57:34.650652 10391 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 5608c5f9b65b4da7b2fbb56417c92d88 (table default.Table_f37bcee5db2044f2bea9c353982b30cd [id=e46d114e5d484b66bb15eb0416b5ce0d]). Expected version 2 got 0
I20260925 09:57:34.660086 10817 tablet_service.cc:1467] Processing CreateTablet for tablet 7765f477485a4c16a071594c495e9ecb (DEFAULT_TABLE table=default.Table_7678491b0b014f7fa63359e06ffbcd3e [id=5c947e635b0c4c30a00588ca8bdb69e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:34.661180 10817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7765f477485a4c16a071594c495e9ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:34.673818 12206 raft_consensus.cc:482] T defaef4d6b754ca5a2dfc21b7a596a5b P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:34.674311 12206 raft_consensus.cc:504] T defaef4d6b754ca5a2dfc21b7a596a5b P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.675580 12209 raft_consensus.cc:482] T 7a31b999f0fd447eacff13de71aa3fff P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:34.676043 12209 raft_consensus.cc:504] T 7a31b999f0fd447eacff13de71aa3fff P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.676744 12206 leader_election.cc:288] T defaef4d6b754ca5a2dfc21b7a596a5b P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:34.678797 12209 leader_election.cc:288] T 7a31b999f0fd447eacff13de71aa3fff P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:34.680876 11082 raft_consensus.cc:348] T ab6bdba5ca5d41ae921a90c1f77006c5 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.681536 11082 raft_consensus.cc:374] T ab6bdba5ca5d41ae921a90c1f77006c5 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:34.681799 11082 raft_consensus.cc:729] T ab6bdba5ca5d41ae921a90c1f77006c5 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:34.682710 11082 consensus_queue.cc:260] T ab6bdba5ca5d41ae921a90c1f77006c5 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.684551 10655 tablet_service.cc:1467] Processing CreateTablet for tablet eaf431bc73a54898a7f6f411990bfc0a (DEFAULT_TABLE table=default.Table_acebd27bd1c54fcd8668a2fac337ebdd [id=3ac229f940da4d56b43337b114ee3435]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:34.685657 10655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eaf431bc73a54898a7f6f411990bfc0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:34.694085 10653 tablet_service.cc:1467] Processing CreateTablet for tablet 7765f477485a4c16a071594c495e9ecb (DEFAULT_TABLE table=default.Table_7678491b0b014f7fa63359e06ffbcd3e [id=5c947e635b0c4c30a00588ca8bdb69e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:34.695344 10653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7765f477485a4c16a071594c495e9ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:34.702075 10386 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 2f07dc9548364870923165f4ea524310 (table default.Table_0db927e92a3b4970a7efea3fb765e65e [id=a988264fe43c4eb88a84f8b6359e77a6]). Expected version 1 got 0
I20260925 09:57:34.706391 11084 tablet_bootstrap.cc:492] T 8a3149fb2325435699af36338e77d042 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:34.706809 11084 ts_tablet_manager.cc:1397] T 8a3149fb2325435699af36338e77d042 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.079s user 0.010s sys 0.000s
I20260925 09:57:34.709307 11084 raft_consensus.cc:348] T 8a3149fb2325435699af36338e77d042 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.709668 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a31b999f0fd447eacff13de71aa3fff" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:34.710070 11084 raft_consensus.cc:374] T 8a3149fb2325435699af36338e77d042 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:34.710304 11084 raft_consensus.cc:729] T 8a3149fb2325435699af36338e77d042 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:34.710225 10684 raft_consensus.cc:2454] T 7a31b999f0fd447eacff13de71aa3fff P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
W20260925 09:57:34.711056 10621 leader_election.cc:341] T 8aff53cfd4994c629a408ee8250280c5 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:34.711041 11084 consensus_queue.cc:260] T 8a3149fb2325435699af36338e77d042 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:34.712044 10621 leader_election.cc:341] T 8aff53cfd4994c629a408ee8250280c5 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:34.713047 11082 ts_tablet_manager.cc:1428] T ab6bdba5ca5d41ae921a90c1f77006c5 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.090s user 0.001s sys 0.004s
I20260925 09:57:34.716750 10386 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 5608c5f9b65b4da7b2fbb56417c92d88 (table default.Table_f37bcee5db2044f2bea9c353982b30cd [id=e46d114e5d484b66bb15eb0416b5ce0d]). Expected version 2 got 0
W20260925 09:57:34.717185 10621 consensus_peers.cc:487] T a5d872c3620646e5ac33ceb95de2cbd4 P b8850818e2ee42a082802d0095c08d21 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:34.717605 11082 tablet_bootstrap.cc:492] T 92814437d53143a79f99199f8ee3a742 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:34.719305 11084 ts_tablet_manager.cc:1428] T 8a3149fb2325435699af36338e77d042 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.012s user 0.004s sys 0.000s
I20260925 09:57:34.720152 11084 tablet_bootstrap.cc:492] T abfc8f390d434c47a8c4990f8406c270 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:34.727880 10386 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet e684798155d445bfb325a1b7c5c48753 (table default.Table_dfb73f4afe7c464d8dc98356500054e2 [id=ab3c143dc00c43e78531514a114bfd16]). Expected version 1 got 0
W20260925 09:57:34.729599 10621 consensus_peers.cc:487] T 6540aba5854a44419addc5d454e473b7 P b8850818e2ee42a082802d0095c08d21 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:34.730444 10386 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 6540aba5854a44419addc5d454e473b7 (table default.Table_8180842f664042b0a9073b7008438ff7 [id=416a4b989570457e919fb7e7b16a017d]). Expected version 1 got 0
I20260925 09:57:34.731654 10386 catalog_manager.cc:5495] T d5956be0da6441c79723e72f12dfe54a P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
I20260925 09:57:34.734923 12123 raft_consensus.cc:482] T 4e3120347c8e4a3eab57d6562ae40e8d P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:34.743942 12123 raft_consensus.cc:504] T 4e3120347c8e4a3eab57d6562ae40e8d P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.745909 12123 leader_election.cc:288] T 4e3120347c8e4a3eab57d6562ae40e8d P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:34.748463 10830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "defaef4d6b754ca5a2dfc21b7a596a5b" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:34.759133 11082 tablet_bootstrap.cc:654] T 92814437d53143a79f99199f8ee3a742 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:34.765069 12348 tablet.cc:1722] T 5608c5f9b65b4da7b2fbb56417c92d88 P b8850818e2ee42a082802d0095c08d21: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260925 09:57:34.768743 10621 leader_election.cc:341] T defaef4d6b754ca5a2dfc21b7a596a5b P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:34.788183 11082 tablet_bootstrap.cc:492] T 92814437d53143a79f99199f8ee3a742 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:34.789211 11082 ts_tablet_manager.cc:1397] T 92814437d53143a79f99199f8ee3a742 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.074s user 0.010s sys 0.000s
I20260925 09:57:34.803937 10956 tablet_service.cc:1467] Processing CreateTablet for tablet cf7b2d3f3b3949a0b81283bc68e7b131 (DEFAULT_TABLE table=default.Table_835c3ec6bfb943ceac4d44f0735bb91c [id=e499d19b87184375b75fe2fb263d92b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:34.804217 10950 tablet_service.cc:1467] Processing CreateTablet for tablet 7765f477485a4c16a071594c495e9ecb (DEFAULT_TABLE table=default.Table_7678491b0b014f7fa63359e06ffbcd3e [id=5c947e635b0c4c30a00588ca8bdb69e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:34.804250 10973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "defaef4d6b754ca5a2dfc21b7a596a5b" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:34.806290 10956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf7b2d3f3b3949a0b81283bc68e7b131. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:34.806221 10973 raft_consensus.cc:2454] T defaef4d6b754ca5a2dfc21b7a596a5b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:34.808445 10952 tablet_service.cc:1467] Processing CreateTablet for tablet eaf431bc73a54898a7f6f411990bfc0a (DEFAULT_TABLE table=default.Table_acebd27bd1c54fcd8668a2fac337ebdd [id=3ac229f940da4d56b43337b114ee3435]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:34.810024 10952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eaf431bc73a54898a7f6f411990bfc0a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:34.812235 10772 leader_election.cc:341] T 4e3120347c8e4a3eab57d6562ae40e8d P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 4e3120347c8e4a3eab57d6562ae40e8d
I20260925 09:57:34.815213 10950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7765f477485a4c16a071594c495e9ecb. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:34.824543 10772 leader_election.cc:341] T 7a31b999f0fd447eacff13de71aa3fff P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 7a31b999f0fd447eacff13de71aa3fff
I20260925 09:57:34.825919 12027 ts_tablet_manager.cc:1918] T a44d9f2f7e3e48aeb0f31fa29ae1988a P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:34.827051 12206 consensus_queue.cc:1035] T a5d872c3620646e5ac33ceb95de2cbd4 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:34.829442 10619 leader_election.cc:302] T defaef4d6b754ca5a2dfc21b7a596a5b P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:34.830646 12206 raft_consensus.cc:2781] T defaef4d6b754ca5a2dfc21b7a596a5b P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:34.831029 12206 raft_consensus.cc:482] T defaef4d6b754ca5a2dfc21b7a596a5b P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:34.831489 12206 raft_consensus.cc:3037] T defaef4d6b754ca5a2dfc21b7a596a5b P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:34.838903 12027 log.cc:1196] T a44d9f2f7e3e48aeb0f31fa29ae1988a P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/a44d9f2f7e3e48aeb0f31fa29ae1988a
I20260925 09:57:34.844776 11084 tablet_bootstrap.cc:654] T abfc8f390d434c47a8c4990f8406c270 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:34.856415 12251 raft_consensus.cc:482] T 61aa6380d30a4b21b70ee2d321247bd8 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:34.856882 12251 raft_consensus.cc:504] T 61aa6380d30a4b21b70ee2d321247bd8 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.859040 12251 leader_election.cc:288] T 61aa6380d30a4b21b70ee2d321247bd8 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:34.869436 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61aa6380d30a4b21b70ee2d321247bd8" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:34.869679 11727 ts_tablet_manager.cc:1918] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:34.870116 11727 log.cc:1196] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/3ff4cf556eee494d8cd590884466f09f
I20260925 09:57:34.870635 11084 tablet_bootstrap.cc:492] T abfc8f390d434c47a8c4990f8406c270 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:34.870978 11727 ts_tablet_manager.cc:1939] T 3ff4cf556eee494d8cd590884466f09f P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:34.871193 11084 ts_tablet_manager.cc:1397] T abfc8f390d434c47a8c4990f8406c270 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.151s user 0.010s sys 0.003s
W20260925 09:57:34.874229 10918 leader_election.cc:341] T 61aa6380d30a4b21b70ee2d321247bd8 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:34.877662 11084 raft_consensus.cc:348] T abfc8f390d434c47a8c4990f8406c270 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.878404 11084 raft_consensus.cc:374] T abfc8f390d434c47a8c4990f8406c270 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:34.878667 11084 raft_consensus.cc:729] T abfc8f390d434c47a8c4990f8406c270 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:34.888245 11085 tablet_bootstrap.cc:654] T 2f07dc9548364870923165f4ea524310 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:34.879539 11084 consensus_queue.cc:260] T abfc8f390d434c47a8c4990f8406c270 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.848935 11082 raft_consensus.cc:348] T 92814437d53143a79f99199f8ee3a742 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.891878 11082 raft_consensus.cc:374] T 92814437d53143a79f99199f8ee3a742 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260925 09:57:34.892503 10918 consensus_peers.cc:487] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:34.894902 11082 raft_consensus.cc:729] T 92814437d53143a79f99199f8ee3a742 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:34.874905 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 304) took 4976 ms. Trace:
I20260925 09:57:34.895478 11727 rpcz_store.cc:276] 0925 09:57:29.898957 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:29.903683 (+ 4726us) service_pool.cc:227] Handling call
0925 09:57:34.449888 (+4546205us) ts_tablet_manager.cc:1145] Deleting tablet 3ff4cf556eee494d8cd590884466f09f
0925 09:57:34.449928 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:34.669926 (+219998us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:34.868739 (+198813us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:34.874835 (+ 6096us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4545566}
I20260925 09:57:34.899170 11082 consensus_queue.cc:260] T 92814437d53143a79f99199f8ee3a742 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.907661 11082 ts_tablet_manager.cc:1428] T 92814437d53143a79f99199f8ee3a742 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.118s user 0.006s sys 0.000s
I20260925 09:57:34.911787 11084 ts_tablet_manager.cc:1428] T abfc8f390d434c47a8c4990f8406c270 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.040s user 0.000s sys 0.007s
I20260925 09:57:34.914002 11082 tablet_bootstrap.cc:492] T a436fe287a0d48a1821191adcd031d74 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:34.914108 12209 raft_consensus.cc:482] T 977a59175e5b43eeb6f23b938afd00a3 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:34.914552 12209 raft_consensus.cc:504] T 977a59175e5b43eeb6f23b938afd00a3 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.917562 12209 leader_election.cc:288] T 977a59175e5b43eeb6f23b938afd00a3 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:34.920850 12323 raft_consensus.cc:482] T dba14a44a9de492886d629441606767e P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:34.923071 12323 raft_consensus.cc:504] T dba14a44a9de492886d629441606767e P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.925211 12323 leader_election.cc:288] T dba14a44a9de492886d629441606767e P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:34.930404 11085 tablet_bootstrap.cc:492] T 2f07dc9548364870923165f4ea524310 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:34.930820 11085 ts_tablet_manager.cc:1397] T 2f07dc9548364870923165f4ea524310 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.349s user 0.008s sys 0.001s
I20260925 09:57:34.931936 12027 ts_tablet_manager.cc:1939] T a44d9f2f7e3e48aeb0f31fa29ae1988a P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:34.933128 11085 raft_consensus.cc:348] T 2f07dc9548364870923165f4ea524310 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.933820 11085 raft_consensus.cc:374] T 2f07dc9548364870923165f4ea524310 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:34.934065 11085 raft_consensus.cc:729] T 2f07dc9548364870923165f4ea524310 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:34.934137 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 312) took 4342 ms. Trace:
W20260925 09:57:34.934612 10775 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 192509us
I20260925 09:57:34.934533 12027 rpcz_store.cc:276] 0925 09:57:30.592170 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:30.592314 (+ 144us) service_pool.cc:227] Handling call
0925 09:57:34.225785 (+3633471us) ts_tablet_manager.cc:1145] Deleting tablet a44d9f2f7e3e48aeb0f31fa29ae1988a
0925 09:57:34.225838 (+ 53us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:34.546171 (+320333us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:34.779253 (+233082us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:34.934086 (+154833us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3632756}
I20260925 09:57:34.936056 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e3120347c8e4a3eab57d6562ae40e8d" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:34.936594 10684 raft_consensus.cc:2454] T 4e3120347c8e4a3eab57d6562ae40e8d P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
I20260925 09:57:34.937839 12027 tablet_replica.cc:330] T 6161774f61ae48ab9764a05f2188a80b P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:34.939141 12027 raft_consensus.cc:2229] T 6161774f61ae48ab9764a05f2188a80b P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:34.948771 12027 raft_consensus.cc:2258] T 6161774f61ae48ab9764a05f2188a80b P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:34.949576 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dba14a44a9de492886d629441606767e" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:34.951331 12027 ts_tablet_manager.cc:1905] T 6161774f61ae48ab9764a05f2188a80b P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:34.934665 11085 consensus_queue.cc:260] T 2f07dc9548364870923165f4ea524310 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.954923 10775 leader_election.cc:302] T 7a31b999f0fd447eacff13de71aa3fff P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters: be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:34.973448 10982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dba14a44a9de492886d629441606767e" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:34.974186 10982 raft_consensus.cc:2454] T dba14a44a9de492886d629441606767e P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:34.973914 12360 tablet.cc:1722] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 09:57:34.974964 12352 raft_consensus.cc:482] T 54157e776efa4c6c8e2bd4c58321fcda P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:34.975499 12352 raft_consensus.cc:504] T 54157e776efa4c6c8e2bd4c58321fcda P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.977396 12352 leader_election.cc:288] T 54157e776efa4c6c8e2bd4c58321fcda P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:34.981160 12251 raft_consensus.cc:482] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b8850818e2ee42a082802d0095c08d21)
I20260925 09:57:34.981223 12352 raft_consensus.cc:482] T b39f9e7102ff4a9c96a2cc5d2676f03d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:34.981709 12251 raft_consensus.cc:504] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.984083 12251 leader_election.cc:288] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:34.987515 10679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61aa6380d30a4b21b70ee2d321247bd8" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:34.988591 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54157e776efa4c6c8e2bd4c58321fcda" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:34.981709 12352 raft_consensus.cc:504] T b39f9e7102ff4a9c96a2cc5d2676f03d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:34.991873 10919 leader_election.cc:341] T 61aa6380d30a4b21b70ee2d321247bd8 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:34.992349 10919 leader_election.cc:302] T 61aa6380d30a4b21b70ee2d321247bd8 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:34.992417 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b39f9e7102ff4a9c96a2cc5d2676f03d" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:34.994887 11082 tablet_bootstrap.cc:654] T a436fe287a0d48a1821191adcd031d74 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:34.998006 12206 raft_consensus.cc:504] T defaef4d6b754ca5a2dfc21b7a596a5b P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:34.993126 12353 consensus_queue.cc:1035] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:34.995044 12251 raft_consensus.cc:482] T c9fccada64d1411eb1f75480c8001c49 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:34.995471 12352 leader_election.cc:288] T b39f9e7102ff4a9c96a2cc5d2676f03d P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:34.999065 12251 raft_consensus.cc:504] T c9fccada64d1411eb1f75480c8001c49 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.001080 12251 leader_election.cc:288] T c9fccada64d1411eb1f75480c8001c49 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:35.001689 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 3ff4cf556eee494d8cd590884466f09f (table default.Table_66b8a45b14bf441d80ab32ee756de608 [id=2aeb0fb9ff8c416aba7b24d74354fcb0]) successfully deleted
W20260925 09:57:35.003450 10775 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 48828.1us
I20260925 09:57:35.004065 10775 leader_election.cc:302] T 4e3120347c8e4a3eab57d6562ae40e8d P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters: be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:35.005043 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet a44d9f2f7e3e48aeb0f31fa29ae1988a (table default.Table_eb065d5979c64931abf91d82b8623a0a [id=5b74f0855d2046e3a9463f732ae19e62]) successfully deleted
I20260925 09:57:35.005921 12123 raft_consensus.cc:2781] T 7a31b999f0fd447eacff13de71aa3fff P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:35.006225 12123 raft_consensus.cc:482] T 7a31b999f0fd447eacff13de71aa3fff P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:35.006520 12123 raft_consensus.cc:3037] T 7a31b999f0fd447eacff13de71aa3fff P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:35.006603 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b39f9e7102ff4a9c96a2cc5d2676f03d" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:35.007625 10830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "197f61525b0c4f9da441d98701d231a3" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:35.008172 10830 raft_consensus.cc:2454] T 197f61525b0c4f9da441d98701d231a3 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be97ab5d75934a648d4ab76341193ec0 in term 1.
I20260925 09:57:35.009579 10830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9fccada64d1411eb1f75480c8001c49" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:35.009794 10621 leader_election.cc:341] T dba14a44a9de492886d629441606767e P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:35.010545 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54157e776efa4c6c8e2bd4c58321fcda" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:35.014182 12355 raft_consensus.cc:2781] T 4e3120347c8e4a3eab57d6562ae40e8d P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260925 09:57:35.014530 10621 consensus_peers.cc:487] T 37bab60afe7a4c758a08ef7c9c24aba0 P b8850818e2ee42a082802d0095c08d21 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:34.973284 12356 tablet.cc:1722] T 5608c5f9b65b4da7b2fbb56417c92d88 P c43461c4ee854ae1ac52b873ccada318: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 09:57:35.014599 11085 ts_tablet_manager.cc:1428] T 2f07dc9548364870923165f4ea524310 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.083s user 0.004s sys 0.002s
I20260925 09:57:35.020747 12123 raft_consensus.cc:504] T 7a31b999f0fd447eacff13de71aa3fff P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.023556 12123 leader_election.cc:288] T 7a31b999f0fd447eacff13de71aa3fff P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:35.031694 11085 tablet_bootstrap.cc:492] T 6540aba5854a44419addc5d454e473b7 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:35.014612 12355 raft_consensus.cc:482] T 4e3120347c8e4a3eab57d6562ae40e8d P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:35.036286 12355 raft_consensus.cc:3037] T 4e3120347c8e4a3eab57d6562ae40e8d P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:35.037978 11082 tablet_bootstrap.cc:492] T a436fe287a0d48a1821191adcd031d74 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:35.038374 11082 ts_tablet_manager.cc:1397] T a436fe287a0d48a1821191adcd031d74 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.125s user 0.004s sys 0.004s
I20260925 09:57:35.040446 11082 raft_consensus.cc:348] T a436fe287a0d48a1821191adcd031d74 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.041060 11082 raft_consensus.cc:374] T a436fe287a0d48a1821191adcd031d74 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:35.041287 11082 raft_consensus.cc:729] T a436fe287a0d48a1821191adcd031d74 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:35.050175 12323 raft_consensus.cc:482] T cf775c9d9b114ddcb97dc6c6eca8e0fc P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:35.050841 11085 tablet_bootstrap.cc:654] T 6540aba5854a44419addc5d454e473b7 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:35.050685 12323 raft_consensus.cc:504] T cf775c9d9b114ddcb97dc6c6eca8e0fc P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.052814 12323 leader_election.cc:288] T cf775c9d9b114ddcb97dc6c6eca8e0fc P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:35.053618 12207 raft_consensus.cc:2726] T 61aa6380d30a4b21b70ee2d321247bd8 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:35.054355 10918 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 48113us
W20260925 09:57:35.055912 10918 leader_election.cc:341] T b39f9e7102ff4a9c96a2cc5d2676f03d P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:35.056466 10918 leader_election.cc:302] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters:
W20260925 09:57:35.057372 10918 leader_election.cc:341] T c9fccada64d1411eb1f75480c8001c49 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:35.057699 12353 raft_consensus.cc:2781] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260925 09:57:35.057969 10918 leader_election.cc:341] T 54157e776efa4c6c8e2bd4c58321fcda P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:35.058166 12353 raft_consensus.cc:482] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b8850818e2ee42a082802d0095c08d21)
I20260925 09:57:35.058601 12353 raft_consensus.cc:3037] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:57:35.059952 12207 raft_consensus.cc:482] T f9c259867cf44c4c90e50864475a4970 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:35.051108 12206 leader_election.cc:288] T defaef4d6b754ca5a2dfc21b7a596a5b P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:35.060379 12207 raft_consensus.cc:504] T f9c259867cf44c4c90e50864475a4970 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.061295 11727 tablet_replica.cc:330] T 92814437d53143a79f99199f8ee3a742 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:35.062608 12207 leader_election.cc:288] T f9c259867cf44c4c90e50864475a4970 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:35.063035 11084 tablet_bootstrap.cc:492] T 3d84c717b6ac462eb331e25d80c918e4 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:35.065711 10830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c259867cf44c4c90e50864475a4970" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:35.066165 10775 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 60870.1us
W20260925 09:57:35.066730 10775 leader_election.cc:341] T 977a59175e5b43eeb6f23b938afd00a3 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Not found: Tablet not found: 977a59175e5b43eeb6f23b938afd00a3
W20260925 09:57:35.067349 10918 leader_election.cc:341] T f9c259867cf44c4c90e50864475a4970 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:35.068168 10919 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 76467.4us
W20260925 09:57:35.069895 10919 leader_election.cc:341] T 54157e776efa4c6c8e2bd4c58321fcda P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:35.070300 10919 leader_election.cc:302] T 54157e776efa4c6c8e2bd4c58321fcda P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:35.070966 12207 raft_consensus.cc:2726] T 54157e776efa4c6c8e2bd4c58321fcda P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:35.074441 11084 tablet_bootstrap.cc:654] T 3d84c717b6ac462eb331e25d80c918e4 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:35.076066 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a31b999f0fd447eacff13de71aa3fff" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:35.076596 10687 raft_consensus.cc:3037] T 7a31b999f0fd447eacff13de71aa3fff P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:35.079361 10830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "defaef4d6b754ca5a2dfc21b7a596a5b" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:35.082320 10830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf775c9d9b114ddcb97dc6c6eca8e0fc" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:35.083294 10687 raft_consensus.cc:2454] T 7a31b999f0fd447eacff13de71aa3fff P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
W20260925 09:57:35.084151 10919 leader_election.cc:341] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Not found: Tablet not found: 197f61525b0c4f9da441d98701d231a3
W20260925 09:57:35.087319 10619 consensus_peers.cc:487] T 37bab60afe7a4c758a08ef7c9c24aba0 P b8850818e2ee42a082802d0095c08d21 -> Peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Couldn't send request to peer be97ab5d75934a648d4ab76341193ec0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 37bab60afe7a4c758a08ef7c9c24aba0. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:35.087739 12209 raft_consensus.cc:482] T 2f07dc9548364870923165f4ea524310 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:35.088240 12209 raft_consensus.cc:504] T 2f07dc9548364870923165f4ea524310 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.090070 12209 leader_election.cc:288] T 2f07dc9548364870923165f4ea524310 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:35.091611 12353 raft_consensus.cc:504] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.093217 10619 leader_election.cc:302] T dba14a44a9de492886d629441606767e P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:35.095074 12323 raft_consensus.cc:2781] T dba14a44a9de492886d629441606767e P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:35.095445 12323 raft_consensus.cc:482] T dba14a44a9de492886d629441606767e P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260925 09:57:35.096530 10621 consensus_peers.cc:487] T b61444c31dbf4882ac748f861ea66818 P b8850818e2ee42a082802d0095c08d21 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260925 09:57:35.098062 10621 leader_election.cc:341] T defaef4d6b754ca5a2dfc21b7a596a5b P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:35.098152 10830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "197f61525b0c4f9da441d98701d231a3" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:35.098080 12353 leader_election.cc:288] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:35.098764 10830 raft_consensus.cc:3037] T 197f61525b0c4f9da441d98701d231a3 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:57:35.096276 10775 leader_election.cc:302] T 7a31b999f0fd447eacff13de71aa3fff P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters:
W20260925 09:57:35.099722 10621 leader_election.cc:341] T cf775c9d9b114ddcb97dc6c6eca8e0fc P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:35.100447 12209 raft_consensus.cc:2781] T 7a31b999f0fd447eacff13de71aa3fff P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:35.100700 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f07dc9548364870923165f4ea524310" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:35.101514 12209 raft_consensus.cc:686] T 7a31b999f0fd447eacff13de71aa3fff P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Becoming Leader. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Running, Role: LEADER
I20260925 09:57:35.101890 12323 raft_consensus.cc:3037] T dba14a44a9de492886d629441606767e P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:35.102437 12209 consensus_queue.cc:237] T 7a31b999f0fd447eacff13de71aa3fff P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.109934 12355 raft_consensus.cc:504] T 4e3120347c8e4a3eab57d6562ae40e8d P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:35.085371 10919 leader_election.cc:341] T b39f9e7102ff4a9c96a2cc5d2676f03d P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:35.112097 10919 leader_election.cc:302] T b39f9e7102ff4a9c96a2cc5d2676f03d P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:35.112905 12353 raft_consensus.cc:2726] T b39f9e7102ff4a9c96a2cc5d2676f03d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:35.114053 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9fccada64d1411eb1f75480c8001c49" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:35.117257 10919 leader_election.cc:341] T c9fccada64d1411eb1f75480c8001c49 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:35.117607 10919 leader_election.cc:302] T c9fccada64d1411eb1f75480c8001c49 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:35.118286 12353 raft_consensus.cc:2726] T c9fccada64d1411eb1f75480c8001c49 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:35.116835 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c259867cf44c4c90e50864475a4970" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:35.041882 11082 consensus_queue.cc:260] T a436fe287a0d48a1821191adcd031d74 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.132916 11082 ts_tablet_manager.cc:1428] T a436fe287a0d48a1821191adcd031d74 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.094s user 0.006s sys 0.000s
I20260925 09:57:35.141307 12206 raft_consensus.cc:482] T 96a80382bb7d4395a14ad9ae972f4a25 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:35.141759 12206 raft_consensus.cc:504] T 96a80382bb7d4395a14ad9ae972f4a25 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.152285 12206 leader_election.cc:288] T 96a80382bb7d4395a14ad9ae972f4a25 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:35.157034 11082 tablet_bootstrap.cc:492] T aca5987b14b64152887ac5ee584b257a P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:35.160655 11727 raft_consensus.cc:2229] T 92814437d53143a79f99199f8ee3a742 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:35.161151 11727 raft_consensus.cc:2258] T 92814437d53143a79f99199f8ee3a742 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:35.163048 11727 ts_tablet_manager.cc:1905] T 92814437d53143a79f99199f8ee3a742 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:35.170820 11084 tablet_bootstrap.cc:492] T 3d84c717b6ac462eb331e25d80c918e4 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:35.171537 11084 ts_tablet_manager.cc:1397] T 3d84c717b6ac462eb331e25d80c918e4 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.109s user 0.011s sys 0.008s
W20260925 09:57:35.172909 10919 leader_election.cc:341] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Not found: Tablet not found: 197f61525b0c4f9da441d98701d231a3
W20260925 09:57:35.176632 10919 leader_election.cc:341] T f9c259867cf44c4c90e50864475a4970 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:35.177865 10919 leader_election.cc:302] T f9c259867cf44c4c90e50864475a4970 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:35.177820 12323 raft_consensus.cc:504] T dba14a44a9de492886d629441606767e P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.179045 12353 raft_consensus.cc:2726] T f9c259867cf44c4c90e50864475a4970 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:35.182622 10980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "defaef4d6b754ca5a2dfc21b7a596a5b" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:35.126610 10830 raft_consensus.cc:2454] T 197f61525b0c4f9da441d98701d231a3 P c43461c4ee854ae1ac52b873ccada318 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be97ab5d75934a648d4ab76341193ec0 in term 2.
W20260925 09:57:35.142845 10772 leader_election.cc:341] T 977a59175e5b43eeb6f23b938afd00a3 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 977a59175e5b43eeb6f23b938afd00a3
I20260925 09:57:35.183295 10980 raft_consensus.cc:3037] T defaef4d6b754ca5a2dfc21b7a596a5b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:35.183408 10772 leader_election.cc:302] T 977a59175e5b43eeb6f23b938afd00a3 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
W20260925 09:57:35.185269 10772 leader_election.cc:341] T 7a31b999f0fd447eacff13de71aa3fff P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 7a31b999f0fd447eacff13de71aa3fff
I20260925 09:57:35.147432 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96a80382bb7d4395a14ad9ae972f4a25" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:35.186697 12357 raft_consensus.cc:2726] T 977a59175e5b43eeb6f23b938afd00a3 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:35.187408 10621 leader_election.cc:341] T 96a80382bb7d4395a14ad9ae972f4a25 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:35.161002 11085 tablet_bootstrap.cc:492] T 6540aba5854a44419addc5d454e473b7 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:35.189534 10918 leader_election.cc:302] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21
I20260925 09:57:35.189994 11085 ts_tablet_manager.cc:1397] T 6540aba5854a44419addc5d454e473b7 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.158s user 0.008s sys 0.003s
I20260925 09:57:35.190189 12251 raft_consensus.cc:2781] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 09:57:35.190595 12251 raft_consensus.cc:686] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0 [term 2 LEADER]: Becoming Leader. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Running, Role: LEADER
I20260925 09:57:35.191366 12251 consensus_queue.cc:237] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.192576 11085 raft_consensus.cc:348] T 6540aba5854a44419addc5d454e473b7 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.193244 11085 raft_consensus.cc:374] T 6540aba5854a44419addc5d454e473b7 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:35.193465 11085 raft_consensus.cc:729] T 6540aba5854a44419addc5d454e473b7 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:35.197613 12355 leader_election.cc:288] T 4e3120347c8e4a3eab57d6562ae40e8d P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:35.198671 10673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e3120347c8e4a3eab57d6562ae40e8d" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:35.199216 10673 raft_consensus.cc:3037] T 4e3120347c8e4a3eab57d6562ae40e8d P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:35.206944 10673 raft_consensus.cc:2454] T 4e3120347c8e4a3eab57d6562ae40e8d P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:35.175715 12027 ts_tablet_manager.cc:1918] T 6161774f61ae48ab9764a05f2188a80b P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:35.180276 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dba14a44a9de492886d629441606767e" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:35.173769 11084 raft_consensus.cc:348] T 3d84c717b6ac462eb331e25d80c918e4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.217664 11084 raft_consensus.cc:374] T 3d84c717b6ac462eb331e25d80c918e4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:35.218335 11084 raft_consensus.cc:729] T 3d84c717b6ac462eb331e25d80c918e4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:35.218058 12027 log.cc:1196] T 6161774f61ae48ab9764a05f2188a80b P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/6161774f61ae48ab9764a05f2188a80b
I20260925 09:57:35.219102 11084 consensus_queue.cc:260] T 3d84c717b6ac462eb331e25d80c918e4 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.220618 12027 ts_tablet_manager.cc:1939] T 6161774f61ae48ab9764a05f2188a80b P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:35.223614 11084 ts_tablet_manager.cc:1428] T 3d84c717b6ac462eb331e25d80c918e4 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.052s user 0.007s sys 0.000s
I20260925 09:57:35.218600 11085 consensus_queue.cc:260] T 6540aba5854a44419addc5d454e473b7 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.229993 11727 ts_tablet_manager.cc:1918] T 92814437d53143a79f99199f8ee3a742 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:35.230351 11727 log.cc:1196] T 92814437d53143a79f99199f8ee3a742 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/92814437d53143a79f99199f8ee3a742
I20260925 09:57:35.230924 11727 ts_tablet_manager.cc:1939] T 92814437d53143a79f99199f8ee3a742 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:35.232844 11085 ts_tablet_manager.cc:1428] T 6540aba5854a44419addc5d454e473b7 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.043s user 0.005s sys 0.000s
W20260925 09:57:35.216526 10621 leader_election.cc:341] T dba14a44a9de492886d629441606767e P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:35.240820 11082 tablet_bootstrap.cc:654] T aca5987b14b64152887ac5ee584b257a P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:35.246671 10983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96a80382bb7d4395a14ad9ae972f4a25" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:35.247592 10983 raft_consensus.cc:2454] T 96a80382bb7d4395a14ad9ae972f4a25 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:35.243688 10775 leader_election.cc:302] T 4e3120347c8e4a3eab57d6562ae40e8d P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:35.247668 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 313) took 4655 ms. Trace:
I20260925 09:57:35.249924 12355 raft_consensus.cc:2781] T 4e3120347c8e4a3eab57d6562ae40e8d P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:35.250694 12355 raft_consensus.cc:686] T 4e3120347c8e4a3eab57d6562ae40e8d P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Becoming Leader. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Running, Role: LEADER
I20260925 09:57:35.251672 12355 consensus_queue.cc:237] T 4e3120347c8e4a3eab57d6562ae40e8d P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.250111 12027 rpcz_store.cc:276] 0925 09:57:30.592517 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:30.592623 (+ 106us) service_pool.cc:227] Handling call
0925 09:57:34.937690 (+4345067us) ts_tablet_manager.cc:1145] Deleting tablet 6161774f61ae48ab9764a05f2188a80b
0925 09:57:34.937741 (+ 51us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:34.994139 (+ 56398us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:35.175517 (+181378us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:35.247614 (+ 72097us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4344605}
I20260925 09:57:35.254702 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 6161774f61ae48ab9764a05f2188a80b (table default.Table_64ba123898174e72a6884dbb9ad07de3 [id=7ee526720b2442119e0fc487b7c4fcd4]) successfully deleted
I20260925 09:57:35.249948 11085 tablet_bootstrap.cc:492] T 8aff53cfd4994c629a408ee8250280c5 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:35.259294 10385 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 6540aba5854a44419addc5d454e473b7 (table default.Table_8180842f664042b0a9073b7008438ff7 [id=416a4b989570457e919fb7e7b16a017d]). Expected version 1 got 0
I20260925 09:57:35.259956 12323 leader_election.cc:288] T dba14a44a9de492886d629441606767e P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:35.262606 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 315) took 4669 ms. Trace:
I20260925 09:57:35.265264 12323 raft_consensus.cc:482] T ab6bdba5ca5d41ae921a90c1f77006c5 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:35.265833 10385 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet f62c62715c9b42de826c1a66fd431eab (table default.Table_bcbc82519cf043738f61029181f29418 [id=b3f7f21ab2214e4cb637bea27ce62abe]). Expected version 1 got 0
I20260925 09:57:35.262945 12027 rpcz_store.cc:276] 0925 09:57:30.593217 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:30.593340 (+ 123us) service_pool.cc:227] Handling call
0925 09:57:35.253373 (+4660033us) ts_tablet_manager.cc:1145] Deleting tablet 61aa6380d30a4b21b70ee2d321247bd8
0925 09:57:35.253430 (+ 57us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:35.262546 (+ 9116us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4659535}
I20260925 09:57:35.266156 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dba14a44a9de492886d629441606767e" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:35.266706 10976 raft_consensus.cc:3037] T dba14a44a9de492886d629441606767e P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:57:35.266782 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 61aa6380d30a4b21b70ee2d321247bd8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 61aa6380d30a4b21b70ee2d321247bd8 already in progress: creating tablet
I20260925 09:57:35.266701 10385 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 5608c5f9b65b4da7b2fbb56417c92d88 (table default.Table_f37bcee5db2044f2bea9c353982b30cd [id=e46d114e5d484b66bb15eb0416b5ce0d]). Expected version 2 got 0
I20260925 09:57:35.267920 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 316) took 4674 ms. Trace:
I20260925 09:57:35.265676 12323 raft_consensus.cc:504] T ab6bdba5ca5d41ae921a90c1f77006c5 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.268213 12027 rpcz_store.cc:276] 0925 09:57:30.593512 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:30.603566 (+ 10054us) service_pool.cc:227] Handling call
0925 09:57:35.267594 (+4664028us) ts_tablet_manager.cc:1145] Deleting tablet 815b4b1bd914433798dcb5deec6ec79c
0925 09:57:35.267641 (+ 47us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:35.267872 (+ 231us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4663403}
W20260925 09:57:35.269285 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 815b4b1bd914433798dcb5deec6ec79c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 815b4b1bd914433798dcb5deec6ec79c already in progress: creating tablet
I20260925 09:57:35.271001 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab6bdba5ca5d41ae921a90c1f77006c5" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:35.272571 10385 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet e684798155d445bfb325a1b7c5c48753 (table default.Table_dfb73f4afe7c464d8dc98356500054e2 [id=ab3c143dc00c43e78531514a114bfd16]). Expected version 1 got 0
W20260925 09:57:35.272986 10621 leader_election.cc:341] T ab6bdba5ca5d41ae921a90c1f77006c5 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:35.273635 10976 raft_consensus.cc:2454] T dba14a44a9de492886d629441606767e P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:35.274112 12323 leader_election.cc:288] T ab6bdba5ca5d41ae921a90c1f77006c5 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:35.274317 10385 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 2f07dc9548364870923165f4ea524310 (table default.Table_0db927e92a3b4970a7efea3fb765e65e [id=a988264fe43c4eb88a84f8b6359e77a6]). Expected version 1 got 0
I20260925 09:57:35.276667 11084 tablet_bootstrap.cc:492] T b3d9a3cf5bda4cf6bc1fa3fae1cb9288 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:35.277336 10385 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 3d652a76d96f4dbeafde15f3c16534a6 (table default.Table_be0f658aa41f4188bc16db37e96f4ad1 [id=73083c1560cb4db5a12964387cb49b73]). Expected version 1 got 0
I20260925 09:57:35.279654 10384 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet a21909e2e5794f57971f4e42795235c7
I20260925 09:57:35.283938 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f07dc9548364870923165f4ea524310" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:35.284010 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 302) took 5642 ms. Trace:
I20260925 09:57:35.284555 11727 rpcz_store.cc:276] 0925 09:57:29.641433 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:29.687412 (+ 45979us) service_pool.cc:227] Handling call
0925 09:57:34.903861 (+5216449us) ts_tablet_manager.cc:1145] Deleting tablet 92814437d53143a79f99199f8ee3a742
0925 09:57:35.061125 (+157264us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:35.174588 (+113463us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:35.229793 (+ 55205us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:35.283958 (+ 54165us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":98375,"tablet-delete.queue_time_us":4964335}
I20260925 09:57:35.285945 11727 tablet_replica.cc:330] T 61aa6380d30a4b21b70ee2d321247bd8 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:35.235239 10980 raft_consensus.cc:2454] T defaef4d6b754ca5a2dfc21b7a596a5b P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:35.288116 10668 tablet_service.cc:1514] Processing DeleteTablet for tablet a21909e2e5794f57971f4e42795235c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:32 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:35.288770 11727 raft_consensus.cc:2229] T 61aa6380d30a4b21b70ee2d321247bd8 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:35.289489 11727 raft_consensus.cc:2258] T 61aa6380d30a4b21b70ee2d321247bd8 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:35.290110 10809 tablet_service.cc:1514] Processing DeleteTablet for tablet a21909e2e5794f57971f4e42795235c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:32 UTC) from {username='slave'} at 127.0.0.1:33344
W20260925 09:57:35.290701 10619 consensus_peers.cc:487] T 545ac025f2cb469da847979011a0b3ce P b8850818e2ee42a082802d0095c08d21 -> Peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Couldn't send request to peer be97ab5d75934a648d4ab76341193ec0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 545ac025f2cb469da847979011a0b3ce. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:35.291730 11727 ts_tablet_manager.cc:1905] T 61aa6380d30a4b21b70ee2d321247bd8 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:35.291816 10384 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:35140 (request call id 348) took 3155 ms. Trace:
I20260925 09:57:35.293474 12206 consensus_queue.cc:1035] T b61444c31dbf4882ac748f861ea66818 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:35.294858 12251 raft_consensus.cc:482] T 8a3149fb2325435699af36338e77d042 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:35.295276 12251 raft_consensus.cc:504] T 8a3149fb2325435699af36338e77d042 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.297346 12251 leader_election.cc:288] T 8a3149fb2325435699af36338e77d042 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:35.298686 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a3149fb2325435699af36338e77d042" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:35.299525 10619 leader_election.cc:341] T cf775c9d9b114ddcb97dc6c6eca8e0fc P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: cf775c9d9b114ddcb97dc6c6eca8e0fc
I20260925 09:57:35.299880 10619 leader_election.cc:302] T cf775c9d9b114ddcb97dc6c6eca8e0fc P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8850818e2ee42a082802d0095c08d21; no voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:35.301009 11084 tablet_bootstrap.cc:654] T b3d9a3cf5bda4cf6bc1fa3fae1cb9288 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:35.301120 10918 leader_election.cc:341] T 8a3149fb2325435699af36338e77d042 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:35.301853 12206 raft_consensus.cc:2726] T cf775c9d9b114ddcb97dc6c6eca8e0fc P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:35.302246 10673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a3149fb2325435699af36338e77d042" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:35.303416 10919 leader_election.cc:341] T 8a3149fb2325435699af36338e77d042 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:35.303759 10919 leader_election.cc:302] T 8a3149fb2325435699af36338e77d042 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:35.304428 12251 raft_consensus.cc:2726] T 8a3149fb2325435699af36338e77d042 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:35.292092 10384 rpcz_store.cc:276] 0925 09:57:32.136718 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:32.137149 (+ 431us) service_pool.cc:227] Handling call
0925 09:57:32.138321 (+ 1172us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:57:32.138852 (+ 531us) catalog_manager.cc:2752] Modifying in-memory table state
0925 09:57:32.138962 (+ 110us) catalog_manager.cc:2761] Locking tablets
0925 09:57:32.157357 (+ 18395us) catalog_manager.cc:2775] Removing table and tablets from system table
0925 09:57:32.238288 (+ 80931us) catalog_manager.cc:2793] Removing table from by-name map
0925 09:57:32.342678 (+104390us) catalog_manager.cc:2809] Committing in-memory state
0925 09:57:35.279452 (+2936774us) catalog_manager.cc:2813] Aborting table tasks
0925 09:57:35.291738 (+ 12286us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:32.199501 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:32.199833 (+ 332us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:32.199855 (+ 22us) write_op.cc:435] Acquired schema lock
0925 09:57:32.199870 (+ 15us) tablet.cc:660] Decoding operations
0925 09:57:32.200170 (+ 300us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:32.200209 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:32.200228 (+ 19us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:32.200489 (+ 261us) tablet.cc:699] Row locks acquired
0925 09:57:32.200501 (+ 12us) write_op.cc:260] PREPARE: finished
0925 09:57:32.200605 (+ 104us) write_op.cc:270] Start()
0925 09:57:32.200677 (+ 72us) write_op.cc:276] Timestamp: P: 1790330252200524 usec, L: 0
0925 09:57:32.200692 (+ 15us) op_driver.cc:331] REPLICATION: starting
0925 09:57:32.201128 (+ 436us) log.cc:842] Serialized 756 byte log entry
0925 09:57:32.233636 (+ 32508us) op_driver.cc:444] REPLICATION: finished
0925 09:57:32.234625 (+ 989us) write_op.cc:301] APPLY: starting
0925 09:57:32.234678 (+ 53us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:32.234784 (+ 106us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:32.234868 (+ 84us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:32.235096 (+ 228us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:32.235228 (+ 132us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 09:57:32.235269 (+ 41us) write_op.cc:312] APPLY: finished
0925 09:57:32.236614 (+ 1345us) log.cc:842] Serialized 29 byte log entry
0925 09:57:32.237212 (+ 598us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:32.237276 (+ 64us) write_op.cc:454] Released schema lock
0925 09:57:32.237307 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:32.237481 (+ 174us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:32.237504 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":843,"apply.run_cpu_time_us":1523,"apply.run_wall_time_us":3060,"num_ops":2,"prepare.queue_time_us":931,"prepare.run_cpu_time_us":1910,"prepare.run_wall_time_us":22030,"raft.queue_time_us":18389,"raft.run_cpu_time_us":14,"raft.run_wall_time_us":14,"replication_time_us":32877}]]}
I20260925 09:57:35.310967 10619 leader_election.cc:302] T 96a80382bb7d4395a14ad9ae972f4a25 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:35.312007 12027 tablet_replica.cc:330] T 2d087d3e35814c90a03eccd90eeea86c P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:35.312228 12206 raft_consensus.cc:2781] T 96a80382bb7d4395a14ad9ae972f4a25 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:35.312611 12206 raft_consensus.cc:482] T 96a80382bb7d4395a14ad9ae972f4a25 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:35.312929 12027 raft_consensus.cc:2229] T 2d087d3e35814c90a03eccd90eeea86c P c43461c4ee854ae1ac52b873ccada318 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:35.312907 12206 raft_consensus.cc:3037] T 96a80382bb7d4395a14ad9ae972f4a25 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:35.314297 12209 raft_consensus.cc:482] T 6540aba5854a44419addc5d454e473b7 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:35.314738 12209 raft_consensus.cc:504] T 6540aba5854a44419addc5d454e473b7 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.317010 12209 leader_election.cc:288] T 6540aba5854a44419addc5d454e473b7 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:35.325654 10673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6540aba5854a44419addc5d454e473b7" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:35.329555 12206 raft_consensus.cc:504] T 96a80382bb7d4395a14ad9ae972f4a25 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.307381 11085 tablet_bootstrap.cc:654] T 8aff53cfd4994c629a408ee8250280c5 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:35.332211 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96a80382bb7d4395a14ad9ae972f4a25" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
W20260925 09:57:35.336373 10621 leader_election.cc:341] T 96a80382bb7d4395a14ad9ae972f4a25 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:35.338196 10384 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.2:40761 (request call id 41) took 1533 ms (client timeout 1000 ms). Trace:
W20260925 09:57:35.338567 10384 rpcz_store.cc:269] 0925 09:57:33.805031 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:35.337660 (+1532629us) service_pool.cc:214] Skipping call since client already timed out
0925 09:57:35.338119 (+ 459us) inbound_call.cc:173] Queueing failure response
Metrics: {}
I20260925 09:57:35.343969 10619 leader_election.cc:302] T dba14a44a9de492886d629441606767e P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:35.344755 12080 raft_consensus.cc:2781] T dba14a44a9de492886d629441606767e P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:35.345144 12080 raft_consensus.cc:686] T dba14a44a9de492886d629441606767e P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:35.345784 12080 consensus_queue.cc:237] T dba14a44a9de492886d629441606767e P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.313444 12027 raft_consensus.cc:2258] T 2d087d3e35814c90a03eccd90eeea86c P c43461c4ee854ae1ac52b873ccada318 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260925 09:57:35.352746 10389 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e83d6dc8657d4a1f89ab0893f435abf6 has no consensus state
I20260925 09:57:35.353216 12027 ts_tablet_manager.cc:1905] T 2d087d3e35814c90a03eccd90eeea86c P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:35.353415 10389 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:35140 (request call id 340) took 3677 ms. Trace:
I20260925 09:57:35.352770 10387 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 93e1b5837aff48d7907890c40dfcd0d0
I20260925 09:57:35.357159 10387 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:35140 (request call id 351) took 3186 ms. Trace:
I20260925 09:57:35.360718 10772 leader_election.cc:302] T 2f07dc9548364870923165f4ea524310 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:35.357410 10387 rpcz_store.cc:276] 0925 09:57:32.170303 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:32.181068 (+ 10765us) service_pool.cc:227] Handling call
0925 09:57:32.182877 (+ 1809us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:57:32.183432 (+ 555us) catalog_manager.cc:2752] Modifying in-memory table state
0925 09:57:32.183545 (+ 113us) catalog_manager.cc:2761] Locking tablets
0925 09:57:32.184038 (+ 493us) catalog_manager.cc:2775] Removing table and tablets from system table
0925 09:57:32.260297 (+ 76259us) catalog_manager.cc:2793] Removing table from by-name map
0925 09:57:32.347128 (+ 86831us) catalog_manager.cc:2809] Committing in-memory state
0925 09:57:35.352536 (+3005408us) catalog_manager.cc:2813] Aborting table tasks
0925 09:57:35.357103 (+ 4567us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:32.221643 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:32.222003 (+ 360us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:32.222024 (+ 21us) write_op.cc:435] Acquired schema lock
0925 09:57:32.222036 (+ 12us) tablet.cc:660] Decoding operations
0925 09:57:32.222317 (+ 281us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:32.222350 (+ 33us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:32.222371 (+ 21us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:32.222648 (+ 277us) tablet.cc:699] Row locks acquired
0925 09:57:32.222660 (+ 12us) write_op.cc:260] PREPARE: finished
0925 09:57:32.222757 (+ 97us) write_op.cc:270] Start()
0925 09:57:32.222835 (+ 78us) write_op.cc:276] Timestamp: P: 1790330252222724 usec, L: 0
0925 09:57:32.222849 (+ 14us) op_driver.cc:331] REPLICATION: starting
0925 09:57:32.223280 (+ 431us) log.cc:842] Serialized 762 byte log entry
0925 09:57:32.257162 (+ 33882us) op_driver.cc:444] REPLICATION: finished
0925 09:57:32.257910 (+ 748us) write_op.cc:301] APPLY: starting
0925 09:57:32.257968 (+ 58us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:32.258074 (+ 106us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:32.258161 (+ 87us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:32.258385 (+ 224us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:32.258528 (+ 143us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 09:57:32.258568 (+ 40us) write_op.cc:312] APPLY: finished
0925 09:57:32.258740 (+ 172us) log.cc:842] Serialized 29 byte log entry
0925 09:57:32.259283 (+ 543us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:32.259346 (+ 63us) write_op.cc:454] Released schema lock
0925 09:57:32.259375 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:32.259974 (+ 599us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:32.259997 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":114048,"child_traces":[["op",{"apply.queue_time_us":601,"apply.run_cpu_time_us":1456,"apply.run_wall_time_us":2263,"num_ops":2,"prepare.queue_time_us":6421,"prepare.run_cpu_time_us":1963,"prepare.run_wall_time_us":18437,"raft.queue_time_us":8293,"raft.run_cpu_time_us":12,"raft.run_wall_time_us":13,"replication_time_us":34238}]]}
I20260925 09:57:35.362654 12209 raft_consensus.cc:2726] T 2f07dc9548364870923165f4ea524310 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:35.363222 10667 tablet_service.cc:1514] Processing DeleteTablet for tablet 93e1b5837aff48d7907890c40dfcd0d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:32 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:35.363310 10809 tablet_service.cc:1514] Processing DeleteTablet for tablet 93e1b5837aff48d7907890c40dfcd0d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:32 UTC) from {username='slave'} at 127.0.0.1:33344
W20260925 09:57:35.364205 10387 rpcz_store.cc:267] Call kudu.master.MasterService.Ping from 127.10.9.1:56563 (request call id 42) took 832 ms (client timeout 1000 ms). Trace:
W20260925 09:57:35.364548 10387 rpcz_store.cc:269] 0925 09:57:34.531327 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:35.364027 (+832700us) service_pool.cc:227] Handling call
0925 09:57:35.364142 (+ 115us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260925 09:57:35.365289 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 92814437d53143a79f99199f8ee3a742 (table default.Table_755c3887714845d2bd8bb30bd444adef [id=d8fb4efa035549159e9b06af8723d715]) successfully deleted
W20260925 09:57:35.365505 10772 leader_election.cc:341] T 4e3120347c8e4a3eab57d6562ae40e8d P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 4e3120347c8e4a3eab57d6562ae40e8d
I20260925 09:57:35.353672 10389 rpcz_store.cc:276] 0925 09:57:31.676255 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:31.676365 (+ 110us) service_pool.cc:227] Handling call
0925 09:57:31.684127 (+ 7762us) spinlock_profiling.cc:243] Waited 7.11 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2adb7d6 00007fabf0621634 00007fabf06215c2 00007fabf06214f1 00007fabf062146c 00007fabf0620722 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:31.684762 (+ 635us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:57:31.685408 (+ 646us) catalog_manager.cc:3616] Apply alter schema
0925 09:57:31.685941 (+ 533us) catalog_manager.cc:3639] Acquired catalog manager lock
0925 09:57:31.686122 (+ 181us) catalog_manager.cc:3786] Updating metadata on disk
0925 09:57:31.882358 (+196236us) catalog_manager.cc:3817] Committing alterations to in-memory state
0925 09:57:35.353316 (+3470958us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:31.711107 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:31.711463 (+ 356us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:31.711487 (+ 24us) write_op.cc:435] Acquired schema lock
0925 09:57:31.711499 (+ 12us) tablet.cc:660] Decoding operations
0925 09:57:31.711740 (+ 241us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:31.711774 (+ 34us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:31.711796 (+ 22us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:31.711926 (+ 130us) tablet.cc:699] Row locks acquired
0925 09:57:31.711939 (+ 13us) write_op.cc:260] PREPARE: finished
0925 09:57:31.712038 (+ 99us) write_op.cc:270] Start()
0925 09:57:31.718920 (+ 6882us) write_op.cc:276] Timestamp: P: 1790330251712000 usec, L: 0
0925 09:57:31.718940 (+ 20us) op_driver.cc:331] REPLICATION: starting
0925 09:57:31.719362 (+ 422us) log.cc:842] Serialized 405 byte log entry
0925 09:57:31.879656 (+160294us) op_driver.cc:444] REPLICATION: finished
0925 09:57:31.880471 (+ 815us) write_op.cc:301] APPLY: starting
0925 09:57:31.880527 (+ 56us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:31.880621 (+ 94us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:31.880706 (+ 85us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:31.880865 (+ 159us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:31.880988 (+ 123us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:31.881028 (+ 40us) write_op.cc:312] APPLY: finished
0925 09:57:31.881195 (+ 167us) log.cc:842] Serialized 23 byte log entry
0925 09:57:31.881826 (+ 631us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:31.881882 (+ 56us) write_op.cc:454] Released schema lock
0925 09:57:31.881917 (+ 35us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:31.882611 (+ 694us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:31.882637 (+ 26us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":16359680,"child_traces":[["op",{"apply.queue_time_us":663,"apply.run_cpu_time_us":1350,"apply.run_wall_time_us":2308,"num_ops":1,"prepare.queue_time_us":23869,"prepare.run_cpu_time_us":1698,"prepare.run_wall_time_us":10187,"replication_time_us":160631}]]}
I20260925 09:57:35.369179 10980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab6bdba5ca5d41ae921a90c1f77006c5" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:35.369895 10980 raft_consensus.cc:2454] T ab6bdba5ca5d41ae921a90c1f77006c5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:35.382459 10736 heartbeater.cc:347] Connected to a master server at 127.10.9.61:33047
I20260925 09:57:35.385299 10387 master_service.cc:946] ReplaceTablet: received request to replace tablet 4c3e66f6ad23475f9fb831afc1d579c4 from {username='slave'} at 127.0.0.1:35140
I20260925 09:57:35.387055 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6540aba5854a44419addc5d454e473b7" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:35.387863 11082 tablet_bootstrap.cc:492] T aca5987b14b64152887ac5ee584b257a P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:35.388259 11082 ts_tablet_manager.cc:1397] T aca5987b14b64152887ac5ee584b257a P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.255s user 0.010s sys 0.003s
I20260925 09:57:35.389513 10956 tablet_service.cc:1514] Processing DeleteTablet for tablet a21909e2e5794f57971f4e42795235c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:32 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:35.390143 10952 tablet_service.cc:1514] Processing DeleteTablet for tablet 93e1b5837aff48d7907890c40dfcd0d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:32 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:35.391484 10391 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: default.Table_7a8e20521b384bfa8f659eaaf46bae54 [id=cf07ed8756904011ab3dac900ea6705a] alter complete (version 2)
I20260925 09:57:35.398138 10772 leader_election.cc:302] T 6540aba5854a44419addc5d454e473b7 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:35.399858 10619 leader_election.cc:302] T defaef4d6b754ca5a2dfc21b7a596a5b P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:35.401105 12257 raft_consensus.cc:2781] T defaef4d6b754ca5a2dfc21b7a596a5b P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:35.401508 12257 raft_consensus.cc:686] T defaef4d6b754ca5a2dfc21b7a596a5b P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
W20260925 09:57:35.401600 10391 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.2:40761 (request call id 40) took 2619 ms (client timeout 1000 ms). Trace:
I20260925 09:57:35.397643 11082 raft_consensus.cc:348] T aca5987b14b64152887ac5ee584b257a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.402017 11082 raft_consensus.cc:374] T aca5987b14b64152887ac5ee584b257a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:35.402243 11082 raft_consensus.cc:729] T aca5987b14b64152887ac5ee584b257a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:35.402298 12257 consensus_queue.cc:237] T defaef4d6b754ca5a2dfc21b7a596a5b P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.403826 12206 leader_election.cc:288] T 96a80382bb7d4395a14ad9ae972f4a25 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:35.405926 10619 leader_election.cc:302] T ab6bdba5ca5d41ae921a90c1f77006c5 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:35.407011 12322 raft_consensus.cc:2781] T ab6bdba5ca5d41ae921a90c1f77006c5 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:35.407303 12322 raft_consensus.cc:482] T ab6bdba5ca5d41ae921a90c1f77006c5 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:35.407585 12322 raft_consensus.cc:3037] T ab6bdba5ca5d41ae921a90c1f77006c5 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:57:35.401883 10391 rpcz_store.cc:269] 0925 09:57:32.782557 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:32.829092 (+ 46535us) service_pool.cc:227] Handling call
0925 09:57:32.896130 (+ 67038us) spinlock_profiling.cc:243] Waited 57.6 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2ad9c3e 00007fabf0610fb4 00007fabf0610f42 00007fabf0610e71 00007fabf0610dec 00007fabf06100a2 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:35.401526 (+2505396us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:34.663056 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:34.663564 (+ 508us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:34.663591 (+ 27us) write_op.cc:435] Acquired schema lock
0925 09:57:34.663605 (+ 14us) tablet.cc:660] Decoding operations
0925 09:57:34.663904 (+ 299us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:34.663944 (+ 40us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:34.663968 (+ 24us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:34.664122 (+ 154us) tablet.cc:699] Row locks acquired
0925 09:57:34.664145 (+ 23us) write_op.cc:260] PREPARE: finished
0925 09:57:34.664269 (+ 124us) write_op.cc:270] Start()
0925 09:57:34.664374 (+ 105us) write_op.cc:276] Timestamp: P: 1790330254664222 usec, L: 0
0925 09:57:34.664394 (+ 20us) op_driver.cc:331] REPLICATION: starting
0925 09:57:34.664875 (+ 481us) log.cc:842] Serialized 356 byte log entry
0925 09:57:34.672498 (+ 7623us) op_driver.cc:444] REPLICATION: finished
0925 09:57:34.672746 (+ 248us) write_op.cc:301] APPLY: starting
0925 09:57:34.672803 (+ 57us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:34.672904 (+ 101us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:34.672988 (+ 84us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:34.673227 (+ 239us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:34.673354 (+ 127us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:34.673400 (+ 46us) write_op.cc:312] APPLY: finished
0925 09:57:34.673568 (+ 168us) log.cc:842] Serialized 23 byte log entry
0925 09:57:34.673701 (+ 133us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:34.673765 (+ 64us) write_op.cc:454] Released schema lock
0925 09:57:34.673796 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:34.673995 (+ 199us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:34.674021 (+ 26us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":8,"spinlock_wait_cycles":132383488,"child_traces":[["op",{"apply.queue_time_us":82,"apply.run_cpu_time_us":1477,"apply.run_wall_time_us":1476,"num_ops":1,"prepare.queue_time_us":151,"prepare.run_cpu_time_us":2224,"prepare.run_wall_time_us":2237,"raft.queue_time_us":2422,"raft.run_cpu_time_us":604,"raft.run_wall_time_us":603,"replication_time_us":8032}]]}
I20260925 09:57:35.411856 11030 heartbeater.cc:347] Connected to a master server at 127.10.9.61:33047
I20260925 09:57:35.414736 12322 raft_consensus.cc:504] T ab6bdba5ca5d41ae921a90c1f77006c5 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.402788 11082 consensus_queue.cc:260] T aca5987b14b64152887ac5ee584b257a P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.423777 10391 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_3275db3804df4b12ad6e64e4d88bc75b" } new_table_name: "default.Table_8dbaa500b6404f54848d6d9a0d9890c8" modify_external_catalogs: true
I20260925 09:57:35.428224 12251 raft_consensus.cc:482] T abfc8f390d434c47a8c4990f8406c270 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:35.428726 12251 raft_consensus.cc:504] T abfc8f390d434c47a8c4990f8406c270 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.432610 12251 leader_election.cc:288] T abfc8f390d434c47a8c4990f8406c270 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:35.433667 11084 tablet_bootstrap.cc:492] T b3d9a3cf5bda4cf6bc1fa3fae1cb9288 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:35.433503 10673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abfc8f390d434c47a8c4990f8406c270" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:35.434139 11084 ts_tablet_manager.cc:1397] T b3d9a3cf5bda4cf6bc1fa3fae1cb9288 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.158s user 0.005s sys 0.003s
W20260925 09:57:35.435024 10919 leader_election.cc:341] T abfc8f390d434c47a8c4990f8406c270 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:35.436852 10830 raft_consensus.cc:3037] T 2f07dc9548364870923165f4ea524310 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:35.437503 10809 tablet_service.cc:1514] Processing DeleteTablet for tablet c7cd1d6a603642afa2c8c737079961b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:29 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:35.440477 12365 raft_consensus.cc:2726] T 6540aba5854a44419addc5d454e473b7 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:35.440622 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abfc8f390d434c47a8c4990f8406c270" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:35.442231 10918 leader_election.cc:341] T abfc8f390d434c47a8c4990f8406c270 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:35.442673 10918 leader_election.cc:302] T abfc8f390d434c47a8c4990f8406c270 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:35.443426 12251 raft_consensus.cc:2726] T abfc8f390d434c47a8c4990f8406c270 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:35.445188 12322 leader_election.cc:288] T ab6bdba5ca5d41ae921a90c1f77006c5 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:35.446031 11085 tablet_bootstrap.cc:492] T 8aff53cfd4994c629a408ee8250280c5 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:35.446619 11085 ts_tablet_manager.cc:1397] T 8aff53cfd4994c629a408ee8250280c5 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.197s user 0.011s sys 0.000s
I20260925 09:57:35.456162 11085 raft_consensus.cc:348] T 8aff53cfd4994c629a408ee8250280c5 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.457374 11085 raft_consensus.cc:374] T 8aff53cfd4994c629a408ee8250280c5 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:35.459093 11085 raft_consensus.cc:729] T 8aff53cfd4994c629a408ee8250280c5 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:35.460695 11084 raft_consensus.cc:348] T b3d9a3cf5bda4cf6bc1fa3fae1cb9288 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.461344 11084 raft_consensus.cc:374] T b3d9a3cf5bda4cf6bc1fa3fae1cb9288 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:35.461576 11084 raft_consensus.cc:729] T b3d9a3cf5bda4cf6bc1fa3fae1cb9288 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:35.460760 11085 consensus_queue.cc:260] T 8aff53cfd4994c629a408ee8250280c5 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.464800 11085 ts_tablet_manager.cc:1428] T 8aff53cfd4994c629a408ee8250280c5 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.018s user 0.006s sys 0.000s
I20260925 09:57:35.468327 11085 tablet_bootstrap.cc:492] T fb9c51f7e7fd4c029be2a710cb9462eb P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:35.474568 12379 tablet_replica.cc:330] T a21909e2e5794f57971f4e42795235c7 P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:35.475993 12080 consensus_queue.cc:1035] T 545ac025f2cb469da847979011a0b3ce P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:35.478566 12251 raft_consensus.cc:482] T f62c62715c9b42de826c1a66fd431eab P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:35.479139 12251 raft_consensus.cc:504] T f62c62715c9b42de826c1a66fd431eab P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.481261 12251 leader_election.cc:288] T f62c62715c9b42de826c1a66fd431eab P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:35.485411 11084 consensus_queue.cc:260] T b3d9a3cf5bda4cf6bc1fa3fae1cb9288 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.487195 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f62c62715c9b42de826c1a66fd431eab" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:35.487679 11084 ts_tablet_manager.cc:1428] T b3d9a3cf5bda4cf6bc1fa3fae1cb9288 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.053s user 0.006s sys 0.001s
I20260925 09:57:35.488436 11085 tablet_bootstrap.cc:654] T fb9c51f7e7fd4c029be2a710cb9462eb P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:35.488490 11084 tablet_bootstrap.cc:492] T 2b1b2d66397b44f38056077d79941ac4 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
W20260925 09:57:35.492592 10918 leader_election.cc:341] T f62c62715c9b42de826c1a66fd431eab P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:35.483590 10673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f62c62715c9b42de826c1a66fd431eab" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:35.483726 12257 raft_consensus.cc:482] T a436fe287a0d48a1821191adcd031d74 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:35.505046 12257 raft_consensus.cc:504] T a436fe287a0d48a1821191adcd031d74 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:35.505651 10919 leader_election.cc:341] T f62c62715c9b42de826c1a66fd431eab P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:35.506023 10919 leader_election.cc:302] T f62c62715c9b42de826c1a66fd431eab P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:35.506970 12207 raft_consensus.cc:2726] T f62c62715c9b42de826c1a66fd431eab P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:35.506914 12257 leader_election.cc:288] T a436fe287a0d48a1821191adcd031d74 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
W20260925 09:57:35.494990 10621 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 56164.8us
I20260925 09:57:35.514365 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab6bdba5ca5d41ae921a90c1f77006c5" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:35.518046 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a436fe287a0d48a1821191adcd031d74" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:35.528378 10668 tablet_service.cc:1514] Processing DeleteTablet for tablet 197f61525b0c4f9da441d98701d231a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:30 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:35.533660 10668 tablet_service.cc:1514] Processing DeleteTablet for tablet 60fdc299865446c89d3e5374d573029a with delete_type TABLET_DATA_DELETED (replaced by tablet 61aa6380d30a4b21b70ee2d321247bd8) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:35.547001 11084 tablet_bootstrap.cc:654] T 2b1b2d66397b44f38056077d79941ac4 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:35.483863 10619 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 66090.1us
I20260925 09:57:35.553221 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96a80382bb7d4395a14ad9ae972f4a25" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:35.553898 10976 raft_consensus.cc:3037] T 96a80382bb7d4395a14ad9ae972f4a25 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:35.555756 10980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab6bdba5ca5d41ae921a90c1f77006c5" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:35.556448 10980 raft_consensus.cc:3037] T ab6bdba5ca5d41ae921a90c1f77006c5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:35.557722 10983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a436fe287a0d48a1821191adcd031d74" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:35.558341 10983 raft_consensus.cc:2454] T a436fe287a0d48a1821191adcd031d74 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
W20260925 09:57:35.562547 10621 outbound_call.cc:322] RPC callback for RPC call kudu.master.MasterService.Ping -> {remote=127.10.9.60:36985, user_credentials={real_user=slave}} blocked reactor thread for 49242.1us
W20260925 09:57:35.566265 10621 leader_election.cc:341] T ab6bdba5ca5d41ae921a90c1f77006c5 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:35.566787 10621 leader_election.cc:341] T a436fe287a0d48a1821191adcd031d74 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:35.571705 10836 raft_consensus.cc:3037] T 8aff53cfd4994c629a408ee8250280c5 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:35.581302 11727 ts_tablet_manager.cc:1918] T 61aa6380d30a4b21b70ee2d321247bd8 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:35.581660 11727 log.cc:1196] T 61aa6380d30a4b21b70ee2d321247bd8 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/61aa6380d30a4b21b70ee2d321247bd8
I20260925 09:57:35.582293 11727 ts_tablet_manager.cc:1939] T 61aa6380d30a4b21b70ee2d321247bd8 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:35.591472 10980 raft_consensus.cc:2454] T ab6bdba5ca5d41ae921a90c1f77006c5 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:35.598757 12080 consensus_queue.cc:1035] T 8aff53cfd4994c629a408ee8250280c5 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:35.597911 10619 leader_election.cc:302] T a436fe287a0d48a1821191adcd031d74 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:35.619985 12080 raft_consensus.cc:2781] T a436fe287a0d48a1821191adcd031d74 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:35.620445 12080 raft_consensus.cc:482] T a436fe287a0d48a1821191adcd031d74 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:35.620882 12080 raft_consensus.cc:3037] T a436fe287a0d48a1821191adcd031d74 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:35.594209 10976 raft_consensus.cc:2454] T 96a80382bb7d4395a14ad9ae972f4a25 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:35.596365 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 312) took 5448 ms. Trace:
I20260925 09:57:35.625313 11727 rpcz_store.cc:276] 0925 09:57:30.148079 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:30.148212 (+ 133us) service_pool.cc:227] Handling call
0925 09:57:35.285780 (+5137568us) ts_tablet_manager.cc:1145] Deleting tablet 61aa6380d30a4b21b70ee2d321247bd8
0925 09:57:35.285835 (+ 55us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:35.363340 (+ 77505us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:35.581083 (+217743us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:35.596313 (+ 15230us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5049697}
I20260925 09:57:35.626955 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 61aa6380d30a4b21b70ee2d321247bd8 (table default.Table_32e7bdf7eaf147ea90740b6b3cfb2fd3 [id=be059d72d5db4915ba767959c6a1c7b2]) successfully deleted
I20260925 09:57:35.627189 11727 tablet_replica.cc:330] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:35.627102 12323 consensus_queue.cc:1035] T 8aff53cfd4994c629a408ee8250280c5 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:35.627806 12379 raft_consensus.cc:2229] T a21909e2e5794f57971f4e42795235c7 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:35.628289 11727 raft_consensus.cc:2229] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:35.629832 11727 raft_consensus.cc:2258] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:35.632162 12379 raft_consensus.cc:2258] T a21909e2e5794f57971f4e42795235c7 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:35.633095 11084 tablet_bootstrap.cc:492] T 2b1b2d66397b44f38056077d79941ac4 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:35.633468 11084 ts_tablet_manager.cc:1397] T 2b1b2d66397b44f38056077d79941ac4 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.145s user 0.009s sys 0.000s
I20260925 09:57:35.634653 12379 ts_tablet_manager.cc:1905] T a21909e2e5794f57971f4e42795235c7 P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:35.635764 12207 raft_consensus.cc:482] T 3d84c717b6ac462eb331e25d80c918e4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:35.635622 11084 raft_consensus.cc:348] T 2b1b2d66397b44f38056077d79941ac4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.636260 11084 raft_consensus.cc:374] T 2b1b2d66397b44f38056077d79941ac4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:35.636483 11084 raft_consensus.cc:729] T 2b1b2d66397b44f38056077d79941ac4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:35.636157 12207 raft_consensus.cc:504] T 3d84c717b6ac462eb331e25d80c918e4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.637928 12207 leader_election.cc:288] T 3d84c717b6ac462eb331e25d80c918e4 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:35.629802 10619 leader_election.cc:302] T ab6bdba5ca5d41ae921a90c1f77006c5 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:35.630128 12080 raft_consensus.cc:504] T a436fe287a0d48a1821191adcd031d74 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.640201 10673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d84c717b6ac462eb331e25d80c918e4" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:35.640224 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d84c717b6ac462eb331e25d80c918e4" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:35.641088 12080 leader_election.cc:288] T a436fe287a0d48a1821191adcd031d74 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:35.632614 11082 ts_tablet_manager.cc:1428] T aca5987b14b64152887ac5ee584b257a P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.244s user 0.035s sys 0.030s
W20260925 09:57:35.641356 10918 leader_election.cc:341] T 3d84c717b6ac462eb331e25d80c918e4 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:35.641841 10919 leader_election.cc:341] T 3d84c717b6ac462eb331e25d80c918e4 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:35.642174 10919 leader_election.cc:302] T 3d84c717b6ac462eb331e25d80c918e4 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:35.642390 11082 tablet_bootstrap.cc:492] T 61aa6380d30a4b21b70ee2d321247bd8 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:35.644855 10667 tablet_service.cc:1514] Processing DeleteTablet for tablet a44d9f2f7e3e48aeb0f31fa29ae1988a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:18 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:35.645673 12207 raft_consensus.cc:2726] T 3d84c717b6ac462eb331e25d80c918e4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:35.646624 10667 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d087d3e35814c90a03eccd90eeea86c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:30 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:35.647562 12323 raft_consensus.cc:2781] T ab6bdba5ca5d41ae921a90c1f77006c5 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:35.647994 12323 raft_consensus.cc:686] T ab6bdba5ca5d41ae921a90c1f77006c5 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:35.648967 10667 tablet_service.cc:1514] Processing DeleteTablet for tablet c7cd1d6a603642afa2c8c737079961b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:29 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:35.648747 12323 consensus_queue.cc:237] T ab6bdba5ca5d41ae921a90c1f77006c5 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.649154 11727 ts_tablet_manager.cc:1905] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:35.658239 10619 leader_election.cc:302] T 96a80382bb7d4395a14ad9ae972f4a25 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:35.659523 11084 consensus_queue.cc:260] T 2b1b2d66397b44f38056077d79941ac4 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.662535 10667 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a31b999f0fd447eacff13de71aa3fff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:23 UTC) from {username='slave'} at 127.0.0.1:37876
W20260925 09:57:35.663484 10378 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 25ac35fb43d34c9b85b8e61ad6e386a6 (table default.Table_2672598ccbce42eb8f11019f47a6fdda [id=887ce3cafef64210bb69d424ed5dae8a], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 25ac35fb43d34c9b85b8e61ad6e386a6 consensus state has no leader
I20260925 09:57:35.665414 11084 ts_tablet_manager.cc:1428] T 2b1b2d66397b44f38056077d79941ac4 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.032s user 0.007s sys 0.000s
I20260925 09:57:35.666674 11082 tablet_bootstrap.cc:654] T 61aa6380d30a4b21b70ee2d321247bd8 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:35.671502 10952 tablet_service.cc:1514] Processing DeleteTablet for tablet 61aa6380d30a4b21b70ee2d321247bd8 with delete_type TABLET_DATA_DELETED (replaced by tablet 77ebb59faf3e4fb187bbce29612c25ea) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:35.672327 10952 tablet_service.cc:1514] Processing DeleteTablet for tablet 92814437d53143a79f99199f8ee3a742 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:29 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:35.673110 10952 tablet_service.cc:1514] Processing DeleteTablet for tablet 417058493c6547c69e72347cef373292 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:25 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:35.675683 10668 tablet_service.cc:1514] Processing DeleteTablet for tablet 6161774f61ae48ab9764a05f2188a80b with delete_type TABLET_DATA_DELETED (replaced by tablet 10657a81058241fc988d37e5ea7ff8cf) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:35.675808 11084 tablet_bootstrap.cc:492] T 88e53debe6de43e29b12e85e5171c8b5 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:35.687413 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a436fe287a0d48a1821191adcd031d74" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:35.688158 10976 raft_consensus.cc:3037] T a436fe287a0d48a1821191adcd031d74 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:35.691780 10824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a436fe287a0d48a1821191adcd031d74" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:35.698938 12027 ts_tablet_manager.cc:1918] T 2d087d3e35814c90a03eccd90eeea86c P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:35.669688 10653 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e3120347c8e4a3eab57d6562ae40e8d with delete_type TABLET_DATA_DELETED (replaced by tablet c52745a8cdee461aba7485b0531245d6) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:35.662549 12390 raft_consensus.cc:2781] T 96a80382bb7d4395a14ad9ae972f4a25 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:35.679443 12379 ts_tablet_manager.cc:1918] T a21909e2e5794f57971f4e42795235c7 P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260925 09:57:35.680799 10621 consensus_peers.cc:487] T 6161774f61ae48ab9764a05f2188a80b P b8850818e2ee42a082802d0095c08d21 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6161774f61ae48ab9764a05f2188a80b. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:35.699645 12379 log.cc:1196] T a21909e2e5794f57971f4e42795235c7 P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/a21909e2e5794f57971f4e42795235c7
I20260925 09:57:35.699674 12390 raft_consensus.cc:686] T 96a80382bb7d4395a14ad9ae972f4a25 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:35.700492 12379 ts_tablet_manager.cc:1939] T a21909e2e5794f57971f4e42795235c7 P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:35.687240 11082 tablet_bootstrap.cc:492] T 61aa6380d30a4b21b70ee2d321247bd8 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:35.701274 11082 ts_tablet_manager.cc:1397] T 61aa6380d30a4b21b70ee2d321247bd8 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.059s user 0.010s sys 0.000s
I20260925 09:57:35.700721 12390 consensus_queue.cc:237] T 96a80382bb7d4395a14ad9ae972f4a25 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.703339 11082 raft_consensus.cc:348] T 61aa6380d30a4b21b70ee2d321247bd8 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.705086 11082 raft_consensus.cc:374] T 61aa6380d30a4b21b70ee2d321247bd8 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:35.705349 11082 raft_consensus.cc:729] T 61aa6380d30a4b21b70ee2d321247bd8 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:35.705870 11082 consensus_queue.cc:260] T 61aa6380d30a4b21b70ee2d321247bd8 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.706612 12379 tablet_replica.cc:330] T 93e1b5837aff48d7907890c40dfcd0d0 P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:35.708190 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet a21909e2e5794f57971f4e42795235c7 (table default.Table_48f8cb39156845c8bffbb2f9f17a900b [id=093bf6e622054652ad406255d557d0c8]) successfully deleted
I20260925 09:57:35.707350 11082 ts_tablet_manager.cc:1428] T 61aa6380d30a4b21b70ee2d321247bd8 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260925 09:57:35.707806 12379 raft_consensus.cc:2229] T 93e1b5837aff48d7907890c40dfcd0d0 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:35.709625 12379 raft_consensus.cc:2258] T 93e1b5837aff48d7907890c40dfcd0d0 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:35.710726 11082 tablet_bootstrap.cc:492] T 350904aa5957496d90a2aa2e637b591c P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:35.713387 10653 tablet_service.cc:1514] Processing DeleteTablet for tablet 545ac025f2cb469da847979011a0b3ce with delete_type TABLET_DATA_DELETED (replaced by tablet ab6bdba5ca5d41ae921a90c1f77006c5) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:35.714152 12379 ts_tablet_manager.cc:1905] T 93e1b5837aff48d7907890c40dfcd0d0 P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:35.719554 11084 tablet_bootstrap.cc:654] T 88e53debe6de43e29b12e85e5171c8b5 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:35.722409 10621 leader_election.cc:341] T a436fe287a0d48a1821191adcd031d74 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:35.727295 11082 tablet_bootstrap.cc:654] T 350904aa5957496d90a2aa2e637b591c P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:35.734812 10393 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.1:56563 (request call id 40) took 3432 ms (client timeout 1000 ms). Trace:
I20260925 09:57:35.735881 10668 tablet_service.cc:1514] Processing DeleteTablet for tablet 417058493c6547c69e72347cef373292 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:25 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:35.744618 10952 tablet_service.cc:1514] Processing DeleteTablet for tablet b555eeba505740988c704df5c734e7c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:26 UTC) from {username='slave'} at 127.0.0.1:53560
W20260925 09:57:35.749117 10390 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.1:56563 (request call id 39) took 4482 ms (client timeout 1000 ms). Trace:
I20260925 09:57:35.754086 10976 raft_consensus.cc:2454] T a436fe287a0d48a1821191adcd031d74 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:35.754559 11084 tablet_bootstrap.cc:492] T 88e53debe6de43e29b12e85e5171c8b5 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:35.755437 11084 ts_tablet_manager.cc:1397] T 88e53debe6de43e29b12e85e5171c8b5 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.080s user 0.009s sys 0.003s
I20260925 09:57:35.755581 11727 ts_tablet_manager.cc:1918] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:35.756310 11727 log.cc:1196] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/c7cd1d6a603642afa2c8c737079961b5
I20260925 09:57:35.757112 11727 ts_tablet_manager.cc:1939] T c7cd1d6a603642afa2c8c737079961b5 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:35.758177 10668 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ff4cf556eee494d8cd590884466f09f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:29 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:35.760509 11082 tablet_bootstrap.cc:492] T 350904aa5957496d90a2aa2e637b591c P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:35.760980 11082 ts_tablet_manager.cc:1397] T 350904aa5957496d90a2aa2e637b591c P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.052s user 0.010s sys 0.002s
I20260925 09:57:35.757779 11084 raft_consensus.cc:348] T 88e53debe6de43e29b12e85e5171c8b5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.763965 11084 raft_consensus.cc:374] T 88e53debe6de43e29b12e85e5171c8b5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260925 09:57:35.731537 10892 tablet.cc:2369] T 8aff53cfd4994c629a408ee8250280c5 P c43461c4ee854ae1ac52b873ccada318: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:57:35.767648 12027 log.cc:1196] T 2d087d3e35814c90a03eccd90eeea86c P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/2d087d3e35814c90a03eccd90eeea86c
I20260925 09:57:35.768296 12027 ts_tablet_manager.cc:1939] T 2d087d3e35814c90a03eccd90eeea86c P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:35.770395 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 324) took 5130 ms. Trace:
I20260925 09:57:35.733140 11085 tablet_bootstrap.cc:492] T fb9c51f7e7fd4c029be2a710cb9462eb P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:35.771109 11085 ts_tablet_manager.cc:1397] T fb9c51f7e7fd4c029be2a710cb9462eb P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.303s user 0.029s sys 0.061s
I20260925 09:57:35.773247 11085 raft_consensus.cc:348] T fb9c51f7e7fd4c029be2a710cb9462eb P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:35.773573 10619 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 44547.7us
I20260925 09:57:35.773859 11085 raft_consensus.cc:374] T fb9c51f7e7fd4c029be2a710cb9462eb P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:35.774072 11085 raft_consensus.cc:729] T fb9c51f7e7fd4c029be2a710cb9462eb P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:35.774572 11085 consensus_queue.cc:260] T fb9c51f7e7fd4c029be2a710cb9462eb P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.780478 10619 leader_election.cc:302] T a436fe287a0d48a1821191adcd031d74 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:35.784583 10830 raft_consensus.cc:3037] T fb9c51f7e7fd4c029be2a710cb9462eb P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:35.788686 11084 raft_consensus.cc:729] T 88e53debe6de43e29b12e85e5171c8b5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:35.789449 11084 consensus_queue.cc:260] T 88e53debe6de43e29b12e85e5171c8b5 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.763322 11082 raft_consensus.cc:348] T 350904aa5957496d90a2aa2e637b591c P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.794934 11082 raft_consensus.cc:374] T 350904aa5957496d90a2aa2e637b591c P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:35.795459 11082 raft_consensus.cc:729] T 350904aa5957496d90a2aa2e637b591c P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:35.797901 11084 ts_tablet_manager.cc:1428] T 88e53debe6de43e29b12e85e5171c8b5 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.042s user 0.004s sys 0.001s
W20260925 09:57:35.735189 10393 rpcz_store.cc:269] 0925 09:57:32.302747 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:32.303994 (+ 1247us) service_pool.cc:227] Handling call
0925 09:57:35.734631 (+3430637us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:33.308973 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:33.309324 (+ 351us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:33.309345 (+ 21us) write_op.cc:435] Acquired schema lock
0925 09:57:33.309357 (+ 12us) tablet.cc:660] Decoding operations
0925 09:57:33.309776 (+ 419us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:33.309812 (+ 36us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:33.309833 (+ 21us) tablet.cc:683] Acquiring locks for 4 operations
0925 09:57:33.310118 (+ 285us) tablet.cc:699] Row locks acquired
0925 09:57:33.310131 (+ 13us) write_op.cc:260] PREPARE: finished
0925 09:57:33.310246 (+ 115us) write_op.cc:270] Start()
0925 09:57:33.310325 (+ 79us) write_op.cc:276] Timestamp: P: 1790330253310202 usec, L: 0
0925 09:57:33.310340 (+ 15us) op_driver.cc:331] REPLICATION: starting
0925 09:57:33.310841 (+ 501us) log.cc:842] Serialized 1638 byte log entry
0925 09:57:33.342122 (+ 31281us) op_driver.cc:444] REPLICATION: finished
0925 09:57:33.344146 (+ 2024us) write_op.cc:301] APPLY: starting
0925 09:57:33.344192 (+ 46us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:33.344337 (+ 145us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:33.344426 (+ 89us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:33.344811 (+ 385us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:33.344948 (+ 137us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0925 09:57:33.344985 (+ 37us) write_op.cc:312] APPLY: finished
0925 09:57:33.345149 (+ 164us) log.cc:842] Serialized 41 byte log entry
0925 09:57:33.345282 (+ 133us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:33.345355 (+ 73us) write_op.cc:454] Released schema lock
0925 09:57:33.345389 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:33.346542 (+ 1153us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:33.346565 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":20757,"reported_tablets":32,"child_traces":[["op",{"apply.queue_time_us":1905,"apply.run_cpu_time_us":1612,"apply.run_wall_time_us":2596,"num_ops":4,"prepare.queue_time_us":87,"prepare.run_cpu_time_us":2067,"prepare.run_wall_time_us":2067,"replication_time_us":31717}]]}
I20260925 09:57:35.808821 10952 tablet_service.cc:1514] Processing DeleteTablet for tablet aba253f22afa4200926f31a1b43e3970 with delete_type TABLET_DATA_DELETED (replaced by tablet 815b4b1bd914433798dcb5deec6ec79c) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:35.809535 10956 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ff4cf556eee494d8cd590884466f09f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:29 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:35.811714 10956 tablet_service.cc:1514] Processing DeleteTablet for tablet cf775c9d9b114ddcb97dc6c6eca8e0fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:26 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:35.813207 10952 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a73a1f242e7430cbf52229c07b4dc82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:26 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:35.813267 10956 tablet_service.cc:1514] Processing DeleteTablet for tablet 37bab60afe7a4c758a08ef7c9c24aba0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:26 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:35.822373 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 306) took 5766 ms. Trace:
I20260925 09:57:35.822793 11727 rpcz_store.cc:276] 0925 09:57:30.055493 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:30.059636 (+ 4143us) service_pool.cc:227] Handling call
0925 09:57:35.627042 (+5567406us) ts_tablet_manager.cc:1145] Deleting tablet c7cd1d6a603642afa2c8c737079961b5
0925 09:57:35.627091 (+ 49us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:35.724141 (+ 97050us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:35.755361 (+ 31220us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:35.822176 (+ 66815us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5378056}
W20260925 09:57:35.824466 10619 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 44080.8us
I20260925 09:57:35.824688 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 313) took 5645 ms. Trace:
I20260925 09:57:35.826105 11727 rpcz_store.cc:276] 0925 09:57:30.179256 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:30.219621 (+ 40365us) service_pool.cc:227] Handling call
0925 09:57:35.824324 (+5604703us) ts_tablet_manager.cc:1145] Deleting tablet 815b4b1bd914433798dcb5deec6ec79c
0925 09:57:35.824373 (+ 49us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:35.824640 (+ 267us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5558286}
I20260925 09:57:35.827797 11084 tablet_bootstrap.cc:492] T dbc7c7b9a2ee437eabe10150bdbc2e4a P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
W20260925 09:57:35.829955 11036 tablet.cc:2369] T 2b1b2d66397b44f38056077d79941ac4 P be97ab5d75934a648d4ab76341193ec0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:57:35.828732 11727 tablet_replica.cc:330] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:35.836222 12322 raft_consensus.cc:2781] T a436fe287a0d48a1821191adcd031d74 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:35.836670 12322 raft_consensus.cc:686] T a436fe287a0d48a1821191adcd031d74 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:35.823902 11082 consensus_queue.cc:260] T 350904aa5957496d90a2aa2e637b591c P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.837416 12322 consensus_queue.cc:237] T a436fe287a0d48a1821191adcd031d74 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:35.749441 10390 rpcz_store.cc:269] 0925 09:57:31.266686 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:31.360926 (+ 94240us) service_pool.cc:227] Handling call
0925 09:57:35.748908 (+4387982us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:32.736677 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:32.737040 (+ 363us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:32.737065 (+ 25us) write_op.cc:435] Acquired schema lock
0925 09:57:32.737077 (+ 12us) tablet.cc:660] Decoding operations
0925 09:57:32.737397 (+ 320us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:32.737435 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:32.737455 (+ 20us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:32.737725 (+ 270us) tablet.cc:699] Row locks acquired
0925 09:57:32.737738 (+ 13us) write_op.cc:260] PREPARE: finished
0925 09:57:32.737852 (+ 114us) write_op.cc:270] Start()
0925 09:57:32.737928 (+ 76us) write_op.cc:276] Timestamp: P: 1790330252737806 usec, L: 0
0925 09:57:32.737942 (+ 14us) op_driver.cc:331] REPLICATION: starting
0925 09:57:32.738441 (+ 499us) log.cc:842] Serialized 895 byte log entry
0925 09:57:32.749657 (+ 11216us) spinlock_profiling.cc:243] Waited 7.74 ms on lock 0x7b58000b03b0. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0a85834 00007fabf0a8a771 00007fabf0a8a6f9 00007fabf0a8a689 00007fabf0a8a651 00007fabf0a8981d 0000000000580784 00000000005805b9 00007fabe9bb66b6 00007fabe9bb9b11
0925 09:57:32.749900 (+ 243us) op_driver.cc:444] REPLICATION: finished
0925 09:57:32.750123 (+ 223us) write_op.cc:301] APPLY: starting
0925 09:57:32.750203 (+ 80us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:32.750308 (+ 105us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:32.750326 (+ 18us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:32.750569 (+ 243us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:32.750687 (+ 118us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 09:57:32.750730 (+ 43us) write_op.cc:312] APPLY: finished
0925 09:57:32.750895 (+ 165us) log.cc:842] Serialized 29 byte log entry
0925 09:57:32.751472 (+ 577us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:32.751540 (+ 68us) write_op.cc:454] Released schema lock
0925 09:57:32.751575 (+ 35us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:32.751733 (+ 158us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:32.751755 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":32,"child_traces":[["op",{"apply.queue_time_us":79,"apply.run_cpu_time_us":1389,"apply.run_wall_time_us":1813,"num_ops":2,"prepare.queue_time_us":130498,"prepare.run_cpu_time_us":2066,"prepare.run_wall_time_us":2066,"raft.queue_time_us":3963,"raft.run_cpu_time_us":2206,"raft.run_wall_time_us":9836,"replication_time_us":11876,"spinlock_wait_cycles":17805056}]]}
I20260925 09:57:35.823774 12391 raft_consensus.cc:482] T aca5987b14b64152887ac5ee584b257a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:35.775468 12027 rpcz_store.cc:276] 0925 09:57:30.640192 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:30.640292 (+ 100us) service_pool.cc:227] Handling call
0925 09:57:35.311845 (+4671553us) ts_tablet_manager.cc:1145] Deleting tablet 2d087d3e35814c90a03eccd90eeea86c
0925 09:57:35.311905 (+ 60us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:35.517462 (+205557us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:35.698699 (+181237us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:35.770286 (+ 71587us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4671054}
I20260925 09:57:35.840724 12391 raft_consensus.cc:504] T aca5987b14b64152887ac5ee584b257a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.842011 12027 tablet_replica.cc:330] T 197f61525b0c4f9da441d98701d231a3 P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:35.842764 12391 leader_election.cc:288] T aca5987b14b64152887ac5ee584b257a P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:35.846233 11082 ts_tablet_manager.cc:1428] T 350904aa5957496d90a2aa2e637b591c P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.085s user 0.006s sys 0.000s
I20260925 09:57:35.846393 11085 ts_tablet_manager.cc:1428] T fb9c51f7e7fd4c029be2a710cb9462eb P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.075s user 0.004s sys 0.000s
I20260925 09:57:35.847172 11082 tablet_bootstrap.cc:492] T f62c62715c9b42de826c1a66fd431eab P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:35.848860 12379 ts_tablet_manager.cc:1918] T 93e1b5837aff48d7907890c40dfcd0d0 P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:57:35.849262 12379 log.cc:1196] T 93e1b5837aff48d7907890c40dfcd0d0 P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/93e1b5837aff48d7907890c40dfcd0d0
I20260925 09:57:35.850005 12379 ts_tablet_manager.cc:1939] T 93e1b5837aff48d7907890c40dfcd0d0 P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:35.853489 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aca5987b14b64152887ac5ee584b257a" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:35.853763 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 2d087d3e35814c90a03eccd90eeea86c (table default.Table_bcf18c926c6e4a62b31f3bffe9b0037e [id=53df4f0ffd794afab1aa17fcfb4d56bc]) successfully deleted
I20260925 09:57:35.854110 10976 raft_consensus.cc:2454] T aca5987b14b64152887ac5ee584b257a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:35.855115 12390 raft_consensus.cc:482] T 92814437d53143a79f99199f8ee3a742 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:35.855535 12390 raft_consensus.cc:504] T 92814437d53143a79f99199f8ee3a742 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:35.813702 10388 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.3:41469 (request call id 28) took 3685 ms (client timeout 1000 ms). Trace:
I20260925 09:57:35.825126 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet c7cd1d6a603642afa2c8c737079961b5 (table default.Table_0507f47042b9445abaf8671c979f8994 [id=55860c5dba9f4c818cfa6d523041ec5f]) successfully deleted
I20260925 09:57:35.856701 12027 raft_consensus.cc:2229] T 197f61525b0c4f9da441d98701d231a3 P c43461c4ee854ae1ac52b873ccada318 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:35.857167 12027 raft_consensus.cc:2258] T 197f61525b0c4f9da441d98701d231a3 P c43461c4ee854ae1ac52b873ccada318 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:35.857684 12390 leader_election.cc:288] T 92814437d53143a79f99199f8ee3a742 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:35.859022 12027 ts_tablet_manager.cc:1905] T 197f61525b0c4f9da441d98701d231a3 P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 09:57:35.858952 10377 catalog_manager.cc:4821] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 815b4b1bd914433798dcb5deec6ec79c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 815b4b1bd914433798dcb5deec6ec79c already in progress: creating tablet
W20260925 09:57:35.859737 10388 rpcz_store.cc:269] 0925 09:57:32.128419 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:32.128775 (+ 356us) service_pool.cc:227] Handling call
0925 09:57:35.813624 (+3684849us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":64}
I20260925 09:57:35.864775 10619 leader_election.cc:302] T aca5987b14b64152887ac5ee584b257a P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
W20260925 09:57:35.865691 10619 leader_election.cc:341] T 92814437d53143a79f99199f8ee3a742 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 92814437d53143a79f99199f8ee3a742
I20260925 09:57:35.866161 12392 raft_consensus.cc:2781] T aca5987b14b64152887ac5ee584b257a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:35.866640 12392 raft_consensus.cc:482] T aca5987b14b64152887ac5ee584b257a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:35.867079 12392 raft_consensus.cc:3037] T aca5987b14b64152887ac5ee584b257a P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:35.873122 11084 tablet_bootstrap.cc:654] T dbc7c7b9a2ee437eabe10150bdbc2e4a P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:35.875281 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 93e1b5837aff48d7907890c40dfcd0d0 (table default.Table_f8b067e4e4cf498bbf44d7bf24c29363 [id=23d14ed1d2d647fc8b3b5ef40c7dc0bb]) successfully deleted
I20260925 09:57:35.875432 10390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_91c768bb83ce44d88f0320b446d6d74a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:35.877791 10390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_91c768bb83ce44d88f0320b446d6d74a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 09:57:35.883028 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 197f61525b0c4f9da441d98701d231a3 because the tablet was not found. No further retry: Not found: Tablet not found: 197f61525b0c4f9da441d98701d231a3
I20260925 09:57:35.884037 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92814437d53143a79f99199f8ee3a742" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:35.885195 10832 raft_consensus.cc:3037] T 6540aba5854a44419addc5d454e473b7 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:57:35.886708 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 60fdc299865446c89d3e5374d573029a because the tablet was not found. No further retry: Not found: Tablet not found: 60fdc299865446c89d3e5374d573029a
W20260925 09:57:35.890579 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet a44d9f2f7e3e48aeb0f31fa29ae1988a because the tablet was not found. No further retry: Not found: Tablet not found: a44d9f2f7e3e48aeb0f31fa29ae1988a
W20260925 09:57:35.895674 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 2d087d3e35814c90a03eccd90eeea86c because the tablet was not found. No further retry: Not found: Tablet not found: 2d087d3e35814c90a03eccd90eeea86c
I20260925 09:57:35.897181 10830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aca5987b14b64152887ac5ee584b257a" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:35.899314 11085 tablet_bootstrap.cc:492] T a5d872c3620646e5ac33ceb95de2cbd4 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
W20260925 09:57:35.901070 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet c7cd1d6a603642afa2c8c737079961b5 because the tablet was not found. No further retry: Not found: Tablet not found: c7cd1d6a603642afa2c8c737079961b5
I20260925 09:57:35.902040 11727 raft_consensus.cc:2229] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:35.902547 11727 raft_consensus.cc:2258] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:35.904402 11727 ts_tablet_manager.cc:1905] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:35.909634 12424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.9.60:36985 every 8 connection(s)
I20260925 09:57:35.910274 11435 rpc_server.cc:307] RPC server started. Bound to: 127.10.9.60:36985
I20260925 09:57:35.914757 11435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/master-2/data/info.pb
I20260925 09:57:35.918546 12379 tablet_replica.cc:330] T 7a31b999f0fd447eacff13de71aa3fff P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:35.919607 12379 raft_consensus.cc:2229] T 7a31b999f0fd447eacff13de71aa3fff P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:35.920142 12379 raft_consensus.cc:2258] T 7a31b999f0fd447eacff13de71aa3fff P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:35.922003 12379 ts_tablet_manager.cc:1905] T 7a31b999f0fd447eacff13de71aa3fff P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:35.922493 10276 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu as pid 11435
I20260925 09:57:35.925661 11084 tablet_bootstrap.cc:492] T dbc7c7b9a2ee437eabe10150bdbc2e4a P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:35.926080 11084 ts_tablet_manager.cc:1397] T dbc7c7b9a2ee437eabe10150bdbc2e4a P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.098s user 0.008s sys 0.000s
I20260925 09:57:35.926685 10388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_22ca44b09d854c16beb77aad966419d0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:35.928340 11084 raft_consensus.cc:348] T dbc7c7b9a2ee437eabe10150bdbc2e4a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.928960 11084 raft_consensus.cc:374] T dbc7c7b9a2ee437eabe10150bdbc2e4a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:35.929194 11084 raft_consensus.cc:729] T dbc7c7b9a2ee437eabe10150bdbc2e4a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
W20260925 09:57:35.929533 10388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_22ca44b09d854c16beb77aad966419d0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:35.929872 11084 consensus_queue.cc:260] T dbc7c7b9a2ee437eabe10150bdbc2e4a P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.933799 11084 ts_tablet_manager.cc:1428] T dbc7c7b9a2ee437eabe10150bdbc2e4a P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260925 09:57:35.934691 11084 tablet_bootstrap.cc:492] T f3bba45d6b7241f9a07907c79b892f7d P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
W20260925 09:57:35.941797 10621 leader_election.cc:341] T 92814437d53143a79f99199f8ee3a742 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:35.942359 10621 leader_election.cc:302] T 92814437d53143a79f99199f8ee3a742 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8850818e2ee42a082802d0095c08d21; no voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:35.943825 12322 raft_consensus.cc:2726] T 92814437d53143a79f99199f8ee3a742 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:35.958391 10621 leader_election.cc:341] T aca5987b14b64152887ac5ee584b257a P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:35.963718 11082 tablet_bootstrap.cc:700] T f62c62715c9b42de826c1a66fd431eab P b8850818e2ee42a082802d0095c08d21: Time spent opening tablet: real 0.105s user 0.002s sys 0.000s
I20260925 09:57:35.964923 11082 tablet_bootstrap.cc:654] T f62c62715c9b42de826c1a66fd431eab P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:35.973745 12207 raft_consensus.cc:482] T 9744b9814b69470396ce039348fe5e45 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:35.974290 12207 raft_consensus.cc:504] T 9744b9814b69470396ce039348fe5e45 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.977948 10673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9744b9814b69470396ce039348fe5e45" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:35.979584 10919 leader_election.cc:341] T 9744b9814b69470396ce039348fe5e45 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:35.980859 12207 leader_election.cc:288] T 9744b9814b69470396ce039348fe5e45 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:35.982610 10830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9744b9814b69470396ce039348fe5e45" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:35.987756 10918 leader_election.cc:341] T 9744b9814b69470396ce039348fe5e45 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:35.988132 10918 leader_election.cc:302] T 9744b9814b69470396ce039348fe5e45 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:35.989549 12251 raft_consensus.cc:2726] T 9744b9814b69470396ce039348fe5e45 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:35.995558 12322 raft_consensus.cc:482] T 61aa6380d30a4b21b70ee2d321247bd8 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:35.996169 12322 raft_consensus.cc:504] T 61aa6380d30a4b21b70ee2d321247bd8 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:35.996654 11085 tablet_bootstrap.cc:654] T a5d872c3620646e5ac33ceb95de2cbd4 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:36.003190 10775 consensus_peers.cc:487] T 7a31b999f0fd447eacff13de71aa3fff P c43461c4ee854ae1ac52b873ccada318 -> Peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Couldn't send request to peer b8850818e2ee42a082802d0095c08d21. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:36.004323 12365 raft_consensus.cc:482] T cfa619b699004882a82976f0ef8204ad P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b8850818e2ee42a082802d0095c08d21)
I20260925 09:57:36.005332 12365 raft_consensus.cc:504] T cfa619b699004882a82976f0ef8204ad P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:36.008005 12365 leader_election.cc:288] T cfa619b699004882a82976f0ef8204ad P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:36.011482 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa619b699004882a82976f0ef8204ad" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:36.014024 12322 leader_election.cc:288] T 61aa6380d30a4b21b70ee2d321247bd8 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
W20260925 09:57:36.016171 10775 leader_election.cc:341] T cfa619b699004882a82976f0ef8204ad P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Not found: Tablet not found: cfa619b699004882a82976f0ef8204ad
I20260925 09:57:36.016515 10775 leader_election.cc:302] T cfa619b699004882a82976f0ef8204ad P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:36.017953 12365 raft_consensus.cc:2726] T cfa619b699004882a82976f0ef8204ad P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 09:57:36.020274 10619 leader_election.cc:341] T 61aa6380d30a4b21b70ee2d321247bd8 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 61aa6380d30a4b21b70ee2d321247bd8
I20260925 09:57:36.025799 11727 ts_tablet_manager.cc:1918] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:36.026160 11727 log.cc:1196] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/2d087d3e35814c90a03eccd90eeea86c
I20260925 09:57:36.026847 11727 ts_tablet_manager.cc:1939] T 2d087d3e35814c90a03eccd90eeea86c P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:36.029254 12027 ts_tablet_manager.cc:1918] T 197f61525b0c4f9da441d98701d231a3 P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:36.029587 12027 log.cc:1196] T 197f61525b0c4f9da441d98701d231a3 P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/197f61525b0c4f9da441d98701d231a3
I20260925 09:57:36.030292 12027 ts_tablet_manager.cc:1939] T 197f61525b0c4f9da441d98701d231a3 P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:36.031091 10830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61aa6380d30a4b21b70ee2d321247bd8" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:36.035647 10621 leader_election.cc:341] T 61aa6380d30a4b21b70ee2d321247bd8 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:36.036083 10621 leader_election.cc:302] T 61aa6380d30a4b21b70ee2d321247bd8 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8850818e2ee42a082802d0095c08d21; no voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:36.039454 10387 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 4c3e66f6ad23475f9fb831afc1d579c4
I20260925 09:57:36.039564 12322 raft_consensus.cc:2726] T 61aa6380d30a4b21b70ee2d321247bd8 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:36.041116 10668 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c3e66f6ad23475f9fb831afc1d579c4 with delete_type TABLET_DATA_DELETED (replaced by tablet 7a92074fdf254af2b52b39e41ff51689) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:36.041939 10387 catalog_manager.cc:6565] ReplaceTablet: tablet 4c3e66f6ad23475f9fb831afc1d579c4 deleted and replaced by tablet 7a92074fdf254af2b52b39e41ff51689
I20260925 09:57:36.047601 10387 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:35140 (request call id 365) took 2746 ms. Trace:
I20260925 09:57:36.047689 10809 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c3e66f6ad23475f9fb831afc1d579c4 with delete_type TABLET_DATA_DELETED (replaced by tablet 7a92074fdf254af2b52b39e41ff51689) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:36.048066 10387 rpcz_store.cc:276] 0925 09:57:33.300966 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:35.384913 (+2083947us) service_pool.cc:227] Handling call
0925 09:57:36.047541 (+662628us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:35.688322 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:35.688782 (+ 460us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:35.688807 (+ 25us) write_op.cc:435] Acquired schema lock
0925 09:57:35.688821 (+ 14us) tablet.cc:660] Decoding operations
0925 09:57:35.710027 (+ 21206us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:35.710076 (+ 49us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:35.710108 (+ 32us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:35.710345 (+ 237us) tablet.cc:699] Row locks acquired
0925 09:57:35.710358 (+ 13us) write_op.cc:260] PREPARE: finished
0925 09:57:35.715659 (+ 5301us) write_op.cc:270] Start()
0925 09:57:35.715760 (+ 101us) write_op.cc:276] Timestamp: P: 1790330255715606 usec, L: 0
0925 09:57:35.715774 (+ 14us) op_driver.cc:331] REPLICATION: starting
0925 09:57:35.716299 (+ 525us) log.cc:842] Serialized 619 byte log entry
0925 09:57:35.880448 (+164149us) op_driver.cc:444] REPLICATION: finished
0925 09:57:35.915823 (+ 35375us) write_op.cc:301] APPLY: starting
0925 09:57:35.915897 (+ 74us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:35.916029 (+ 132us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:35.916146 (+ 117us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:35.916521 (+ 375us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:35.916731 (+ 210us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:35.916772 (+ 41us) write_op.cc:312] APPLY: finished
0925 09:57:35.916973 (+ 201us) log.cc:842] Serialized 29 byte log entry
0925 09:57:35.917576 (+ 603us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:35.917639 (+ 63us) write_op.cc:454] Released schema lock
0925 09:57:35.917669 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:35.917866 (+ 197us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:35.917890 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":35131,"apply.run_cpu_time_us":1899,"apply.run_wall_time_us":2324,"num_ops":2,"prepare.queue_time_us":298944,"prepare.run_cpu_time_us":2285,"prepare.run_wall_time_us":49677,"replication_time_us":164591,"spinlock_wait_cycles":8960,"thread_start_us":34870,"threads_started":1}]]}
I20260925 09:57:35.968483 12392 raft_consensus.cc:504] T aca5987b14b64152887ac5ee584b257a P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:36.058772 12392 leader_election.cc:288] T aca5987b14b64152887ac5ee584b257a P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:36.073308 11082 tablet_bootstrap.cc:492] T f62c62715c9b42de826c1a66fd431eab P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:36.073738 11082 ts_tablet_manager.cc:1397] T f62c62715c9b42de826c1a66fd431eab P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.227s user 0.006s sys 0.003s
I20260925 09:57:36.075697 10830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aca5987b14b64152887ac5ee584b257a" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
W20260925 09:57:36.077446 10621 leader_election.cc:341] T aca5987b14b64152887ac5ee584b257a P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:36.085914 11082 raft_consensus.cc:348] T f62c62715c9b42de826c1a66fd431eab P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:36.086853 11082 raft_consensus.cc:374] T f62c62715c9b42de826c1a66fd431eab P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:36.087158 11082 raft_consensus.cc:729] T f62c62715c9b42de826c1a66fd431eab P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:36.089493 10956 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c3e66f6ad23475f9fb831afc1d579c4 with delete_type TABLET_DATA_DELETED (replaced by tablet 7a92074fdf254af2b52b39e41ff51689) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:36.092270 10978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aca5987b14b64152887ac5ee584b257a" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:36.092859 10978 raft_consensus.cc:3037] T aca5987b14b64152887ac5ee584b257a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:36.095297 11085 tablet_bootstrap.cc:492] T a5d872c3620646e5ac33ceb95de2cbd4 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:36.098444 11085 ts_tablet_manager.cc:1397] T a5d872c3620646e5ac33ceb95de2cbd4 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.199s user 0.010s sys 0.000s
I20260925 09:57:36.124948 11082 consensus_queue.cc:260] T f62c62715c9b42de826c1a66fd431eab P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:36.133283 11082 ts_tablet_manager.cc:1428] T f62c62715c9b42de826c1a66fd431eab P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.059s user 0.008s sys 0.000s
I20260925 09:57:36.140051 11082 tablet_bootstrap.cc:492] T 69028aa7908c45b896da4ba8deef07b1 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:36.142408 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 328) took 5336 ms. Trace:
I20260925 09:57:36.142802 12027 rpcz_store.cc:276] 0925 09:57:30.805585 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:30.827659 (+ 22074us) service_pool.cc:227] Handling call
0925 09:57:35.841830 (+5014171us) ts_tablet_manager.cc:1145] Deleting tablet 197f61525b0c4f9da441d98701d231a3
0925 09:57:35.841881 (+ 51us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:35.935657 (+ 93776us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:36.029046 (+ 93389us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:36.142350 (+113304us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":21930,"tablet-delete.queue_time_us":4976663}
I20260925 09:57:36.144681 11085 raft_consensus.cc:348] T a5d872c3620646e5ac33ceb95de2cbd4 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:36.145135 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 329) took 4608 ms. Trace:
I20260925 09:57:36.155375 11084 tablet_bootstrap.cc:654] T f3bba45d6b7241f9a07907c79b892f7d P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:36.156498 10978 raft_consensus.cc:2454] T aca5987b14b64152887ac5ee584b257a P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:36.157775 10619 leader_election.cc:302] T aca5987b14b64152887ac5ee584b257a P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:36.158031 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 321) took 5561 ms. Trace:
I20260925 09:57:36.158514 11727 rpcz_store.cc:276] 0925 09:57:30.596896 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:30.616033 (+ 19137us) service_pool.cc:227] Handling call
0925 09:57:35.828472 (+5212439us) ts_tablet_manager.cc:1145] Deleting tablet 2d087d3e35814c90a03eccd90eeea86c
0925 09:57:35.828521 (+ 49us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:35.911864 (+ 83343us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:36.025564 (+113700us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:36.157980 (+132416us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":66431,"tablet-delete.queue_time_us":5200204}
I20260925 09:57:36.159552 12392 raft_consensus.cc:2781] T aca5987b14b64152887ac5ee584b257a P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:36.159715 11727 tablet_replica.cc:330] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:36.159996 12392 raft_consensus.cc:686] T aca5987b14b64152887ac5ee584b257a P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:36.160657 11727 raft_consensus.cc:2229] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0 [term 2 LEADER]: Raft consensus shutting down.
I20260925 09:57:36.160786 12392 consensus_queue.cc:237] T aca5987b14b64152887ac5ee584b257a P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:36.165830 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 2d087d3e35814c90a03eccd90eeea86c (table default.Table_bcf18c926c6e4a62b31f3bffe9b0037e [id=53df4f0ffd794afab1aa17fcfb4d56bc]) successfully deleted
I20260925 09:57:36.186517 11727 pending_rounds.cc:62] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0: Trying to abort 1 pending ops.
I20260925 09:57:36.187211 10734 heartbeater.cc:347] Connected to a master server at 127.10.9.60:36985
I20260925 09:57:36.188890 11084 tablet_bootstrap.cc:492] T f3bba45d6b7241f9a07907c79b892f7d P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:36.189268 11084 ts_tablet_manager.cc:1397] T f3bba45d6b7241f9a07907c79b892f7d P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.255s user 0.009s sys 0.004s
I20260925 09:57:36.186941 11727 pending_rounds.cc:68] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7333192725272104960 op_type: NO_OP noop_request { }
I20260925 09:57:36.190169 11727 raft_consensus.cc:2866] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:57:36.190810 11727 raft_consensus.cc:2258] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:36.145885 12027 rpcz_store.cc:276] 0925 09:57:31.536542 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:31.536640 (+ 98us) service_pool.cc:227] Handling call
0925 09:57:36.144489 (+4607849us) ts_tablet_manager.cc:1145] Deleting tablet a58cf1dc81454710a9ad1a8baf155c9f
0925 09:57:36.144576 (+ 87us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:36.145063 (+ 487us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4607288}
I20260925 09:57:36.195586 11084 raft_consensus.cc:348] T f3bba45d6b7241f9a07907c79b892f7d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:36.197904 11084 raft_consensus.cc:374] T f3bba45d6b7241f9a07907c79b892f7d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:36.198173 11084 raft_consensus.cc:729] T f3bba45d6b7241f9a07907c79b892f7d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:36.145354 11085 raft_consensus.cc:374] T a5d872c3620646e5ac33ceb95de2cbd4 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:36.199674 11085 raft_consensus.cc:729] T a5d872c3620646e5ac33ceb95de2cbd4 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:36.205202 11084 consensus_queue.cc:260] T f3bba45d6b7241f9a07907c79b892f7d P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:36.206585 11082 tablet_bootstrap.cc:654] T 69028aa7908c45b896da4ba8deef07b1 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:36.212584 11084 ts_tablet_manager.cc:1428] T f3bba45d6b7241f9a07907c79b892f7d P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.023s user 0.002s sys 0.004s
I20260925 09:57:36.213505 11084 tablet_bootstrap.cc:492] T 56f8c08025b94c6eb8b429bd7b0408aa P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:36.217881 12436 sys_catalog.cc:263] Verifying existing consensus state
I20260925 09:57:36.219890 11727 ts_tablet_manager.cc:1905] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:36.227813 11084 tablet_bootstrap.cc:654] T 56f8c08025b94c6eb8b429bd7b0408aa P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:36.200723 11085 consensus_queue.cc:260] T a5d872c3620646e5ac33ceb95de2cbd4 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:36.243479 12436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2: Bootstrap starting.
W20260925 09:57:36.222519 10379 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.62:34455, user_credentials={real_user=slave}} blocked reactor thread for 106523us
I20260925 09:57:36.259428 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 197f61525b0c4f9da441d98701d231a3 (table default.Table_1299538965b945cf859dbbd0cb7003f1 [id=cdc838b49cbb40778c9f440e9f83e96d]) successfully deleted
I20260925 09:57:36.267549 11084 tablet_bootstrap.cc:492] T 56f8c08025b94c6eb8b429bd7b0408aa P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:36.267948 11084 ts_tablet_manager.cc:1397] T 56f8c08025b94c6eb8b429bd7b0408aa P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.055s user 0.011s sys 0.005s
I20260925 09:57:36.270715 11084 raft_consensus.cc:348] T 56f8c08025b94c6eb8b429bd7b0408aa P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:36.271344 11084 raft_consensus.cc:374] T 56f8c08025b94c6eb8b429bd7b0408aa P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:36.271613 11084 raft_consensus.cc:729] T 56f8c08025b94c6eb8b429bd7b0408aa P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:36.272280 11084 consensus_queue.cc:260] T 56f8c08025b94c6eb8b429bd7b0408aa P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:36.274065 10772 consensus_peers.cc:487] T 7a31b999f0fd447eacff13de71aa3fff P c43461c4ee854ae1ac52b873ccada318 -> Peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Couldn't send request to peer be97ab5d75934a648d4ab76341193ec0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a31b999f0fd447eacff13de71aa3fff. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:36.275872 11084 ts_tablet_manager.cc:1428] T 56f8c08025b94c6eb8b429bd7b0408aa P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
W20260925 09:57:36.276463 10391 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0ec6a95911ef44f09e08dfc531966a83 consensus state has no leader
I20260925 09:57:36.277027 10391 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:35140 (request call id 366) took 2962 ms. Trace:
I20260925 09:57:36.279650 10388 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35140 (request call id 368) took 2728 ms. Trace:
I20260925 09:57:36.277736 10390 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35140 (request call id 367) took 2746 ms. Trace:
I20260925 09:57:36.280016 10390 rpcz_store.cc:276] 0925 09:57:33.530944 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:35.840540 (+2309596us) service_pool.cc:227] Handling call
0925 09:57:35.873995 (+ 33455us) spinlock_profiling.cc:243] Waited 32.6 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2adb156 00007fabf0618034 00007fabf0617fc2 00007fabf0617ef1 00007fabf0617e6c 00007fabf0617122 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:36.031471 (+157476us) catalog_manager.cc:2075] Acquired catalog manager lock
0925 09:57:36.033013 (+ 1542us) catalog_manager.cc:2121] Created new table and tablet info
0925 09:57:36.224502 (+191489us) catalog_manager.cc:2181] Wrote table and tablets to system table
0925 09:57:36.277435 (+ 52933us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0925 09:57:36.277671 (+ 236us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:36.082231 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:36.082665 (+ 434us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:36.082693 (+ 28us) write_op.cc:435] Acquired schema lock
0925 09:57:36.082708 (+ 15us) tablet.cc:660] Decoding operations
0925 09:57:36.083040 (+ 332us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:36.083081 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:36.083111 (+ 30us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:36.083330 (+ 219us) tablet.cc:699] Row locks acquired
0925 09:57:36.083347 (+ 17us) write_op.cc:260] PREPARE: finished
0925 09:57:36.083490 (+ 143us) write_op.cc:270] Start()
0925 09:57:36.083585 (+ 95us) write_op.cc:276] Timestamp: P: 1790330256083411 usec, L: 0
0925 09:57:36.083606 (+ 21us) op_driver.cc:331] REPLICATION: starting
0925 09:57:36.095860 (+ 12254us) spinlock_profiling.cc:243] Waited 9.22 ms on lock 0x7b5400012398. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf18330c4 00007fabf0adf4f9 00007fabf0a9af91 00007fabf0a9ac6a 00007fabf0b06cd5 00007fabf0b07179 00007fabf0f68ade 00007fabf0f682b8 00007fabf0f6cba1 00007fabf0f6cb59 00007fabf0f6cae9
0925 09:57:36.096239 (+ 379us) log.cc:842] Serialized 433 byte log entry
0925 09:57:36.216393 (+120154us) spinlock_profiling.cc:243] Waited 37.4 ms on lock 0x7b58000b03b0. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0a85834 00007fabf0a8a771 00007fabf0a8a6f9 00007fabf0a8a689 00007fabf0a8a651 00007fabf0a8981d 0000000000580784 00000000005805b9 00007fabe9bb66b6 00007fabe9bb9b11
0925 09:57:36.222262 (+ 5869us) op_driver.cc:444] REPLICATION: finished
0925 09:57:36.222658 (+ 396us) write_op.cc:301] APPLY: starting
0925 09:57:36.222713 (+ 55us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:36.222813 (+ 100us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:36.222832 (+ 19us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:36.223297 (+ 465us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:36.223466 (+ 169us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 09:57:36.223508 (+ 42us) write_op.cc:312] APPLY: finished
0925 09:57:36.223692 (+ 184us) log.cc:842] Serialized 29 byte log entry
0925 09:57:36.223838 (+ 146us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:36.223902 (+ 64us) write_op.cc:454] Released schema lock
0925 09:57:36.223933 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:36.224093 (+ 160us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:36.224118 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":74873600,"child_traces":[["op",{"apply.queue_time_us":113,"apply.run_cpu_time_us":1604,"apply.run_wall_time_us":1604,"num_ops":2,"prepare.queue_time_us":48106,"prepare.run_cpu_time_us":9133,"prepare.run_wall_time_us":26250,"raft.queue_time_us":10561,"raft.run_cpu_time_us":13199,"raft.run_wall_time_us":207992,"replication_time_us":138568,"spinlock_wait_cycles":107165696,"thread_start_us":10152,"threads_started":1}]]}
I20260925 09:57:36.284782 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 330) took 4537 ms. Trace:
I20260925 09:57:36.285192 12027 rpcz_store.cc:276] 0925 09:57:31.747565 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:31.747667 (+ 102us) service_pool.cc:227] Handling call
0925 09:57:36.193706 (+4446039us) ts_tablet_manager.cc:1145] Deleting tablet 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef
0925 09:57:36.235333 (+ 41627us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:36.284714 (+ 49381us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4445503}
I20260925 09:57:36.286726 12027 tablet_replica.cc:330] T 977a59175e5b43eeb6f23b938afd00a3 P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:36.279938 10388 rpcz_store.cc:276] 0925 09:57:33.551677 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:35.861277 (+2309600us) service_pool.cc:227] Handling call
0925 09:57:35.926395 (+ 65118us) spinlock_profiling.cc:243] Waited 64 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2adb156 00007fabf0618034 00007fabf0617fc2 00007fabf0617ef1 00007fabf0617e6c 00007fabf0617122 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:36.031950 (+105555us) catalog_manager.cc:2075] Acquired catalog manager lock
0925 09:57:36.033265 (+ 1315us) catalog_manager.cc:2121] Created new table and tablet info
0925 09:57:36.279062 (+245797us) catalog_manager.cc:2181] Wrote table and tablets to system table
0925 09:57:36.279361 (+ 299us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0925 09:57:36.279603 (+ 242us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:36.108633 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:36.109171 (+ 538us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:36.109209 (+ 38us) write_op.cc:435] Acquired schema lock
0925 09:57:36.109227 (+ 18us) tablet.cc:660] Decoding operations
0925 09:57:36.109623 (+ 396us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:36.109680 (+ 57us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:36.109720 (+ 40us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:36.110011 (+ 291us) tablet.cc:699] Row locks acquired
0925 09:57:36.110037 (+ 26us) write_op.cc:260] PREPARE: finished
0925 09:57:36.110178 (+ 141us) write_op.cc:270] Start()
0925 09:57:36.110289 (+ 111us) write_op.cc:276] Timestamp: P: 1790330256110091 usec, L: 0
0925 09:57:36.110311 (+ 22us) op_driver.cc:331] REPLICATION: starting
0925 09:57:36.111317 (+ 1006us) log.cc:842] Serialized 433 byte log entry
0925 09:57:36.276971 (+165654us) op_driver.cc:444] REPLICATION: finished
0925 09:57:36.277262 (+ 291us) write_op.cc:301] APPLY: starting
0925 09:57:36.277329 (+ 67us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:36.277439 (+ 110us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:36.277534 (+ 95us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:36.277910 (+ 376us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:36.278038 (+ 128us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 09:57:36.278082 (+ 44us) write_op.cc:312] APPLY: finished
0925 09:57:36.278293 (+ 211us) log.cc:842] Serialized 29 byte log entry
0925 09:57:36.278427 (+ 134us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:36.278494 (+ 67us) write_op.cc:454] Released schema lock
0925 09:57:36.278523 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:36.278677 (+ 154us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:36.278698 (+ 21us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":147310848,"child_traces":[["op",{"apply.queue_time_us":146,"apply.run_cpu_time_us":1580,"apply.run_wall_time_us":1580,"mutex_wait_us":47395,"num_ops":2,"prepare.queue_time_us":74480,"prepare.run_cpu_time_us":10191,"prepare.run_wall_time_us":148509,"raft.queue_time_us":104902,"raft.run_cpu_time_us":31,"raft.run_wall_time_us":3436,"replication_time_us":166582,"thread_start_us":139453,"threads_started":1}]]}
I20260925 09:57:36.289390 10388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_35aa99e4c3c9430ab411483bad074740"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:36.293891 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef already in progress: creating tablet
W20260925 09:57:36.299221 10388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_35aa99e4c3c9430ab411483bad074740 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:36.302021 11085 ts_tablet_manager.cc:1428] T a5d872c3620646e5ac33ceb95de2cbd4 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.203s user 0.006s sys 0.000s
I20260925 09:57:36.308036 11085 tablet_bootstrap.cc:492] T dba14a44a9de492886d629441606767e P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:36.309209 10885 heartbeater.cc:347] Connected to a master server at 127.10.9.60:36985
I20260925 09:57:36.310877 12436 log.cc:824] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2: Log is configured to *not* fsync() on all Append() calls
W20260925 09:57:36.317266 10621 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 49917.2us
I20260925 09:57:36.319326 11029 heartbeater.cc:347] Connected to a master server at 127.10.9.60:36985
I20260925 09:57:36.316051 10390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_0ac0877feb3445159347c33d900cb3ac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:36.328397 10390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0ac0877feb3445159347c33d900cb3ac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:36.215931 12379 ts_tablet_manager.cc:1918] T 7a31b999f0fd447eacff13de71aa3fff P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:36.334184 12379 log.cc:1196] T 7a31b999f0fd447eacff13de71aa3fff P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/7a31b999f0fd447eacff13de71aa3fff
I20260925 09:57:36.334854 12379 ts_tablet_manager.cc:1939] T 7a31b999f0fd447eacff13de71aa3fff P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:36.278034 10391 rpcz_store.cc:276] 0925 09:57:33.314984 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:35.411358 (+2096374us) service_pool.cc:227] Handling call
0925 09:57:35.424437 (+ 13079us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:57:35.425719 (+ 1282us) catalog_manager.cc:3616] Apply alter schema
0925 09:57:35.426314 (+ 595us) catalog_manager.cc:3639] Acquired catalog manager lock
0925 09:57:35.426564 (+ 250us) catalog_manager.cc:3786] Updating metadata on disk
0925 09:57:35.943324 (+516760us) catalog_manager.cc:3817] Committing alterations to in-memory state
0925 09:57:36.276956 (+333632us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:35.738096 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:35.738508 (+ 412us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:35.738532 (+ 24us) write_op.cc:435] Acquired schema lock
0925 09:57:35.738551 (+ 19us) tablet.cc:660] Decoding operations
0925 09:57:35.738836 (+ 285us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:35.738875 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:35.738899 (+ 24us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:35.739050 (+ 151us) tablet.cc:699] Row locks acquired
0925 09:57:35.739066 (+ 16us) write_op.cc:260] PREPARE: finished
0925 09:57:35.739176 (+ 110us) write_op.cc:270] Start()
0925 09:57:35.739259 (+ 83us) write_op.cc:276] Timestamp: P: 1790330255739129 usec, L: 0
0925 09:57:35.739277 (+ 18us) op_driver.cc:331] REPLICATION: starting
0925 09:57:35.739825 (+ 548us) log.cc:842] Serialized 405 byte log entry
0925 09:57:35.915693 (+175868us) op_driver.cc:444] REPLICATION: finished
0925 09:57:35.922884 (+ 7191us) write_op.cc:301] APPLY: starting
0925 09:57:35.922942 (+ 58us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:35.923034 (+ 92us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:35.923122 (+ 88us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:35.923305 (+ 183us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:35.923456 (+ 151us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:35.923506 (+ 50us) write_op.cc:312] APPLY: finished
0925 09:57:35.923664 (+ 158us) log.cc:842] Serialized 23 byte log entry
0925 09:57:35.939980 (+ 16316us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:35.940049 (+ 69us) write_op.cc:454] Released schema lock
0925 09:57:35.940081 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:35.943775 (+ 3694us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:35.943804 (+ 29us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":7028,"apply.run_cpu_time_us":1597,"apply.run_wall_time_us":21499,"num_ops":1,"prepare.queue_time_us":98158,"prepare.run_cpu_time_us":1930,"prepare.run_wall_time_us":2859,"replication_time_us":176313,"thread_start_us":7443,"threads_started":1}]]}
I20260925 09:57:36.344568 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 7a31b999f0fd447eacff13de71aa3fff (table default.Table_489e2c74ed6249ce9cf54541e0a491cd [id=8d2c4997954141129ebbec9e72658d90]) successfully deleted
I20260925 09:57:36.345230 10391 master_service.cc:946] ReplaceTablet: received request to replace tablet 2b1b2d66397b44f38056077d79941ac4 from {username='slave'} at 127.0.0.1:35140
I20260925 09:57:36.356606 10388 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35140 (request call id 369) took 2749 ms. Trace:
I20260925 09:57:36.356945 10388 rpcz_store.cc:276] 0925 09:57:33.607547 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:36.288944 (+2681397us) service_pool.cc:227] Handling call
0925 09:57:36.299657 (+ 10713us) catalog_manager.cc:2075] Acquired catalog manager lock
0925 09:57:36.301327 (+ 1670us) catalog_manager.cc:2121] Created new table and tablet info
0925 09:57:36.327506 (+ 26179us) catalog_manager.cc:2181] Wrote table and tablets to system table
0925 09:57:36.355737 (+ 28231us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0925 09:57:36.356477 (+ 740us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:36.302566 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:36.302994 (+ 428us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:36.303027 (+ 33us) write_op.cc:435] Acquired schema lock
0925 09:57:36.303044 (+ 17us) tablet.cc:660] Decoding operations
0925 09:57:36.303369 (+ 325us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:36.303432 (+ 63us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:36.303461 (+ 29us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:36.303693 (+ 232us) tablet.cc:699] Row locks acquired
0925 09:57:36.303711 (+ 18us) write_op.cc:260] PREPARE: finished
0925 09:57:36.303848 (+ 137us) write_op.cc:270] Start()
0925 09:57:36.303931 (+ 83us) write_op.cc:276] Timestamp: P: 1790330256303791 usec, L: 0
0925 09:57:36.303950 (+ 19us) op_driver.cc:331] REPLICATION: starting
0925 09:57:36.304492 (+ 542us) log.cc:842] Serialized 433 byte log entry
0925 09:57:36.317639 (+ 13147us) op_driver.cc:444] REPLICATION: finished
0925 09:57:36.318168 (+ 529us) write_op.cc:301] APPLY: starting
0925 09:57:36.318224 (+ 56us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:36.318335 (+ 111us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:36.318459 (+ 124us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:36.318846 (+ 387us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:36.318975 (+ 129us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 09:57:36.319084 (+ 109us) write_op.cc:312] APPLY: finished
0925 09:57:36.319291 (+ 207us) log.cc:842] Serialized 29 byte log entry
0925 09:57:36.319454 (+ 163us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:36.319571 (+ 117us) write_op.cc:454] Released schema lock
0925 09:57:36.319630 (+ 59us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:36.319807 (+ 177us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:36.319828 (+ 21us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":384,"apply.run_cpu_time_us":1853,"apply.run_wall_time_us":1853,"num_ops":2,"prepare.queue_time_us":103,"prepare.run_cpu_time_us":2208,"prepare.run_wall_time_us":2921,"raft.queue_time_us":76,"raft.run_cpu_time_us":572,"raft.run_wall_time_us":571,"replication_time_us":13601}]]}
I20260925 09:57:36.361483 10388 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_574ab2069f754ba398e8d4a2e98817cf" } modify_external_catalogs: true
I20260925 09:57:36.362066 10388 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_574ab2069f754ba398e8d4a2e98817cf" } modify_external_catalogs: true
I20260925 09:57:36.363670 12390 consensus_queue.cc:1035] T dba14a44a9de492886d629441606767e P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260925 09:57:36.367455 12379 tablet_replica.cc:330] T 4e3120347c8e4a3eab57d6562ae40e8d P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
W20260925 09:57:36.372706 10621 consensus_peers.cc:487] T dba14a44a9de492886d629441606767e P b8850818e2ee42a082802d0095c08d21 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:36.372874 12379 raft_consensus.cc:2229] T 4e3120347c8e4a3eab57d6562ae40e8d P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:36.377728 12379 raft_consensus.cc:2258] T 4e3120347c8e4a3eab57d6562ae40e8d P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:36.380146 11082 tablet_bootstrap.cc:492] T 69028aa7908c45b896da4ba8deef07b1 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:36.380576 11082 ts_tablet_manager.cc:1397] T 69028aa7908c45b896da4ba8deef07b1 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.241s user 0.012s sys 0.001s
I20260925 09:57:36.380805 10390 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35140 (request call id 370) took 2740 ms. Trace:
I20260925 09:57:36.381170 12379 ts_tablet_manager.cc:1905] T 4e3120347c8e4a3eab57d6562ae40e8d P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:36.382682 11082 raft_consensus.cc:348] T 69028aa7908c45b896da4ba8deef07b1 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:36.381124 10390 rpcz_store.cc:276] 0925 09:57:33.640557 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:36.315451 (+2674894us) service_pool.cc:227] Handling call
0925 09:57:36.355803 (+ 40352us) catalog_manager.cc:2075] Acquired catalog manager lock
0925 09:57:36.357582 (+ 1779us) catalog_manager.cc:2121] Created new table and tablet info
0925 09:57:36.380166 (+ 22584us) catalog_manager.cc:2181] Wrote table and tablets to system table
0925 09:57:36.380485 (+ 319us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0925 09:57:36.380732 (+ 247us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:36.359311 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:36.360259 (+ 948us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:36.360287 (+ 28us) write_op.cc:435] Acquired schema lock
0925 09:57:36.360304 (+ 17us) tablet.cc:660] Decoding operations
0925 09:57:36.360622 (+ 318us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:36.360821 (+ 199us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:36.361523 (+ 702us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:36.362021 (+ 498us) tablet.cc:699] Row locks acquired
0925 09:57:36.362043 (+ 22us) write_op.cc:260] PREPARE: finished
0925 09:57:36.362161 (+ 118us) write_op.cc:270] Start()
0925 09:57:36.362274 (+ 113us) write_op.cc:276] Timestamp: P: 1790330256362104 usec, L: 0
0925 09:57:36.362300 (+ 26us) op_driver.cc:331] REPLICATION: starting
0925 09:57:36.363164 (+ 864us) log.cc:842] Serialized 433 byte log entry
0925 09:57:36.373264 (+ 10100us) op_driver.cc:444] REPLICATION: finished
0925 09:57:36.373602 (+ 338us) write_op.cc:301] APPLY: starting
0925 09:57:36.373663 (+ 61us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:36.373774 (+ 111us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:36.373875 (+ 101us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:36.374299 (+ 424us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:36.374434 (+ 135us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 09:57:36.374479 (+ 45us) write_op.cc:312] APPLY: finished
0925 09:57:36.374664 (+ 185us) log.cc:842] Serialized 29 byte log entry
0925 09:57:36.374803 (+ 139us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:36.374872 (+ 69us) write_op.cc:454] Released schema lock
0925 09:57:36.374905 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:36.375074 (+ 169us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:36.375098 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":186,"apply.run_cpu_time_us":1667,"apply.run_wall_time_us":1667,"num_ops":2,"prepare.queue_time_us":165,"prepare.run_cpu_time_us":2929,"prepare.run_wall_time_us":11429,"raft.queue_time_us":136,"raft.run_cpu_time_us":690,"raft.run_wall_time_us":690,"replication_time_us":10877}]]}
I20260925 09:57:36.399169 10390 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_f37bcee5db2044f2bea9c353982b30cd" } modify_external_catalogs: true
I20260925 09:57:36.403918 10390 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_f37bcee5db2044f2bea9c353982b30cd" } modify_external_catalogs: true
I20260925 09:57:36.392081 12322 consensus_queue.cc:1035] T defaef4d6b754ca5a2dfc21b7a596a5b P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:36.413422 12436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=1 applied=6 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 09:57:36.413857 12322 consensus_queue.cc:1035] T ab6bdba5ca5d41ae921a90c1f77006c5 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:36.414561 12436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2: Bootstrap complete.
I20260925 09:57:36.419096 12251 raft_consensus.cc:482] T dbc7c7b9a2ee437eabe10150bdbc2e4a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:36.427685 12207 raft_consensus.cc:482] T b3d9a3cf5bda4cf6bc1fa3fae1cb9288 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:36.427742 12251 raft_consensus.cc:504] T dbc7c7b9a2ee437eabe10150bdbc2e4a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:36.428177 12207 raft_consensus.cc:504] T b3d9a3cf5bda4cf6bc1fa3fae1cb9288 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:36.430154 12207 leader_election.cc:288] T b3d9a3cf5bda4cf6bc1fa3fae1cb9288 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:36.431834 10673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3d9a3cf5bda4cf6bc1fa3fae1cb9288" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:36.433228 10919 leader_election.cc:341] T b3d9a3cf5bda4cf6bc1fa3fae1cb9288 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:36.435195 10673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbc7c7b9a2ee437eabe10150bdbc2e4a" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:36.436430 10919 leader_election.cc:341] T dbc7c7b9a2ee437eabe10150bdbc2e4a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:36.436962 12251 leader_election.cc:288] T dbc7c7b9a2ee437eabe10150bdbc2e4a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:36.437968 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3d9a3cf5bda4cf6bc1fa3fae1cb9288" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:36.438262 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbc7c7b9a2ee437eabe10150bdbc2e4a" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:36.439313 10918 leader_election.cc:341] T b3d9a3cf5bda4cf6bc1fa3fae1cb9288 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:36.439980 10918 leader_election.cc:302] T b3d9a3cf5bda4cf6bc1fa3fae1cb9288 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
W20260925 09:57:36.440971 10918 leader_election.cc:341] T dbc7c7b9a2ee437eabe10150bdbc2e4a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:36.441284 10918 leader_election.cc:302] T dbc7c7b9a2ee437eabe10150bdbc2e4a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:36.442170 12207 raft_consensus.cc:2726] T b3d9a3cf5bda4cf6bc1fa3fae1cb9288 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:36.442687 12207 raft_consensus.cc:2726] T dbc7c7b9a2ee437eabe10150bdbc2e4a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:36.445801 12379 ts_tablet_manager.cc:1918] T 4e3120347c8e4a3eab57d6562ae40e8d P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:36.454154 12251 raft_consensus.cc:482] T 2b1b2d66397b44f38056077d79941ac4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:36.454658 12251 raft_consensus.cc:504] T 2b1b2d66397b44f38056077d79941ac4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:36.457942 12251 leader_election.cc:288] T 2b1b2d66397b44f38056077d79941ac4 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
W20260925 09:57:36.449980 10621 consensus_peers.cc:487] T defaef4d6b754ca5a2dfc21b7a596a5b P b8850818e2ee42a082802d0095c08d21 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:36.457441 10673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b1b2d66397b44f38056077d79941ac4" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:36.470883 12379 log.cc:1196] T 4e3120347c8e4a3eab57d6562ae40e8d P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/4e3120347c8e4a3eab57d6562ae40e8d
W20260925 09:57:36.472043 10919 leader_election.cc:341] T 2b1b2d66397b44f38056077d79941ac4 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:36.473157 12379 ts_tablet_manager.cc:1939] T 4e3120347c8e4a3eab57d6562ae40e8d P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
W20260925 09:57:36.393739 10742 tablet.cc:2369] T f62c62715c9b42de826c1a66fd431eab P b8850818e2ee42a082802d0095c08d21: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:57:36.474347 12379 tablet_replica.cc:330] T 6161774f61ae48ab9764a05f2188a80b P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:36.474498 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 4e3120347c8e4a3eab57d6562ae40e8d (table default.Table_8e186a99ccba425b9b588041d2364a40 [id=2d6bc0e2aaee4b1483b01fc678505bb0]) successfully deleted
I20260925 09:57:36.479694 10668 tablet_service.cc:1467] Processing CreateTablet for tablet eb3b94bcd7514cec90a614253e346d6f (DEFAULT_TABLE table=default.Table_065ec6e727eb4cbebcc6acb042d70574 [id=b5e5eaa7cbc842978b1a69745c7b7304]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:36.484772 12251 raft_consensus.cc:482] T f3bba45d6b7241f9a07907c79b892f7d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:36.485500 10830 raft_consensus.cc:3037] T a5d872c3620646e5ac33ceb95de2cbd4 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:36.485347 12251 raft_consensus.cc:504] T f3bba45d6b7241f9a07907c79b892f7d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:36.488154 10809 tablet_service.cc:1467] Processing CreateTablet for tablet eb3b94bcd7514cec90a614253e346d6f (DEFAULT_TABLE table=default.Table_065ec6e727eb4cbebcc6acb042d70574 [id=b5e5eaa7cbc842978b1a69745c7b7304]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:36.489234 10809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb3b94bcd7514cec90a614253e346d6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:36.489171 10673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3bba45d6b7241f9a07907c79b892f7d" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:36.490437 10919 leader_election.cc:341] T f3bba45d6b7241f9a07907c79b892f7d P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:36.493394 10956 tablet_service.cc:1467] Processing CreateTablet for tablet eb3b94bcd7514cec90a614253e346d6f (DEFAULT_TABLE table=default.Table_065ec6e727eb4cbebcc6acb042d70574 [id=b5e5eaa7cbc842978b1a69745c7b7304]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:36.496721 10653 tablet_service.cc:1467] Processing CreateTablet for tablet d5e85d00c54b4831863f933f71e55dd1 (DEFAULT_TABLE table=default.Table_47144aad84a8469dbd8f97ce52d04e9f [id=4529a0272fa1492788ae012cf5c76af2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:36.501525 12446 consensus_queue.cc:1035] T 4e3120347c8e4a3eab57d6562ae40e8d P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:36.468314 11082 raft_consensus.cc:374] T 69028aa7908c45b896da4ba8deef07b1 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:36.503365 11082 raft_consensus.cc:729] T 69028aa7908c45b896da4ba8deef07b1 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:36.504072 11082 consensus_queue.cc:260] T 69028aa7908c45b896da4ba8deef07b1 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:36.505894 10810 tablet_service.cc:1467] Processing CreateTablet for tablet d5e85d00c54b4831863f933f71e55dd1 (DEFAULT_TABLE table=default.Table_47144aad84a8469dbd8f97ce52d04e9f [id=4529a0272fa1492788ae012cf5c76af2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:36.506878 10810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5e85d00c54b4831863f933f71e55dd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:36.506978 10952 tablet_service.cc:1467] Processing CreateTablet for tablet d5e85d00c54b4831863f933f71e55dd1 (DEFAULT_TABLE table=default.Table_47144aad84a8469dbd8f97ce52d04e9f [id=4529a0272fa1492788ae012cf5c76af2]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 09:57:36.508615 10772 consensus_peers.cc:487] T 4e3120347c8e4a3eab57d6562ae40e8d P c43461c4ee854ae1ac52b873ccada318 -> Peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Couldn't send request to peer be97ab5d75934a648d4ab76341193ec0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4e3120347c8e4a3eab57d6562ae40e8d. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:36.511664 10667 tablet_service.cc:1467] Processing CreateTablet for tablet 6a15383bca53477c9e90021b1a64bf80 (DEFAULT_TABLE table=default.Table_014617e901574abeb4c2cc2ef676bea3 [id=d5e53d6c12c34571b403b0aaed9845f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:36.521062 12207 raft_consensus.cc:482] T 88e53debe6de43e29b12e85e5171c8b5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:36.521591 12207 raft_consensus.cc:504] T 88e53debe6de43e29b12e85e5171c8b5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:36.525763 12207 leader_election.cc:288] T 88e53debe6de43e29b12e85e5171c8b5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:36.527179 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88e53debe6de43e29b12e85e5171c8b5" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:36.532235 10919 leader_election.cc:341] T 88e53debe6de43e29b12e85e5171c8b5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:36.541394 11084 tablet_bootstrap.cc:492] T c52745a8cdee461aba7485b0531245d6 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:36.544476 10956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb3b94bcd7514cec90a614253e346d6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:36.548003 10952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5e85d00c54b4831863f933f71e55dd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:36.550572 10388 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet d822e8ac13d9479aa9ce4f74ace55dd5
I20260925 09:57:36.552541 10388 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:35140 (request call id 372) took 1043 ms. Trace:
I20260925 09:57:36.554553 10817 tablet_service.cc:1467] Processing CreateTablet for tablet 6a15383bca53477c9e90021b1a64bf80 (DEFAULT_TABLE table=default.Table_014617e901574abeb4c2cc2ef676bea3 [id=d5e53d6c12c34571b403b0aaed9845f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:36.554849 11084 tablet_bootstrap.cc:654] T c52745a8cdee461aba7485b0531245d6 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:36.555616 10817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a15383bca53477c9e90021b1a64bf80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:36.559615 10806 tablet_service.cc:1514] Processing DeleteTablet for tablet d822e8ac13d9479aa9ce4f74ace55dd5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:36 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:36.561507 10806 tablet_service.cc:1467] Processing CreateTablet for tablet 8d9fe9c2159c416ab8eb165f04ab577b (DEFAULT_TABLE table=default.Table_9f0d5b7a076747f1aadbb6ac661362b8 [id=92519e03bab74976ae742562485849c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:36.562520 10806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d9fe9c2159c416ab8eb165f04ab577b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:36.564066 10391 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 2b1b2d66397b44f38056077d79941ac4
I20260925 09:57:36.565387 10391 catalog_manager.cc:6565] ReplaceTablet: tablet 2b1b2d66397b44f38056077d79941ac4 deleted and replaced by tablet f535784bf0a74769a6d97115fb3b0c8a
I20260925 09:57:36.565701 10391 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:35140 (request call id 371) took 2440 ms. Trace:
I20260925 09:57:36.566139 10655 tablet_service.cc:1514] Processing DeleteTablet for tablet d822e8ac13d9479aa9ce4f74ace55dd5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:36 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:36.567528 10655 tablet_service.cc:1467] Processing CreateTablet for tablet 8d9fe9c2159c416ab8eb165f04ab577b (DEFAULT_TABLE table=default.Table_9f0d5b7a076747f1aadbb6ac661362b8 [id=92519e03bab74976ae742562485849c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:36.568397 10813 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b1b2d66397b44f38056077d79941ac4 with delete_type TABLET_DATA_DELETED (replaced by tablet f535784bf0a74769a6d97115fb3b0c8a) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:36.552848 10388 rpcz_store.cc:276] 0925 09:57:35.508761 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:36.361179 (+852418us) service_pool.cc:227] Handling call
0925 09:57:36.362516 (+ 1337us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:57:36.366326 (+ 3810us) catalog_manager.cc:2752] Modifying in-memory table state
0925 09:57:36.366446 (+ 120us) catalog_manager.cc:2761] Locking tablets
0925 09:57:36.480057 (+113611us) catalog_manager.cc:2775] Removing table and tablets from system table
0925 09:57:36.550172 (+ 70115us) catalog_manager.cc:2793] Removing table from by-name map
0925 09:57:36.550387 (+ 215us) catalog_manager.cc:2809] Committing in-memory state
0925 09:57:36.550491 (+ 104us) catalog_manager.cc:2813] Aborting table tasks
0925 09:57:36.552469 (+ 1978us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:36.494000 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:36.494422 (+ 422us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:36.494448 (+ 26us) write_op.cc:435] Acquired schema lock
0925 09:57:36.494459 (+ 11us) tablet.cc:660] Decoding operations
0925 09:57:36.494824 (+ 365us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:36.494864 (+ 40us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:36.494894 (+ 30us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:36.495230 (+ 336us) tablet.cc:699] Row locks acquired
0925 09:57:36.495247 (+ 17us) write_op.cc:260] PREPARE: finished
0925 09:57:36.518541 (+ 23294us) write_op.cc:270] Start()
0925 09:57:36.518645 (+ 104us) write_op.cc:276] Timestamp: P: 1790330256495345 usec, L: 0
0925 09:57:36.518664 (+ 19us) op_driver.cc:331] REPLICATION: starting
0925 09:57:36.519187 (+ 523us) log.cc:842] Serialized 722 byte log entry
0925 09:57:36.543969 (+ 24782us) op_driver.cc:444] REPLICATION: finished
0925 09:57:36.546393 (+ 2424us) write_op.cc:301] APPLY: starting
0925 09:57:36.546443 (+ 50us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:36.546547 (+ 104us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:36.546631 (+ 84us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:36.546903 (+ 272us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:36.547032 (+ 129us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 09:57:36.547111 (+ 79us) write_op.cc:312] APPLY: finished
0925 09:57:36.547331 (+ 220us) log.cc:842] Serialized 29 byte log entry
0925 09:57:36.547606 (+ 275us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:36.547727 (+ 121us) write_op.cc:454] Released schema lock
0925 09:57:36.547790 (+ 63us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:36.548054 (+ 264us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:36.548077 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2235,"apply.run_cpu_time_us":1876,"apply.run_wall_time_us":1876,"num_ops":2,"prepare.queue_time_us":12157,"prepare.run_cpu_time_us":2439,"prepare.run_wall_time_us":25495,"raft.queue_time_us":110,"raft.run_cpu_time_us":616,"raft.run_wall_time_us":616,"replication_time_us":25223}]]}
I20260925 09:57:36.569387 10665 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b1b2d66397b44f38056077d79941ac4 with delete_type TABLET_DATA_DELETED (replaced by tablet f535784bf0a74769a6d97115fb3b0c8a) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:36.570747 10947 tablet_service.cc:1514] Processing DeleteTablet for tablet d822e8ac13d9479aa9ce4f74ace55dd5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:36 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:36.571429 10950 tablet_service.cc:1467] Processing CreateTablet for tablet 6a15383bca53477c9e90021b1a64bf80 (DEFAULT_TABLE table=default.Table_014617e901574abeb4c2cc2ef676bea3 [id=d5e53d6c12c34571b403b0aaed9845f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:36.572540 10950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a15383bca53477c9e90021b1a64bf80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:36.576078 10813 tablet_service.cc:1467] Processing CreateTablet for tablet 0008fa457ad3443f86ec2dbc37e8bed0 (DEFAULT_TABLE table=default.Table_400af62bad0248dfa8846e2b7637b0f7 [id=dd7de5603c1643bcadd02057052b16e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:36.577024 10813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0008fa457ad3443f86ec2dbc37e8bed0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:36.580380 10947 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b1b2d66397b44f38056077d79941ac4 with delete_type TABLET_DATA_DELETED (replaced by tablet f535784bf0a74769a6d97115fb3b0c8a) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:36.529388 10389 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 2f07dc9548364870923165f4ea524310 (table default.Table_0db927e92a3b4970a7efea3fb765e65e [id=a988264fe43c4eb88a84f8b6359e77a6]). Expected version 1 got 0
I20260925 09:57:36.590019 10665 tablet_service.cc:1467] Processing CreateTablet for tablet 0008fa457ad3443f86ec2dbc37e8bed0 (DEFAULT_TABLE table=default.Table_400af62bad0248dfa8846e2b7637b0f7 [id=dd7de5603c1643bcadd02057052b16e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:36.596380 10948 tablet_service.cc:1467] Processing CreateTablet for tablet 8d9fe9c2159c416ab8eb165f04ab577b (DEFAULT_TABLE table=default.Table_9f0d5b7a076747f1aadbb6ac661362b8 [id=92519e03bab74976ae742562485849c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:36.597539 10948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d9fe9c2159c416ab8eb165f04ab577b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:36.606115 10668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb3b94bcd7514cec90a614253e346d6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:36.565954 10391 rpcz_store.cc:276] 0925 09:57:34.124995 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:36.345079 (+2220084us) service_pool.cc:227] Handling call
0925 09:57:36.565649 (+220570us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:36.519616 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:36.520043 (+ 427us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:36.520069 (+ 26us) write_op.cc:435] Acquired schema lock
0925 09:57:36.520085 (+ 16us) tablet.cc:660] Decoding operations
0925 09:57:36.520673 (+ 588us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:36.520745 (+ 72us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:36.521155 (+ 410us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:36.521669 (+ 514us) tablet.cc:699] Row locks acquired
0925 09:57:36.521722 (+ 53us) write_op.cc:260] PREPARE: finished
0925 09:57:36.521937 (+ 215us) write_op.cc:270] Start()
0925 09:57:36.522145 (+ 208us) write_op.cc:276] Timestamp: P: 1790330256521838 usec, L: 0
0925 09:57:36.522173 (+ 28us) op_driver.cc:331] REPLICATION: starting
0925 09:57:36.543668 (+ 21495us) log.cc:842] Serialized 619 byte log entry
0925 09:57:36.545913 (+ 2245us) op_driver.cc:444] REPLICATION: finished
0925 09:57:36.548397 (+ 2484us) write_op.cc:301] APPLY: starting
0925 09:57:36.548442 (+ 45us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:36.548545 (+ 103us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:36.548631 (+ 86us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:36.549009 (+ 378us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:36.549203 (+ 194us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:36.549249 (+ 46us) write_op.cc:312] APPLY: finished
0925 09:57:36.549435 (+ 186us) log.cc:842] Serialized 29 byte log entry
0925 09:57:36.549566 (+ 131us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:36.549632 (+ 66us) write_op.cc:454] Released schema lock
0925 09:57:36.549661 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:36.549817 (+ 156us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:36.549842 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2355,"apply.run_cpu_time_us":1606,"apply.run_wall_time_us":1606,"num_ops":2,"prepare.queue_time_us":25761,"prepare.run_cpu_time_us":2709,"prepare.run_wall_time_us":81950,"raft.queue_time_us":18794,"raft.run_cpu_time_us":362,"raft.run_wall_time_us":363,"replication_time_us":23667,"thread_start_us":33398,"threads_started":1}]]}
I20260925 09:57:36.611696 10653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5e85d00c54b4831863f933f71e55dd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:36.622114 12251 leader_election.cc:288] T f3bba45d6b7241f9a07907c79b892f7d P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:36.636152 12390 raft_consensus.cc:482] T f62c62715c9b42de826c1a66fd431eab P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:36.636628 12390 raft_consensus.cc:504] T f62c62715c9b42de826c1a66fd431eab P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:36.638993 12390 leader_election.cc:288] T f62c62715c9b42de826c1a66fd431eab P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:36.641656 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f62c62715c9b42de826c1a66fd431eab" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:36.645249 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f62c62715c9b42de826c1a66fd431eab" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:36.645839 10976 raft_consensus.cc:2454] T f62c62715c9b42de826c1a66fd431eab P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:36.647357 10654 tablet_service.cc:1467] Processing CreateTablet for tablet 9de2dbaeb9d44e2b9f2db4f11ea0a050 (DEFAULT_TABLE table=default.Table_e63b9b18da2949e9bad5432a2ef0ca6f [id=eb270f64681849dda6f48da7e965c979]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:36.647716 10947 tablet_service.cc:1467] Processing CreateTablet for tablet 0008fa457ad3443f86ec2dbc37e8bed0 (DEFAULT_TABLE table=default.Table_400af62bad0248dfa8846e2b7637b0f7 [id=dd7de5603c1643bcadd02057052b16e4]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 09:57:36.647881 10621 leader_election.cc:341] T f62c62715c9b42de826c1a66fd431eab P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:36.648903 10947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0008fa457ad3443f86ec2dbc37e8bed0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:36.649884 10619 leader_election.cc:302] T f62c62715c9b42de826c1a66fd431eab P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:36.650640 12390 raft_consensus.cc:2781] T f62c62715c9b42de826c1a66fd431eab P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:36.651051 12390 raft_consensus.cc:482] T f62c62715c9b42de826c1a66fd431eab P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:36.651463 12390 raft_consensus.cc:3037] T f62c62715c9b42de826c1a66fd431eab P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:36.658385 12390 raft_consensus.cc:504] T f62c62715c9b42de826c1a66fd431eab P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:36.661777 11084 tablet_bootstrap.cc:492] T c52745a8cdee461aba7485b0531245d6 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
W20260925 09:57:36.665129 10775 consensus_peers.cc:487] T 4e3120347c8e4a3eab57d6562ae40e8d P c43461c4ee854ae1ac52b873ccada318 -> Peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Couldn't send request to peer b8850818e2ee42a082802d0095c08d21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4e3120347c8e4a3eab57d6562ae40e8d. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:36.683524 10391 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_835c3ec6bfb943ceac4d44f0735bb91c" } new_table_name: "default.Table_f261a668854045ea93bc3cd825d6a685" modify_external_catalogs: true
I20260925 09:57:36.684433 11084 ts_tablet_manager.cc:1397] T c52745a8cdee461aba7485b0531245d6 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.143s user 0.011s sys 0.000s
I20260925 09:57:36.688572 10946 tablet_service.cc:1467] Processing CreateTablet for tablet 9de2dbaeb9d44e2b9f2db4f11ea0a050 (DEFAULT_TABLE table=default.Table_e63b9b18da2949e9bad5432a2ef0ca6f [id=eb270f64681849dda6f48da7e965c979]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:36.693668 11084 raft_consensus.cc:348] T c52745a8cdee461aba7485b0531245d6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:36.704113 12390 leader_election.cc:288] T f62c62715c9b42de826c1a66fd431eab P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:36.704653 10810 tablet_service.cc:1467] Processing CreateTablet for tablet 9de2dbaeb9d44e2b9f2db4f11ea0a050 (DEFAULT_TABLE table=default.Table_e63b9b18da2949e9bad5432a2ef0ca6f [id=eb270f64681849dda6f48da7e965c979]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:36.641913 12436 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 4 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } }
I20260925 09:57:36.705570 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f62c62715c9b42de826c1a66fd431eab" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:36.708346 10976 raft_consensus.cc:3037] T f62c62715c9b42de826c1a66fd431eab P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:36.710337 11085 tablet_bootstrap.cc:654] T dba14a44a9de492886d629441606767e P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:36.711655 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f62c62715c9b42de826c1a66fd431eab" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:36.712841 10810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9de2dbaeb9d44e2b9f2db4f11ea0a050. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:36.715195 10976 raft_consensus.cc:2454] T f62c62715c9b42de826c1a66fd431eab P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:36.716759 11084 raft_consensus.cc:374] T c52745a8cdee461aba7485b0531245d6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:36.717372 11084 raft_consensus.cc:729] T c52745a8cdee461aba7485b0531245d6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
W20260925 09:57:36.719266 10918 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 278303us
I20260925 09:57:36.724839 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88e53debe6de43e29b12e85e5171c8b5" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:36.725127 12436 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8be057e0b6d40a3a4b90f2b1f4a0fa2, State: Initialized, Role: FOLLOWER
I20260925 09:57:36.724380 10809 tablet_service.cc:1467] Processing CreateTablet for tablet d4dd5ebc3d654ef1852d2ee298a584a4 (DEFAULT_TABLE table=default.Table_783b675e5dff47d1a8f7d3b0f535efb9 [id=40ef35ff4add44eb8d1f0c0066897346]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:36.726670 10809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4dd5ebc3d654ef1852d2ee298a584a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:36.727548 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3bba45d6b7241f9a07907c79b892f7d" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:36.728279 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b1b2d66397b44f38056077d79941ac4" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:36.731614 12436 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } }
I20260925 09:57:36.748389 10668 tablet_service.cc:1467] Processing CreateTablet for tablet a6e82e83508440aa94028e71d140f8a3 (DEFAULT_TABLE table=default.Table_d05e29f3834046738be803d038f17c88 [id=66f64e8d69d54408ae9af3056bc0b10d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:36.748577 10944 tablet_service.cc:1467] Processing CreateTablet for tablet d4dd5ebc3d654ef1852d2ee298a584a4 (DEFAULT_TABLE table=default.Table_783b675e5dff47d1a8f7d3b0f535efb9 [id=40ef35ff4add44eb8d1f0c0066897346]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:36.687933 10664 tablet_service.cc:1467] Processing CreateTablet for tablet d4dd5ebc3d654ef1852d2ee298a584a4 (DEFAULT_TABLE table=default.Table_783b675e5dff47d1a8f7d3b0f535efb9 [id=40ef35ff4add44eb8d1f0c0066897346]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:36.718544 11084 consensus_queue.cc:260] T c52745a8cdee461aba7485b0531245d6 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:36.749004 10945 tablet_service.cc:1467] Processing CreateTablet for tablet 154083d32e0f474f9e3a02e6cca940ef (DEFAULT_TABLE table=default.Table_836d63b3fbed4250b0757ef2d432a704 [id=f58c7edcfeb44c90968b95873346f919]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:36.667546 11082 ts_tablet_manager.cc:1428] T 69028aa7908c45b896da4ba8deef07b1 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.287s user 0.002s sys 0.003s
I20260925 09:57:36.716279 10619 leader_election.cc:302] T f62c62715c9b42de826c1a66fd431eab P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:36.724493 10667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a15383bca53477c9e90021b1a64bf80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:36.750772 10655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d9fe9c2159c416ab8eb165f04ab577b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:36.744057 10653 tablet_service.cc:1467] Processing CreateTablet for tablet 154083d32e0f474f9e3a02e6cca940ef (DEFAULT_TABLE table=default.Table_836d63b3fbed4250b0757ef2d432a704 [id=f58c7edcfeb44c90968b95873346f919]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:36.754272 12392 raft_consensus.cc:2781] T f62c62715c9b42de826c1a66fd431eab P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:36.755668 10665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0008fa457ad3443f86ec2dbc37e8bed0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:36.755605 12392 raft_consensus.cc:686] T f62c62715c9b42de826c1a66fd431eab P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:36.756524 12392 consensus_queue.cc:237] T f62c62715c9b42de826c1a66fd431eab P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:36.759709 10654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9de2dbaeb9d44e2b9f2db4f11ea0a050. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:36.763613 10960 tablet_service.cc:1467] Processing CreateTablet for tablet a6e82e83508440aa94028e71d140f8a3 (DEFAULT_TABLE table=default.Table_d05e29f3834046738be803d038f17c88 [id=66f64e8d69d54408ae9af3056bc0b10d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:36.764288 10387 catalog_manager.cc:5495] T 2f07dc9548364870923165f4ea524310 P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: HEALTHY } } }
I20260925 09:57:36.766410 12462 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 4 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } } }
I20260925 09:57:36.766837 12390 consensus_queue.cc:1035] T 96a80382bb7d4395a14ad9ae972f4a25 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:36.767230 12462 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:57:36.773773 12436 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [sys.catalog]: configured and running, proceeding with master startup.
W20260925 09:57:36.777532 10391 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cf7b2d3f3b3949a0b81283bc68e7b131 consensus state has no leader
I20260925 09:57:36.778113 10391 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:35140 (request call id 374) took 1168 ms. Trace:
I20260925 09:57:36.779349 10817 tablet_service.cc:1467] Processing CreateTablet for tablet 154083d32e0f474f9e3a02e6cca940ef (DEFAULT_TABLE table=default.Table_836d63b3fbed4250b0757ef2d432a704 [id=f58c7edcfeb44c90968b95873346f919]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:36.779696 10806 tablet_service.cc:1467] Processing CreateTablet for tablet a6e82e83508440aa94028e71d140f8a3 (DEFAULT_TABLE table=default.Table_d05e29f3834046738be803d038f17c88 [id=66f64e8d69d54408ae9af3056bc0b10d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:36.780263 10817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 154083d32e0f474f9e3a02e6cca940ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:36.780781 10806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6e82e83508440aa94028e71d140f8a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:36.779147 10391 rpcz_store.cc:276] 0925 09:57:35.610210 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:36.608288 (+998078us) service_pool.cc:227] Handling call
0925 09:57:36.684140 (+ 75852us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:57:36.692069 (+ 7929us) catalog_manager.cc:3616] Apply alter schema
0925 09:57:36.692609 (+ 540us) catalog_manager.cc:3639] Acquired catalog manager lock
0925 09:57:36.692830 (+ 221us) catalog_manager.cc:3786] Updating metadata on disk
0925 09:57:36.739611 (+ 46781us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:36.739646 (+ 35us) write_op.cc:454] Released schema lock
0925 09:57:36.739850 (+ 204us) catalog_manager.cc:3817] Committing alterations to in-memory state
0925 09:57:36.778065 (+ 38215us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:36.693975 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:36.694428 (+ 453us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:36.694458 (+ 30us) write_op.cc:435] Acquired schema lock
0925 09:57:36.694474 (+ 16us) tablet.cc:660] Decoding operations
0925 09:57:36.694759 (+ 285us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:36.694814 (+ 55us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:36.694840 (+ 26us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:36.695006 (+ 166us) tablet.cc:699] Row locks acquired
0925 09:57:36.695022 (+ 16us) write_op.cc:260] PREPARE: finished
0925 09:57:36.695137 (+ 115us) write_op.cc:270] Start()
0925 09:57:36.695226 (+ 89us) write_op.cc:276] Timestamp: P: 1790330256695092 usec, L: 0
0925 09:57:36.695247 (+ 21us) op_driver.cc:331] REPLICATION: starting
0925 09:57:36.695982 (+ 735us) log.cc:842] Serialized 405 byte log entry
0925 09:57:36.703706 (+ 7724us) op_driver.cc:444] REPLICATION: finished
0925 09:57:36.704149 (+ 443us) write_op.cc:301] APPLY: starting
0925 09:57:36.704199 (+ 50us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:36.704295 (+ 96us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:36.704398 (+ 103us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:36.704547 (+ 149us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:36.704678 (+ 131us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:36.704727 (+ 49us) write_op.cc:312] APPLY: finished
0925 09:57:36.704906 (+ 179us) log.cc:842] Serialized 23 byte log entry
0925 09:57:36.705474 (+ 568us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:36.705540 (+ 66us) write_op.cc:454] Released schema lock
0925 09:57:36.705570 (+ 30us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":264,"apply.run_cpu_time_us":1225,"apply.run_wall_time_us":1693,"num_ops":1,"prepare.queue_time_us":117,"prepare.run_cpu_time_us":2091,"prepare.run_wall_time_us":3442,"raft.queue_time_us":209,"raft.run_cpu_time_us":653,"raft.run_wall_time_us":719,"replication_time_us":8381}]]}
W20260925 09:57:36.783067 10379 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 9ddb3e3e716d4a3b8e75614b334f8c38 (table default.Table_00b6767f5a634daebb7f179c663cb8b8 [id=e2ef5652a9564363a87f76281d131448], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 9ddb3e3e716d4a3b8e75614b334f8c38 consensus state has no leader
I20260925 09:57:36.785051 10658 tablet_service.cc:1467] Processing CreateTablet for tablet e83d6dc8657d4a1f89ab0893f435abf6 (DEFAULT_TABLE table=default.Table_2a6203820e0d44aa993080049923de10 [id=c90c671ed4f043c2b56a1f84ff9a42cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:36.765682 10387 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 2f07dc9548364870923165f4ea524310 (table default.Table_0db927e92a3b4970a7efea3fb765e65e [id=a988264fe43c4eb88a84f8b6359e77a6]). Expected version 1 got 0
I20260925 09:57:36.786386 10812 tablet_service.cc:1467] Processing CreateTablet for tablet e83d6dc8657d4a1f89ab0893f435abf6 (DEFAULT_TABLE table=default.Table_2a6203820e0d44aa993080049923de10 [id=c90c671ed4f043c2b56a1f84ff9a42cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:36.787313 10812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e83d6dc8657d4a1f89ab0893f435abf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:36.786931 10387 catalog_manager.cc:5495] T a5d872c3620646e5ac33ceb95de2cbd4 P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: HEALTHY } } }
I20260925 09:57:36.789736 10387 catalog_manager.cc:5495] T 6540aba5854a44419addc5d454e473b7 P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: HEALTHY } } }
I20260925 09:57:36.790833 10387 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 6540aba5854a44419addc5d454e473b7 (table default.Table_8180842f664042b0a9073b7008438ff7 [id=416a4b989570457e919fb7e7b16a017d]). Expected version 1 got 0
I20260925 09:57:36.808151 10946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9de2dbaeb9d44e2b9f2db4f11ea0a050. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:36.809032 10945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 154083d32e0f474f9e3a02e6cca940ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:36.811874 10960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6e82e83508440aa94028e71d140f8a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:36.815958 10944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4dd5ebc3d654ef1852d2ee298a584a4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:36.750883 10621 leader_election.cc:341] T f62c62715c9b42de826c1a66fd431eab P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:36.827906 10621 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 110576us
W20260925 09:57:36.828565 10621 consensus_peers.cc:487] T ab6bdba5ca5d41ae921a90c1f77006c5 P b8850818e2ee42a082802d0095c08d21 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:36.829196 10391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_2e0bff2adcb743479f616bb108fe4497"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:36.830957 10391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2e0bff2adcb743479f616bb108fe4497 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:36.831518 10885 heartbeater.cc:347] Connected to a master server at 127.10.9.60:36985
I20260925 09:57:36.834156 12027 raft_consensus.cc:2229] T 977a59175e5b43eeb6f23b938afd00a3 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:36.834581 12027 raft_consensus.cc:2258] T 977a59175e5b43eeb6f23b938afd00a3 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:36.836262 12027 ts_tablet_manager.cc:1905] T 977a59175e5b43eeb6f23b938afd00a3 P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:36.848589 10387 catalog_manager.cc:5495] T b61444c31dbf4882ac748f861ea66818 P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
W20260925 09:57:36.859071 10775 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 192630us
I20260925 09:57:36.862747 12027 ts_tablet_manager.cc:1918] T 977a59175e5b43eeb6f23b938afd00a3 P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:36.866052 12027 log.cc:1196] T 977a59175e5b43eeb6f23b938afd00a3 P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/977a59175e5b43eeb6f23b938afd00a3
I20260925 09:57:36.871071 11082 tablet_bootstrap.cc:492] T 9744b9814b69470396ce039348fe5e45 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:36.872412 12027 ts_tablet_manager.cc:1939] T 977a59175e5b43eeb6f23b938afd00a3 P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:36.873155 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 331) took 5125 ms. Trace:
I20260925 09:57:36.873431 12027 rpcz_store.cc:276] 0925 09:57:31.747831 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:31.777261 (+ 29430us) service_pool.cc:227] Handling call
0925 09:57:36.286475 (+4509214us) ts_tablet_manager.cc:1145] Deleting tablet 977a59175e5b43eeb6f23b938afd00a3
0925 09:57:36.286627 (+ 152us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:36.843856 (+557229us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:36.862555 (+ 18699us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:36.873107 (+ 10552us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":426530,"tablet-delete.queue_time_us":4483561}
I20260925 09:57:36.874456 12452 raft_consensus.cc:482] T 56f8c08025b94c6eb8b429bd7b0408aa P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:36.874564 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 977a59175e5b43eeb6f23b938afd00a3 (table default.Table_ff4e7021d36b43f8b51f19960e707ac8 [id=f17a11d223204599afb24e2e9d1d9d9d]) successfully deleted
I20260925 09:57:36.874957 12452 raft_consensus.cc:504] T 56f8c08025b94c6eb8b429bd7b0408aa P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:36.875783 11084 ts_tablet_manager.cc:1428] T c52745a8cdee461aba7485b0531245d6 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.189s user 0.004s sys 0.001s
I20260925 09:57:36.876632 11084 tablet_bootstrap.cc:492] T cc295f7a9169475f9a9235e657be629f P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:36.880700 12207 raft_consensus.cc:482] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b8850818e2ee42a082802d0095c08d21)
I20260925 09:57:36.881140 12207 raft_consensus.cc:504] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:36.881860 11084 tablet_bootstrap.cc:654] T cc295f7a9169475f9a9235e657be629f P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:36.883031 12207 leader_election.cc:288] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:36.884855 12452 leader_election.cc:288] T 56f8c08025b94c6eb8b429bd7b0408aa P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:36.885450 12452 raft_consensus.cc:482] T c52745a8cdee461aba7485b0531245d6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:36.885949 12452 raft_consensus.cc:504] T c52745a8cdee461aba7485b0531245d6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:36.887972 12452 leader_election.cc:288] T c52745a8cdee461aba7485b0531245d6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:36.895088 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56f8c08025b94c6eb8b429bd7b0408aa" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:36.897104 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c52745a8cdee461aba7485b0531245d6" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:36.899106 10830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa619b699004882a82976f0ef8204ad" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:36.899760 10830 raft_consensus.cc:2454] T cfa619b699004882a82976f0ef8204ad P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be97ab5d75934a648d4ab76341193ec0 in term 1.
I20260925 09:57:36.926622 12392 raft_consensus.cc:482] T 350904aa5957496d90a2aa2e637b591c P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:36.927088 12392 raft_consensus.cc:504] T 350904aa5957496d90a2aa2e637b591c P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:36.931658 12392 leader_election.cc:288] T 350904aa5957496d90a2aa2e637b591c P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:36.939806 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 332) took 5161 ms. Trace:
I20260925 09:57:36.940189 12027 rpcz_store.cc:276] 0925 09:57:31.778026 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:31.778126 (+ 100us) service_pool.cc:227] Handling call
0925 09:57:36.938853 (+5160727us) ts_tablet_manager.cc:1145] Deleting tablet 0a018bea4200474a9904f62aee37240b
0925 09:57:36.939482 (+ 629us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:36.939752 (+ 270us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5135752}
I20260925 09:57:36.941488 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 340) took 4953 ms. Trace:
I20260925 09:57:36.941838 12027 rpcz_store.cc:276] 0925 09:57:31.988514 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:31.988594 (+ 80us) service_pool.cc:227] Handling call
0925 09:57:36.941083 (+4952489us) ts_tablet_manager.cc:1145] Deleting tablet 476567ad4efa4043b6536f3f28d2ee28
0925 09:57:36.941134 (+ 51us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:36.941438 (+ 304us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4952000}
I20260925 09:57:36.943864 12027 tablet_replica.cc:330] T fb9c51f7e7fd4c029be2a710cb9462eb P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
W20260925 09:57:36.944764 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 0a018bea4200474a9904f62aee37240b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0a018bea4200474a9904f62aee37240b already in progress: creating tablet
I20260925 09:57:36.945115 12027 raft_consensus.cc:2229] T fb9c51f7e7fd4c029be2a710cb9462eb P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:36.945601 12027 raft_consensus.cc:2258] T fb9c51f7e7fd4c029be2a710cb9462eb P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 09:57:36.945833 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 476567ad4efa4043b6536f3f28d2ee28 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 476567ad4efa4043b6536f3f28d2ee28 already in progress: creating tablet
I20260925 09:57:36.864697 10387 catalog_manager.cc:5495] T 8aff53cfd4994c629a408ee8250280c5 P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
W20260925 09:57:36.898538 10918 leader_election.cc:341] T 88e53debe6de43e29b12e85e5171c8b5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:36.966045 10918 leader_election.cc:302] T 88e53debe6de43e29b12e85e5171c8b5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
W20260925 09:57:36.966969 10918 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 68459.2us
W20260925 09:57:36.967734 10918 leader_election.cc:341] T f3bba45d6b7241f9a07907c79b892f7d P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:36.968134 10918 leader_election.cc:302] T f3bba45d6b7241f9a07907c79b892f7d P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:36.968683 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56f8c08025b94c6eb8b429bd7b0408aa" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:36.969072 10918 leader_election.cc:341] T 2b1b2d66397b44f38056077d79941ac4 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:36.969374 10918 leader_election.cc:302] T 2b1b2d66397b44f38056077d79941ac4 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:36.970057 12207 raft_consensus.cc:2726] T 88e53debe6de43e29b12e85e5171c8b5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:36.970255 10918 leader_election.cc:341] T 56f8c08025b94c6eb8b429bd7b0408aa P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:36.970553 12207 raft_consensus.cc:2726] T f3bba45d6b7241f9a07907c79b892f7d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:36.970752 10918 leader_election.cc:341] T c52745a8cdee461aba7485b0531245d6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:36.971005 12207 raft_consensus.cc:2726] T 2b1b2d66397b44f38056077d79941ac4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:36.905151 10956 tablet_service.cc:1467] Processing CreateTablet for tablet e83d6dc8657d4a1f89ab0893f435abf6 (DEFAULT_TABLE table=default.Table_2a6203820e0d44aa993080049923de10 [id=c90c671ed4f043c2b56a1f84ff9a42cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:36.934754 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "350904aa5957496d90a2aa2e637b591c" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:36.971659 10976 raft_consensus.cc:2454] T 350904aa5957496d90a2aa2e637b591c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:36.972898 10956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e83d6dc8657d4a1f89ab0893f435abf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:36.973512 10619 leader_election.cc:302] T 350904aa5957496d90a2aa2e637b591c P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:36.974399 12392 raft_consensus.cc:2781] T 350904aa5957496d90a2aa2e637b591c P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:36.974895 12392 raft_consensus.cc:482] T 350904aa5957496d90a2aa2e637b591c P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:36.975281 12392 raft_consensus.cc:3037] T 350904aa5957496d90a2aa2e637b591c P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:36.976744 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c52745a8cdee461aba7485b0531245d6" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:36.980932 12257 consensus_queue.cc:1035] T a436fe287a0d48a1821191adcd031d74 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:36.998687 11085 tablet_bootstrap.cc:492] T dba14a44a9de492886d629441606767e P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:36.999181 11085 ts_tablet_manager.cc:1397] T dba14a44a9de492886d629441606767e P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.691s user 0.010s sys 0.001s
I20260925 09:57:37.008769 12257 consensus_queue.cc:1035] T aca5987b14b64152887ac5ee584b257a P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:37.018884 11084 tablet_bootstrap.cc:492] T cc295f7a9169475f9a9235e657be629f P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:37.019281 11084 ts_tablet_manager.cc:1397] T cc295f7a9169475f9a9235e657be629f P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.143s user 0.001s sys 0.008s
I20260925 09:57:37.021327 11084 raft_consensus.cc:348] T cc295f7a9169475f9a9235e657be629f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:37.021963 11084 raft_consensus.cc:374] T cc295f7a9169475f9a9235e657be629f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:37.022179 11084 raft_consensus.cc:729] T cc295f7a9169475f9a9235e657be629f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:37.023025 10668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6e82e83508440aa94028e71d140f8a3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:37.026049 10621 consensus_peers.cc:487] T 96a80382bb7d4395a14ad9ae972f4a25 P b8850818e2ee42a082802d0095c08d21 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260925 09:57:37.028769 10621 consensus_peers.cc:487] T a436fe287a0d48a1821191adcd031d74 P b8850818e2ee42a082802d0095c08d21 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:36.947263 12027 ts_tablet_manager.cc:1905] T fb9c51f7e7fd4c029be2a710cb9462eb P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:37.031641 10664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4dd5ebc3d654ef1852d2ee298a584a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:37.042635 12379 raft_consensus.cc:2229] T 6161774f61ae48ab9764a05f2188a80b P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:37.043655 10658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e83d6dc8657d4a1f89ab0893f435abf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:37.043754 12379 raft_consensus.cc:2258] T 6161774f61ae48ab9764a05f2188a80b P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:37.037798 11085 raft_consensus.cc:348] T dba14a44a9de492886d629441606767e P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:37.045563 11085 raft_consensus.cc:374] T dba14a44a9de492886d629441606767e P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:37.045826 11085 raft_consensus.cc:729] T dba14a44a9de492886d629441606767e P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:37.045709 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "350904aa5957496d90a2aa2e637b591c" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:37.046977 12379 ts_tablet_manager.cc:1905] T 6161774f61ae48ab9764a05f2188a80b P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:37.053577 11085 consensus_queue.cc:260] T dba14a44a9de492886d629441606767e P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:37.055572 10653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 154083d32e0f474f9e3a02e6cca940ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:37.031988 12392 raft_consensus.cc:504] T 350904aa5957496d90a2aa2e637b591c P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:37.066646 12483 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 09:57:37.067088 12392 leader_election.cc:288] T 350904aa5957496d90a2aa2e637b591c P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:37.071122 10978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "350904aa5957496d90a2aa2e637b591c" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:37.071807 10978 raft_consensus.cc:3037] T 350904aa5957496d90a2aa2e637b591c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:36.971266 10918 leader_election.cc:302] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters:
W20260925 09:57:37.094763 10919 leader_election.cc:341] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Not found: Tablet not found: cfa619b699004882a82976f0ef8204ad
I20260925 09:57:37.096350 11085 ts_tablet_manager.cc:1428] T dba14a44a9de492886d629441606767e P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.097s user 0.005s sys 0.000s
W20260925 09:57:37.096784 10919 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 99988.5us
W20260925 09:57:37.097499 10919 leader_election.cc:341] T 56f8c08025b94c6eb8b429bd7b0408aa P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:37.097906 10919 leader_election.cc:302] T 56f8c08025b94c6eb8b429bd7b0408aa P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
W20260925 09:57:37.098963 10919 leader_election.cc:341] T c52745a8cdee461aba7485b0531245d6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:37.099344 10919 leader_election.cc:302] T c52745a8cdee461aba7485b0531245d6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:37.087939 11082 tablet_bootstrap.cc:654] T 9744b9814b69470396ce039348fe5e45 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:37.100662 12452 raft_consensus.cc:2726] T 56f8c08025b94c6eb8b429bd7b0408aa P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:37.100881 12464 raft_consensus.cc:2726] T c52745a8cdee461aba7485b0531245d6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:37.102411 11084 consensus_queue.cc:260] T cc295f7a9169475f9a9235e657be629f P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:37.034157 10391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_891a8679f0f040fcbeff83cb61afed65"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:37.108678 10391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_891a8679f0f040fcbeff83cb61afed65 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:37.078510 10978 raft_consensus.cc:2454] T 350904aa5957496d90a2aa2e637b591c P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:37.120258 10619 leader_election.cc:302] T 350904aa5957496d90a2aa2e637b591c P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:37.121096 11084 ts_tablet_manager.cc:1428] T cc295f7a9169475f9a9235e657be629f P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.102s user 0.004s sys 0.001s
I20260925 09:57:37.122160 11084 tablet_bootstrap.cc:492] T f1d31dca90df414a83be7c9e404b8d30 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
W20260925 09:57:37.127808 10918 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 156580us
I20260925 09:57:37.128517 12465 raft_consensus.cc:2781] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 09:57:37.128932 12465 raft_consensus.cc:482] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b8850818e2ee42a082802d0095c08d21)
I20260925 09:57:37.129302 12465 raft_consensus.cc:3037] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:57:37.137598 11727 ts_tablet_manager.cc:1918] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260925 09:57:37.138151 11727 log.cc:1196] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/197f61525b0c4f9da441d98701d231a3
I20260925 09:57:37.139108 11727 ts_tablet_manager.cc:1939] T 197f61525b0c4f9da441d98701d231a3 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:37.148974 11082 tablet_bootstrap.cc:492] T 9744b9814b69470396ce039348fe5e45 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:37.149505 11082 ts_tablet_manager.cc:1397] T 9744b9814b69470396ce039348fe5e45 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.279s user 0.010s sys 0.006s
I20260925 09:57:37.153362 12390 raft_consensus.cc:2781] T 350904aa5957496d90a2aa2e637b591c P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:37.163854 12390 raft_consensus.cc:686] T 350904aa5957496d90a2aa2e637b591c P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:37.164726 12390 consensus_queue.cc:237] T 350904aa5957496d90a2aa2e637b591c P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:37.196586 11085 tablet_bootstrap.cc:492] T 37bab60afe7a4c758a08ef7c9c24aba0 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:37.206606 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 325) took 6492 ms. Trace:
W20260925 09:57:37.207578 10918 consensus_peers.cc:487] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:37.211606 12465 raft_consensus.cc:504] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:37.214143 10830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa619b699004882a82976f0ef8204ad" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:37.214705 10830 raft_consensus.cc:3037] T cfa619b699004882a82976f0ef8204ad P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:57:37.206941 11727 rpcz_store.cc:276] 0925 09:57:30.713785 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:30.713956 (+ 171us) service_pool.cc:227] Handling call
0925 09:57:36.159566 (+5445610us) ts_tablet_manager.cc:1145] Deleting tablet 197f61525b0c4f9da441d98701d231a3
0925 09:57:36.159618 (+ 52us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:36.849269 (+689651us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:37.137354 (+288085us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:37.206547 (+ 69193us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5444925}
I20260925 09:57:37.242427 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 197f61525b0c4f9da441d98701d231a3 (table default.Table_1299538965b945cf859dbbd0cb7003f1 [id=cdc838b49cbb40778c9f440e9f83e96d]) successfully deleted
I20260925 09:57:37.242983 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 326) took 6352 ms. Trace:
I20260925 09:57:37.244442 11082 raft_consensus.cc:348] T 9744b9814b69470396ce039348fe5e45 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:37.245086 11082 raft_consensus.cc:374] T 9744b9814b69470396ce039348fe5e45 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:37.245290 11082 raft_consensus.cc:729] T 9744b9814b69470396ce039348fe5e45 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:37.245779 11082 consensus_queue.cc:260] T 9744b9814b69470396ce039348fe5e45 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:37.251212 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "350904aa5957496d90a2aa2e637b591c" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:37.286929 11085 tablet_bootstrap.cc:654] T 37bab60afe7a4c758a08ef7c9c24aba0 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:37.292083 12379 ts_tablet_manager.cc:1918] T 6161774f61ae48ab9764a05f2188a80b P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:37.292521 12379 log.cc:1196] T 6161774f61ae48ab9764a05f2188a80b P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/6161774f61ae48ab9764a05f2188a80b
I20260925 09:57:37.293234 12379 ts_tablet_manager.cc:1939] T 6161774f61ae48ab9764a05f2188a80b P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:37.293977 12379 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 369) took 1650 ms. Trace:
I20260925 09:57:37.294384 12379 rpcz_store.cc:276] 0925 09:57:35.643885 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:35.652683 (+ 8798us) service_pool.cc:227] Handling call
0925 09:57:36.474193 (+821510us) ts_tablet_manager.cc:1145] Deleting tablet 6161774f61ae48ab9764a05f2188a80b
0925 09:57:36.474232 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:37.057367 (+583135us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:37.291854 (+234487us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:37.293932 (+ 2078us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":567385,"tablet-delete.queue_time_us":774317}
I20260925 09:57:37.295680 12379 tablet_replica.cc:330] T 545ac025f2cb469da847979011a0b3ce P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:37.296247 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 6161774f61ae48ab9764a05f2188a80b (table default.Table_64ba123898174e72a6884dbb9ad07de3 [id=7ee526720b2442119e0fc487b7c4fcd4]) successfully deleted
I20260925 09:57:37.296555 12379 raft_consensus.cc:2229] T 545ac025f2cb469da847979011a0b3ce P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:37.297228 12464 raft_consensus.cc:482] T cc295f7a9169475f9a9235e657be629f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:37.297317 12379 raft_consensus.cc:2258] T 545ac025f2cb469da847979011a0b3ce P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:37.297662 12464 raft_consensus.cc:504] T cc295f7a9169475f9a9235e657be629f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:37.299482 12464 leader_election.cc:288] T cc295f7a9169475f9a9235e657be629f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:37.300022 11082 ts_tablet_manager.cc:1428] T 9744b9814b69470396ce039348fe5e45 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.150s user 0.015s sys 0.013s
I20260925 09:57:37.300855 11082 tablet_bootstrap.cc:492] T 1e76f9e0ba554e1cad0cf18863393608 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:37.301198 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc295f7a9169475f9a9235e657be629f" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:37.301819 11084 tablet_bootstrap.cc:700] T f1d31dca90df414a83be7c9e404b8d30 P be97ab5d75934a648d4ab76341193ec0: Time spent opening tablet: real 0.175s user 0.004s sys 0.000s
I20260925 09:57:37.302405 11084 tablet_bootstrap.cc:654] T f1d31dca90df414a83be7c9e404b8d30 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:37.302544 10918 leader_election.cc:341] T cc295f7a9169475f9a9235e657be629f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:37.303710 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc295f7a9169475f9a9235e657be629f" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:37.305061 10919 leader_election.cc:341] T cc295f7a9169475f9a9235e657be629f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:37.305400 10919 leader_election.cc:302] T cc295f7a9169475f9a9235e657be629f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:37.243261 11727 rpcz_store.cc:276] 0925 09:57:30.890260 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:30.938134 (+ 47874us) service_pool.cc:227] Handling call
0925 09:57:37.242588 (+6304454us) ts_tablet_manager.cc:1145] Deleting tablet a58cf1dc81454710a9ad1a8baf155c9f
0925 09:57:37.242640 (+ 52us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:37.242931 (+ 291us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6163577}
I20260925 09:57:37.306581 12464 raft_consensus.cc:2726] T cc295f7a9169475f9a9235e657be629f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:37.306839 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 328) took 5744 ms. Trace:
W20260925 09:57:37.308816 10919 leader_election.cc:341] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Not found: Tablet not found: cfa619b699004882a82976f0ef8204ad
W20260925 09:57:37.311905 10621 leader_election.cc:341] T 350904aa5957496d90a2aa2e637b591c P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:37.314574 12489 raft_consensus.cc:482] T 69028aa7908c45b896da4ba8deef07b1 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:37.316474 10830 raft_consensus.cc:2454] T cfa619b699004882a82976f0ef8204ad P c43461c4ee854ae1ac52b873ccada318 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be97ab5d75934a648d4ab76341193ec0 in term 2.
I20260925 09:57:37.317595 10918 leader_election.cc:302] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21
I20260925 09:57:37.318365 12464 raft_consensus.cc:2781] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 09:57:37.318795 12464 raft_consensus.cc:686] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0 [term 2 LEADER]: Becoming Leader. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Running, Role: LEADER
I20260925 09:57:37.319605 12464 consensus_queue.cc:237] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:37.305491 12465 leader_election.cc:288] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:37.307114 11727 rpcz_store.cc:276] 0925 09:57:31.562175 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:31.562311 (+ 136us) service_pool.cc:227] Handling call
0925 09:57:37.306451 (+5744140us) ts_tablet_manager.cc:1145] Deleting tablet 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef
0925 09:57:37.306508 (+ 57us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:37.306792 (+ 284us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5743544}
W20260925 09:57:37.326701 10377 catalog_manager.cc:4821] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef already in progress: creating tablet
I20260925 09:57:37.326768 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 329) took 5686 ms. Trace:
I20260925 09:57:37.327034 11727 rpcz_store.cc:276] 0925 09:57:31.640779 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:31.640928 (+ 149us) service_pool.cc:227] Handling call
0925 09:57:37.326350 (+5685422us) ts_tablet_manager.cc:1145] Deleting tablet 0a018bea4200474a9904f62aee37240b
0925 09:57:37.326426 (+ 76us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:37.326723 (+ 297us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5684780}
W20260925 09:57:37.328217 10377 catalog_manager.cc:4821] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 0a018bea4200474a9904f62aee37240b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0a018bea4200474a9904f62aee37240b already in progress: creating tablet
I20260925 09:57:37.329490 11727 tablet_replica.cc:330] T dba14a44a9de492886d629441606767e P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:37.335280 11727 raft_consensus.cc:2229] T dba14a44a9de492886d629441606767e P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:37.335831 11727 raft_consensus.cc:2258] T dba14a44a9de492886d629441606767e P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:37.338091 11727 ts_tablet_manager.cc:1905] T dba14a44a9de492886d629441606767e P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:37.344771 10384 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 6540aba5854a44419addc5d454e473b7 (table default.Table_8180842f664042b0a9073b7008438ff7 [id=416a4b989570457e919fb7e7b16a017d]). Expected version 1 got 0
I20260925 09:57:37.346676 11084 tablet_bootstrap.cc:492] T f1d31dca90df414a83be7c9e404b8d30 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:37.347054 11084 ts_tablet_manager.cc:1397] T f1d31dca90df414a83be7c9e404b8d30 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.225s user 0.009s sys 0.001s
I20260925 09:57:37.350011 11084 raft_consensus.cc:348] T f1d31dca90df414a83be7c9e404b8d30 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:37.350651 11084 raft_consensus.cc:374] T f1d31dca90df414a83be7c9e404b8d30 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:37.350874 11084 raft_consensus.cc:729] T f1d31dca90df414a83be7c9e404b8d30 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:37.354403 10384 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet f62c62715c9b42de826c1a66fd431eab (table default.Table_bcbc82519cf043738f61029181f29418 [id=b3f7f21ab2214e4cb637bea27ce62abe]). Expected version 1 got 0
I20260925 09:57:37.359118 11084 consensus_queue.cc:260] T f1d31dca90df414a83be7c9e404b8d30 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:37.371258 12460 raft_consensus.cc:482] T dba14a44a9de492886d629441606767e P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:37.371739 12460 raft_consensus.cc:504] T dba14a44a9de492886d629441606767e P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:37.373483 12460 leader_election.cc:288] T dba14a44a9de492886d629441606767e P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:37.375007 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dba14a44a9de492886d629441606767e" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:37.375769 12489 raft_consensus.cc:504] T 69028aa7908c45b896da4ba8deef07b1 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:37.376509 10772 leader_election.cc:341] T dba14a44a9de492886d629441606767e P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Illegal state: Tablet not RUNNING: STOPPED
I20260925 09:57:37.377765 12489 leader_election.cc:288] T 69028aa7908c45b896da4ba8deef07b1 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:37.383955 11084 ts_tablet_manager.cc:1428] T f1d31dca90df414a83be7c9e404b8d30 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.037s user 0.003s sys 0.004s
I20260925 09:57:37.384723 12027 ts_tablet_manager.cc:1918] T fb9c51f7e7fd4c029be2a710cb9462eb P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:37.385049 12027 log.cc:1196] T fb9c51f7e7fd4c029be2a710cb9462eb P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/fb9c51f7e7fd4c029be2a710cb9462eb
I20260925 09:57:37.385670 12027 ts_tablet_manager.cc:1939] T fb9c51f7e7fd4c029be2a710cb9462eb P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:37.386895 11084 tablet_bootstrap.cc:492] T 2d79538d6474433fbda26c91645a0619 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:37.392653 10384 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 2f07dc9548364870923165f4ea524310 (table default.Table_0db927e92a3b4970a7efea3fb765e65e [id=a988264fe43c4eb88a84f8b6359e77a6]). Expected version 1 got 0
I20260925 09:57:37.398518 11082 tablet_bootstrap.cc:654] T 1e76f9e0ba554e1cad0cf18863393608 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:37.403175 10621 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 87379.9us
W20260925 09:57:37.403781 10619 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 88420.4us
I20260925 09:57:37.406868 11085 tablet_bootstrap.cc:492] T 37bab60afe7a4c758a08ef7c9c24aba0 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
W20260925 09:57:37.405090 10621 leader_election.cc:341] T 350904aa5957496d90a2aa2e637b591c P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:37.407456 11085 ts_tablet_manager.cc:1397] T 37bab60afe7a4c758a08ef7c9c24aba0 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.211s user 0.008s sys 0.002s
I20260925 09:57:37.409657 11085 raft_consensus.cc:348] T 37bab60afe7a4c758a08ef7c9c24aba0 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:37.410516 11085 raft_consensus.cc:374] T 37bab60afe7a4c758a08ef7c9c24aba0 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:37.410849 11085 raft_consensus.cc:729] T 37bab60afe7a4c758a08ef7c9c24aba0 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:37.411581 11085 consensus_queue.cc:260] T 37bab60afe7a4c758a08ef7c9c24aba0 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:37.418288 10621 consensus_peers.cc:487] T aca5987b14b64152887ac5ee584b257a P b8850818e2ee42a082802d0095c08d21 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:37.424026 11727 ts_tablet_manager.cc:1918] T dba14a44a9de492886d629441606767e P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:37.424687 11727 log.cc:1196] T dba14a44a9de492886d629441606767e P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/dba14a44a9de492886d629441606767e
I20260925 09:57:37.425527 11727 ts_tablet_manager.cc:1939] T dba14a44a9de492886d629441606767e P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:37.442097 10830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69028aa7908c45b896da4ba8deef07b1" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:37.445358 11085 ts_tablet_manager.cc:1428] T 37bab60afe7a4c758a08ef7c9c24aba0 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.038s user 0.004s sys 0.000s
I20260925 09:57:37.446287 11085 tablet_bootstrap.cc:492] T 10657a81058241fc988d37e5ea7ff8cf P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:37.398536 12379 ts_tablet_manager.cc:1905] T 545ac025f2cb469da847979011a0b3ce P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:37.448381 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dba14a44a9de492886d629441606767e" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:37.452459 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 341) took 5463 ms. Trace:
I20260925 09:57:37.452759 12027 rpcz_store.cc:276] 0925 09:57:31.988765 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:31.988854 (+ 89us) service_pool.cc:227] Handling call
0925 09:57:36.943712 (+4954858us) ts_tablet_manager.cc:1145] Deleting tablet fb9c51f7e7fd4c029be2a710cb9462eb
0925 09:57:36.943762 (+ 50us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:37.283125 (+339363us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:37.384483 (+101358us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:37.452405 (+ 67922us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4954403}
I20260925 09:57:37.453881 12027 tablet_replica.cc:330] T dba14a44a9de492886d629441606767e P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:37.453981 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet fb9c51f7e7fd4c029be2a710cb9462eb (table default.Table_44ff944ba4b84cb7a677f7dd03f8f7dd [id=80ab4d354310414eb9374b8ac6a3a946]) successfully deleted
I20260925 09:57:37.454592 12027 raft_consensus.cc:2229] T dba14a44a9de492886d629441606767e P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:37.455014 12027 raft_consensus.cc:2258] T dba14a44a9de492886d629441606767e P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:37.480896 11084 tablet_bootstrap.cc:654] T 2d79538d6474433fbda26c91645a0619 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:37.492092 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 336) took 5657 ms. Trace:
I20260925 09:57:37.492496 11727 rpcz_store.cc:276] 0925 09:57:31.835015 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:31.835153 (+ 138us) service_pool.cc:227] Handling call
0925 09:57:37.328447 (+5493294us) ts_tablet_manager.cc:1145] Deleting tablet dba14a44a9de492886d629441606767e
0925 09:57:37.329391 (+ 944us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:37.350685 (+ 21294us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:37.423739 (+ 73054us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:37.492033 (+ 68294us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":35788,"tablet-delete.queue_time_us":5492731}
I20260925 09:57:37.495916 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet dba14a44a9de492886d629441606767e (table default.Table_bb742c618d96453aaabe897d54fc657b [id=68683893b91c4f0190a0050ecdc6d396]) successfully deleted
I20260925 09:57:37.496074 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 337) took 5565 ms. Trace:
I20260925 09:57:37.496428 11727 rpcz_store.cc:276] 0925 09:57:31.930638 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:31.930770 (+ 132us) service_pool.cc:227] Handling call
0925 09:57:37.495710 (+5564940us) ts_tablet_manager.cc:1145] Deleting tablet 476567ad4efa4043b6536f3f28d2ee28
0925 09:57:37.495773 (+ 63us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:37.496028 (+ 255us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5564366}
I20260925 09:57:37.497462 11727 tablet_replica.cc:330] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
W20260925 09:57:37.497519 10377 catalog_manager.cc:4821] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 476567ad4efa4043b6536f3f28d2ee28 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 476567ad4efa4043b6536f3f28d2ee28 already in progress: creating tablet
I20260925 09:57:37.498406 11727 raft_consensus.cc:2229] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0 [term 1 LEADER]: Raft consensus shutting down.
W20260925 09:57:37.509413 10621 leader_election.cc:341] T 69028aa7908c45b896da4ba8deef07b1 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:37.496817 10775 leader_election.cc:302] T dba14a44a9de492886d629441606767e P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
W20260925 09:57:37.510432 10775 raft_consensus.cc:2670] T dba14a44a9de492886d629441606767e P c43461c4ee854ae1ac52b873ccada318: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260925 09:57:37.499337 11727 raft_consensus.cc:2258] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:37.530196 11085 tablet_bootstrap.cc:654] T 10657a81058241fc988d37e5ea7ff8cf P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:37.538834 12464 raft_consensus.cc:482] T 1e76f9e0ba554e1cad0cf18863393608 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:37.539331 12464 raft_consensus.cc:504] T 1e76f9e0ba554e1cad0cf18863393608 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:37.540647 12027 ts_tablet_manager.cc:1905] T dba14a44a9de492886d629441606767e P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:37.542155 10830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e76f9e0ba554e1cad0cf18863393608" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:37.542588 12464 leader_election.cc:288] T 1e76f9e0ba554e1cad0cf18863393608 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
W20260925 09:57:37.543532 10918 leader_election.cc:341] T 1e76f9e0ba554e1cad0cf18863393608 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:37.560065 11727 ts_tablet_manager.cc:1905] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:37.561004 11084 tablet_bootstrap.cc:492] T 2d79538d6474433fbda26c91645a0619 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:37.561445 11084 ts_tablet_manager.cc:1397] T 2d79538d6474433fbda26c91645a0619 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.175s user 0.009s sys 0.002s
I20260925 09:57:37.563900 11084 raft_consensus.cc:348] T 2d79538d6474433fbda26c91645a0619 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:37.564571 11084 raft_consensus.cc:374] T 2d79538d6474433fbda26c91645a0619 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:37.565018 11084 raft_consensus.cc:729] T 2d79538d6474433fbda26c91645a0619 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:37.565682 11084 consensus_queue.cc:260] T 2d79538d6474433fbda26c91645a0619 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:37.555447 10393 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 6540aba5854a44419addc5d454e473b7 (table default.Table_8180842f664042b0a9073b7008438ff7 [id=416a4b989570457e919fb7e7b16a017d]). Expected version 1 got 0
I20260925 09:57:37.567452 11084 ts_tablet_manager.cc:1428] T 2d79538d6474433fbda26c91645a0619 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260925 09:57:37.568289 11084 tablet_bootstrap.cc:492] T a58cf1dc81454710a9ad1a8baf155c9f P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:37.592422 10393 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet f62c62715c9b42de826c1a66fd431eab (table default.Table_bcbc82519cf043738f61029181f29418 [id=b3f7f21ab2214e4cb637bea27ce62abe]). Expected version 1 got 0
W20260925 09:57:37.603915 10621 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 90715.8us
I20260925 09:57:37.613032 11085 tablet_bootstrap.cc:492] T 10657a81058241fc988d37e5ea7ff8cf P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:37.613495 11085 ts_tablet_manager.cc:1397] T 10657a81058241fc988d37e5ea7ff8cf P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.167s user 0.009s sys 0.001s
I20260925 09:57:37.621558 12379 ts_tablet_manager.cc:1918] T 545ac025f2cb469da847979011a0b3ce P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:37.622046 12379 log.cc:1196] T 545ac025f2cb469da847979011a0b3ce P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/545ac025f2cb469da847979011a0b3ce
I20260925 09:57:37.622879 12379 ts_tablet_manager.cc:1939] T 545ac025f2cb469da847979011a0b3ce P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:37.626209 11085 raft_consensus.cc:348] T 10657a81058241fc988d37e5ea7ff8cf P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:37.626926 11085 raft_consensus.cc:374] T 10657a81058241fc988d37e5ea7ff8cf P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:37.627235 11085 raft_consensus.cc:729] T 10657a81058241fc988d37e5ea7ff8cf P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:37.627857 11085 consensus_queue.cc:260] T 10657a81058241fc988d37e5ea7ff8cf P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:37.630180 11085 ts_tablet_manager.cc:1428] T 10657a81058241fc988d37e5ea7ff8cf P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.016s user 0.004s sys 0.000s
I20260925 09:57:37.631192 10393 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 2f07dc9548364870923165f4ea524310 (table default.Table_0db927e92a3b4970a7efea3fb765e65e [id=a988264fe43c4eb88a84f8b6359e77a6]). Expected version 1 got 0
I20260925 09:57:37.635030 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e76f9e0ba554e1cad0cf18863393608" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:37.636103 12379 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 378) took 1923 ms. Trace:
I20260925 09:57:37.636499 12379 rpcz_store.cc:276] 0925 09:57:35.713101 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:35.713219 (+ 118us) service_pool.cc:227] Handling call
0925 09:57:37.295540 (+1582321us) ts_tablet_manager.cc:1145] Deleting tablet 545ac025f2cb469da847979011a0b3ce
0925 09:57:37.295581 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:37.492364 (+196783us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:37.621328 (+128964us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:37.636049 (+ 14721us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":83641,"tablet-delete.queue_time_us":1581686}
W20260925 09:57:37.637758 10919 leader_election.cc:341] T 1e76f9e0ba554e1cad0cf18863393608 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:37.638110 10919 leader_election.cc:302] T 1e76f9e0ba554e1cad0cf18863393608 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:37.639098 12464 raft_consensus.cc:2726] T 1e76f9e0ba554e1cad0cf18863393608 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:37.644016 10645 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:37864 (request call id 32) took 1292 ms. Trace:
I20260925 09:57:37.649698 10645 rpcz_store.cc:276] 0925 09:57:36.351913 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:36.352535 (+ 622us) service_pool.cc:227] Handling call
0925 09:57:36.830164 (+477629us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b54001086d4. stack: 00007fb965334033 00007fb965333f30 00007fb964eb5534 00000000004db2b4 00000000004db249 00007fb96c240471 00007fb96c2b5268 00007fb96d2a16cd 00007fb96bb49e04 00007fb96bb49d92 00007fb96bb49cc1 00007fb96bb49c3c 00007fb96bb48ef2 00007fb96a06a94c 00007fb96a069d86 00007fb96a06972f
0925 09:57:37.288021 (+457857us) spinlock_profiling.cc:243] Waited 2.44 ms on lock 0x7b54000127d4. stack: 00007fb965334033 00007fb965333f30 00007fb964eb5534 00000000004db2b4 00000000004db249 00007fb96c240471 00007fb96c2b5268 00007fb96d2a16cd 00007fb96bb49e04 00007fb96bb49d92 00007fb96bb49cc1 00007fb96bb49c3c 00007fb96bb48ef2 00007fb96a06a94c 00007fb96a069d86 00007fb96a06972f
0925 09:57:37.643955 (+355934us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":449122048}
I20260925 09:57:37.664489 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 545ac025f2cb469da847979011a0b3ce (table default.Table_14d12f6871c847daa860c3a0cfee4231 [id=f61297f0d10c4656bd06a752c9c7596b]) successfully deleted
I20260925 09:57:37.676223 11085 tablet_bootstrap.cc:492] T 92814437d53143a79f99199f8ee3a742 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:37.682260 11085 tablet_bootstrap.cc:654] T 92814437d53143a79f99199f8ee3a742 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:37.688233 12464 raft_consensus.cc:482] T 05e2c7f9820f41679c4d58343ff3f829 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:37.688697 12464 raft_consensus.cc:504] T 05e2c7f9820f41679c4d58343ff3f829 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:37.690634 12464 leader_election.cc:288] T 05e2c7f9820f41679c4d58343ff3f829 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:37.692579 12464 raft_consensus.cc:482] T d54ab15dd054489ab50b7f0b26b79a50 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:37.693029 12464 raft_consensus.cc:504] T d54ab15dd054489ab50b7f0b26b79a50 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:37.696899 12464 leader_election.cc:288] T d54ab15dd054489ab50b7f0b26b79a50 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:37.699106 11727 ts_tablet_manager.cc:1918] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:37.712643 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05e2c7f9820f41679c4d58343ff3f829" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:37.712773 10673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d54ab15dd054489ab50b7f0b26b79a50" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:37.672986 12379 tablet_replica.cc:330] T 417058493c6547c69e72347cef373292 P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:37.718122 12379 raft_consensus.cc:2229] T 417058493c6547c69e72347cef373292 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260925 09:57:37.718359 10919 leader_election.cc:341] T 05e2c7f9820f41679c4d58343ff3f829 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:37.718616 12379 raft_consensus.cc:2258] T 417058493c6547c69e72347cef373292 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 09:57:37.719280 10919 leader_election.cc:341] T d54ab15dd054489ab50b7f0b26b79a50 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:37.731181 11082 tablet_bootstrap.cc:492] T 1e76f9e0ba554e1cad0cf18863393608 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:37.731673 11082 ts_tablet_manager.cc:1397] T 1e76f9e0ba554e1cad0cf18863393608 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.431s user 0.010s sys 0.012s
I20260925 09:57:37.695727 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d54ab15dd054489ab50b7f0b26b79a50" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:37.701824 10830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05e2c7f9820f41679c4d58343ff3f829" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:37.737949 10918 leader_election.cc:341] T d54ab15dd054489ab50b7f0b26b79a50 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:37.738288 10918 leader_election.cc:302] T d54ab15dd054489ab50b7f0b26b79a50 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:37.738909 12464 raft_consensus.cc:2726] T d54ab15dd054489ab50b7f0b26b79a50 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:37.739264 10918 leader_election.cc:341] T 05e2c7f9820f41679c4d58343ff3f829 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:37.739616 10918 leader_election.cc:302] T 05e2c7f9820f41679c4d58343ff3f829 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:37.740675 12464 raft_consensus.cc:2726] T 05e2c7f9820f41679c4d58343ff3f829 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:37.742925 10282 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 68930.1us
I20260925 09:57:37.753738 11727 log.cc:1196] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/fb9c51f7e7fd4c029be2a710cb9462eb
I20260925 09:57:37.754570 11727 ts_tablet_manager.cc:1939] T fb9c51f7e7fd4c029be2a710cb9462eb P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:37.756393 11084 tablet_bootstrap.cc:654] T a58cf1dc81454710a9ad1a8baf155c9f P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:37.756714 11082 raft_consensus.cc:348] T 1e76f9e0ba554e1cad0cf18863393608 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:37.757455 11082 raft_consensus.cc:374] T 1e76f9e0ba554e1cad0cf18863393608 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:37.757747 11082 raft_consensus.cc:729] T 1e76f9e0ba554e1cad0cf18863393608 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:37.758540 11082 consensus_queue.cc:260] T 1e76f9e0ba554e1cad0cf18863393608 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:37.708495 12027 ts_tablet_manager.cc:1918] T dba14a44a9de492886d629441606767e P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:37.762332 12027 log.cc:1196] T dba14a44a9de492886d629441606767e P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/dba14a44a9de492886d629441606767e
I20260925 09:57:37.763113 12027 ts_tablet_manager.cc:1939] T dba14a44a9de492886d629441606767e P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
W20260925 09:57:37.763254 10619 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 337824us
I20260925 09:57:37.758540 12379 ts_tablet_manager.cc:1905] T 417058493c6547c69e72347cef373292 P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:37.779253 11082 ts_tablet_manager.cc:1428] T 1e76f9e0ba554e1cad0cf18863393608 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.047s user 0.004s sys 0.002s
I20260925 09:57:37.784137 11082 tablet_bootstrap.cc:492] T 9a73a1f242e7430cbf52229c07b4dc82 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:37.792793 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 339) took 5804 ms. Trace:
I20260925 09:57:37.793107 12027 rpcz_store.cc:276] 0925 09:57:31.988264 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:31.988350 (+ 86us) service_pool.cc:227] Handling call
0925 09:57:37.453707 (+5465357us) ts_tablet_manager.cc:1145] Deleting tablet dba14a44a9de492886d629441606767e
0925 09:57:37.453772 (+ 65us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:37.649854 (+196082us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:37.708280 (+ 58426us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:37.792733 (+ 84453us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5461335}
I20260925 09:57:37.794078 12027 tablet_replica.cc:330] T 417058493c6547c69e72347cef373292 P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:37.794956 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet dba14a44a9de492886d629441606767e (table default.Table_bb742c618d96453aaabe897d54fc657b [id=68683893b91c4f0190a0050ecdc6d396]) successfully deleted
I20260925 09:57:37.796844 11085 tablet_bootstrap.cc:492] T 92814437d53143a79f99199f8ee3a742 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:37.800122 11085 ts_tablet_manager.cc:1397] T 92814437d53143a79f99199f8ee3a742 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.124s user 0.004s sys 0.005s
I20260925 09:57:37.756645 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 338) took 5791 ms. Trace:
I20260925 09:57:37.803351 12460 raft_consensus.cc:482] T 10657a81058241fc988d37e5ea7ff8cf P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:37.820526 12460 raft_consensus.cc:504] T 10657a81058241fc988d37e5ea7ff8cf P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:37.824601 12460 leader_election.cc:288] T 10657a81058241fc988d37e5ea7ff8cf P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
W20260925 09:57:37.825331 10772 leader_election.cc:341] T 10657a81058241fc988d37e5ea7ff8cf P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 10657a81058241fc988d37e5ea7ff8cf
I20260925 09:57:37.825537 12460 raft_consensus.cc:482] T 37bab60afe7a4c758a08ef7c9c24aba0 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:37.825927 12460 raft_consensus.cc:504] T 37bab60afe7a4c758a08ef7c9c24aba0 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:37.827929 12460 leader_election.cc:288] T 37bab60afe7a4c758a08ef7c9c24aba0 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
W20260925 09:57:37.829239 10772 leader_election.cc:341] T 37bab60afe7a4c758a08ef7c9c24aba0 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 37bab60afe7a4c758a08ef7c9c24aba0
W20260925 09:57:37.838016 10378 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet b025f8ba34694ba09cb01ca959d3629c (table default.Table_2702858b51514e7f9a51a2bf19d36b1f [id=65b6e9ec3f114cea85e70f1509e3d1b6], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet b025f8ba34694ba09cb01ca959d3629c consensus state has no leader
I20260925 09:57:37.802614 11727 rpcz_store.cc:276] 0925 09:57:31.965186 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:31.965320 (+ 134us) service_pool.cc:227] Handling call
0925 09:57:37.497314 (+5531994us) ts_tablet_manager.cc:1145] Deleting tablet fb9c51f7e7fd4c029be2a710cb9462eb
0925 09:57:37.497363 (+ 49us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:37.567375 (+ 70012us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:37.698902 (+131527us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:37.756591 (+ 57689us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5531369}
I20260925 09:57:37.842306 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet fb9c51f7e7fd4c029be2a710cb9462eb (table default.Table_44ff944ba4b84cb7a677f7dd03f8f7dd [id=80ab4d354310414eb9374b8ac6a3a946]) successfully deleted
I20260925 09:57:37.842502 11727 tablet_replica.cc:330] T 1e76f9e0ba554e1cad0cf18863393608 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:37.843199 12500 raft_consensus.cc:482] T 9744b9814b69470396ce039348fe5e45 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:37.843343 11727 raft_consensus.cc:2229] T 1e76f9e0ba554e1cad0cf18863393608 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:37.843676 12500 raft_consensus.cc:504] T 9744b9814b69470396ce039348fe5e45 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:37.845561 12500 leader_election.cc:288] T 9744b9814b69470396ce039348fe5e45 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:37.845908 12464 raft_consensus.cc:482] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b8850818e2ee42a082802d0095c08d21)
I20260925 09:57:37.846362 12464 raft_consensus.cc:504] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:37.847641 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10657a81058241fc988d37e5ea7ff8cf" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:37.848189 10687 raft_consensus.cc:2454] T 10657a81058241fc988d37e5ea7ff8cf P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
I20260925 09:57:37.848057 10673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37bab60afe7a4c758a08ef7c9c24aba0" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:37.849728 10830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9744b9814b69470396ce039348fe5e45" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:37.851521 10919 leader_election.cc:341] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Not found: Tablet not found: a21909e2e5794f57971f4e42795235c7
I20260925 09:57:37.852455 10775 leader_election.cc:302] T 10657a81058241fc988d37e5ea7ff8cf P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters: be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:37.853559 12506 raft_consensus.cc:2781] T 10657a81058241fc988d37e5ea7ff8cf P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:37.853544 12464 leader_election.cc:288] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:37.853860 12506 raft_consensus.cc:482] T 10657a81058241fc988d37e5ea7ff8cf P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:37.854162 12506 raft_consensus.cc:3037] T 10657a81058241fc988d37e5ea7ff8cf P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:37.858916 10830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a21909e2e5794f57971f4e42795235c7" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:37.859846 12510 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8af3f111e8da4d2dad4e4a98ef0fd5d3)
I20260925 09:57:37.860309 12510 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } }
I20260925 09:57:37.860685 10918 leader_election.cc:302] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:37.861418 12464 raft_consensus.cc:2726] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 09:57:37.864959 12506 raft_consensus.cc:504] T 10657a81058241fc988d37e5ea7ff8cf P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:37.869230 12506 leader_election.cc:288] T 10657a81058241fc988d37e5ea7ff8cf P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
W20260925 09:57:37.869948 10621 leader_election.cc:341] T 9744b9814b69470396ce039348fe5e45 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:37.870730 10775 leader_election.cc:302] T 37bab60afe7a4c758a08ef7c9c24aba0 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
W20260925 09:57:37.872561 10772 leader_election.cc:341] T 10657a81058241fc988d37e5ea7ff8cf P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 10657a81058241fc988d37e5ea7ff8cf
I20260925 09:57:37.876034 10673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10657a81058241fc988d37e5ea7ff8cf" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:37.876580 10673 raft_consensus.cc:3037] T 10657a81058241fc988d37e5ea7ff8cf P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:37.877082 11084 tablet_bootstrap.cc:492] T a58cf1dc81454710a9ad1a8baf155c9f P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:37.877655 11084 ts_tablet_manager.cc:1397] T a58cf1dc81454710a9ad1a8baf155c9f P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.310s user 0.006s sys 0.004s
I20260925 09:57:37.883508 10673 raft_consensus.cc:2454] T 10657a81058241fc988d37e5ea7ff8cf P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:37.883553 12506 raft_consensus.cc:2726] T 37bab60afe7a4c758a08ef7c9c24aba0 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:37.885713 11085 raft_consensus.cc:348] T 92814437d53143a79f99199f8ee3a742 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:37.886457 11085 raft_consensus.cc:374] T 92814437d53143a79f99199f8ee3a742 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:37.886847 11085 raft_consensus.cc:729] T 92814437d53143a79f99199f8ee3a742 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:37.887595 11085 consensus_queue.cc:260] T 92814437d53143a79f99199f8ee3a742 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:37.889354 11085 ts_tablet_manager.cc:1428] T 92814437d53143a79f99199f8ee3a742 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.063s user 0.004s sys 0.000s
I20260925 09:57:37.889461 12027 raft_consensus.cc:2229] T 417058493c6547c69e72347cef373292 P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:37.890151 11085 tablet_bootstrap.cc:492] T b61444c31dbf4882ac748f861ea66818 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:37.890273 12027 raft_consensus.cc:2258] T 417058493c6547c69e72347cef373292 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:37.892913 12027 ts_tablet_manager.cc:1905] T 417058493c6547c69e72347cef373292 P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:37.897891 10775 leader_election.cc:302] T 10657a81058241fc988d37e5ea7ff8cf P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters: be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:37.899199 12460 raft_consensus.cc:2781] T 10657a81058241fc988d37e5ea7ff8cf P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:37.899895 12460 raft_consensus.cc:686] T 10657a81058241fc988d37e5ea7ff8cf P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Becoming Leader. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Running, Role: LEADER
I20260925 09:57:37.904312 12445 raft_consensus.cc:482] T 92814437d53143a79f99199f8ee3a742 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:37.904757 12445 raft_consensus.cc:504] T 92814437d53143a79f99199f8ee3a742 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:37.905382 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9744b9814b69470396ce039348fe5e45" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:37.905932 10976 raft_consensus.cc:2454] T 9744b9814b69470396ce039348fe5e45 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:37.906533 12445 leader_election.cc:288] T 92814437d53143a79f99199f8ee3a742 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:37.908020 12460 consensus_queue.cc:237] T 10657a81058241fc988d37e5ea7ff8cf P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:37.910136 10619 leader_election.cc:341] T 69028aa7908c45b896da4ba8deef07b1 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 69028aa7908c45b896da4ba8deef07b1
I20260925 09:57:37.910504 10619 leader_election.cc:302] T 69028aa7908c45b896da4ba8deef07b1 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8850818e2ee42a082802d0095c08d21; no voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:37.911127 12501 raft_consensus.cc:2726] T 69028aa7908c45b896da4ba8deef07b1 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:37.912618 10772 leader_election.cc:341] T 92814437d53143a79f99199f8ee3a742 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 92814437d53143a79f99199f8ee3a742
I20260925 09:57:37.920084 10403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "54c8485a7dcc47a1a75d08ef57647ece" candidate_term: 5 candidate_status { last_received { term: 4 index: 468 } } ignore_live_leader: false dest_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" is_pre_election: true
I20260925 09:57:37.843809 11727 raft_consensus.cc:2258] T 1e76f9e0ba554e1cad0cf18863393608 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:37.935163 11727 ts_tablet_manager.cc:1905] T 1e76f9e0ba554e1cad0cf18863393608 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:37.951920 11082 tablet_bootstrap.cc:654] T 9a73a1f242e7430cbf52229c07b4dc82 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:37.952600 12510 leader_election.cc:288] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8af3f111e8da4d2dad4e4a98ef0fd5d3 (127.10.9.61:33047), b8be057e0b6d40a3a4b90f2b1f4a0fa2 (127.10.9.60:36985)
I20260925 09:57:37.955881 11084 raft_consensus.cc:348] T a58cf1dc81454710a9ad1a8baf155c9f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:37.956604 11084 raft_consensus.cc:374] T a58cf1dc81454710a9ad1a8baf155c9f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:37.956877 11084 raft_consensus.cc:729] T a58cf1dc81454710a9ad1a8baf155c9f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:37.959147 10673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92814437d53143a79f99199f8ee3a742" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:37.959846 10673 raft_consensus.cc:2454] T 92814437d53143a79f99199f8ee3a742 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
I20260925 09:57:37.967661 11084 consensus_queue.cc:260] T a58cf1dc81454710a9ad1a8baf155c9f P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:37.977478 10646 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:37864 (request call id 33) took 1050 ms. Trace:
I20260925 09:57:37.978032 10646 rpcz_store.cc:276] 0925 09:57:36.927116 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:36.927228 (+ 112us) service_pool.cc:227] Handling call
0925 09:57:37.303891 (+376663us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b5400152554. stack: 00007fb965334033 00007fb965333f30 00007fb964eb5534 00000000004db2b4 00000000004db249 00007fb96c240471 00007fb96c2b5268 00007fb96d2a16cd 00007fb96bb49e04 00007fb96bb49d92 00007fb96bb49cc1 00007fb96bb49c3c 00007fb96bb48ef2 00007fb96a06a94c 00007fb96a069d86 00007fb96a06972f
0925 09:57:37.977415 (+673524us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":358959360}
W20260925 09:57:37.980722 10619 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 70612.4us
I20260925 09:57:37.982352 10775 leader_election.cc:302] T 92814437d53143a79f99199f8ee3a742 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters: be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:37.983414 12460 raft_consensus.cc:2781] T 92814437d53143a79f99199f8ee3a742 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:37.983788 12460 raft_consensus.cc:482] T 92814437d53143a79f99199f8ee3a742 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:37.984155 12460 raft_consensus.cc:3037] T 92814437d53143a79f99199f8ee3a742 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:57:37.997112 10621 consensus_peers.cc:487] T f62c62715c9b42de826c1a66fd431eab P b8850818e2ee42a082802d0095c08d21 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:37.999917 12315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "54c8485a7dcc47a1a75d08ef57647ece" candidate_term: 5 candidate_status { last_received { term: 4 index: 468 } } ignore_live_leader: false dest_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" is_pre_election: true
I20260925 09:57:38.001765 10304 leader_election.cc:302] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54c8485a7dcc47a1a75d08ef57647ece; no voters: 8af3f111e8da4d2dad4e4a98ef0fd5d3, b8be057e0b6d40a3a4b90f2b1f4a0fa2
I20260925 09:57:38.002522 12510 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260925 09:57:38.019505 11084 ts_tablet_manager.cc:1428] T a58cf1dc81454710a9ad1a8baf155c9f P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.142s user 0.007s sys 0.000s
I20260925 09:57:38.020416 11084 tablet_bootstrap.cc:492] T f0bf545f70ae4cd6a1ad2540c7158ecc P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:38.025707 12506 raft_consensus.cc:482] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b8850818e2ee42a082802d0095c08d21)
I20260925 09:57:38.026189 12506 raft_consensus.cc:504] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:38.029547 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a21909e2e5794f57971f4e42795235c7" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:38.030194 10976 raft_consensus.cc:2454] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:38.031277 10772 leader_election.cc:302] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:38.031953 12445 raft_consensus.cc:2781] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 09:57:38.032240 12445 raft_consensus.cc:482] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b8850818e2ee42a082802d0095c08d21)
I20260925 09:57:38.032536 12445 raft_consensus.cc:3037] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:57:38.043339 12445 raft_consensus.cc:504] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:38.047585 12506 leader_election.cc:288] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:38.049830 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a21909e2e5794f57971f4e42795235c7" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:38.050365 10976 raft_consensus.cc:3037] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:57:38.056183 12445 leader_election.cc:288] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:38.060479 10976 raft_consensus.cc:2454] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 2.
I20260925 09:57:38.062335 10772 leader_election.cc:302] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:38.066159 12506 raft_consensus.cc:2781] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 09:57:38.066666 12506 raft_consensus.cc:686] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318 [term 2 LEADER]: Becoming Leader. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Running, Role: LEADER
I20260925 09:57:38.070591 12506 consensus_queue.cc:237] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:38.074700 12460 raft_consensus.cc:504] T 92814437d53143a79f99199f8ee3a742 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:38.083763 10619 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 56874.8us
I20260925 09:57:38.085016 10619 leader_election.cc:302] T 9744b9814b69470396ce039348fe5e45 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:38.088532 11085 tablet_bootstrap.cc:654] T b61444c31dbf4882ac748f861ea66818 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:38.089596 10772 leader_election.cc:341] T 92814437d53143a79f99199f8ee3a742 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 92814437d53143a79f99199f8ee3a742
I20260925 09:57:38.090160 12460 leader_election.cc:288] T 92814437d53143a79f99199f8ee3a742 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:38.094144 12491 raft_consensus.cc:2781] T 9744b9814b69470396ce039348fe5e45 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:38.094471 12491 raft_consensus.cc:482] T 9744b9814b69470396ce039348fe5e45 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:38.094789 12491 raft_consensus.cc:3037] T 9744b9814b69470396ce039348fe5e45 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:38.096483 12390 consensus_queue.cc:1035] T f62c62715c9b42de826c1a66fd431eab P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:38.110610 11727 ts_tablet_manager.cc:1918] T 1e76f9e0ba554e1cad0cf18863393608 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:38.110987 11727 log.cc:1196] T 1e76f9e0ba554e1cad0cf18863393608 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/1e76f9e0ba554e1cad0cf18863393608
W20260925 09:57:38.105665 10619 consensus_peers.cc:487] T dba14a44a9de492886d629441606767e P b8850818e2ee42a082802d0095c08d21 -> Peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Couldn't send request to peer be97ab5d75934a648d4ab76341193ec0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dba14a44a9de492886d629441606767e. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:38.120256 11082 tablet_bootstrap.cc:492] T 9a73a1f242e7430cbf52229c07b4dc82 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:38.123658 11082 ts_tablet_manager.cc:1397] T 9a73a1f242e7430cbf52229c07b4dc82 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.340s user 0.010s sys 0.003s
I20260925 09:57:38.127148 12491 raft_consensus.cc:504] T 9744b9814b69470396ce039348fe5e45 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:38.131698 12519 raft_consensus.cc:482] T f1d31dca90df414a83be7c9e404b8d30 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:38.132134 12519 raft_consensus.cc:504] T f1d31dca90df414a83be7c9e404b8d30 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:38.134068 12519 leader_election.cc:288] T f1d31dca90df414a83be7c9e404b8d30 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:38.135022 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1d31dca90df414a83be7c9e404b8d30" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:38.139123 10830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1d31dca90df414a83be7c9e404b8d30" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:38.140421 10918 leader_election.cc:341] T f1d31dca90df414a83be7c9e404b8d30 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:38.145239 10919 leader_election.cc:341] T f1d31dca90df414a83be7c9e404b8d30 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:38.145593 10919 leader_election.cc:302] T f1d31dca90df414a83be7c9e404b8d30 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:38.146193 12519 raft_consensus.cc:2726] T f1d31dca90df414a83be7c9e404b8d30 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:38.147871 11082 raft_consensus.cc:348] T 9a73a1f242e7430cbf52229c07b4dc82 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:38.154302 11029 heartbeater.cc:347] Connected to a master server at 127.10.9.60:36985
I20260925 09:57:38.159497 12519 raft_consensus.cc:482] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:38.160012 12519 raft_consensus.cc:504] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:38.163721 10830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17bd30416fe14c0e9ab2e0c0bf5f2fb5" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:38.165464 10918 leader_election.cc:341] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:38.166266 10830 raft_consensus.cc:1261] T cfa619b699004882a82976f0ef8204ad P c43461c4ee854ae1ac52b873ccada318 [term 2 FOLLOWER]: Refusing update from remote peer be97ab5d75934a648d4ab76341193ec0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260925 09:57:38.167555 12464 consensus_queue.cc:1035] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260925 09:57:38.171072 12519 leader_election.cc:288] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:38.199321 10830 raft_consensus.cc:3037] T 37bab60afe7a4c758a08ef7c9c24aba0 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:38.201148 11085 tablet_bootstrap.cc:492] T b61444c31dbf4882ac748f861ea66818 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:38.201540 11085 ts_tablet_manager.cc:1397] T b61444c31dbf4882ac748f861ea66818 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.312s user 0.005s sys 0.004s
I20260925 09:57:38.203764 11085 raft_consensus.cc:348] T b61444c31dbf4882ac748f861ea66818 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:38.214778 11727 ts_tablet_manager.cc:1939] T 1e76f9e0ba554e1cad0cf18863393608 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:38.214730 11084 tablet_bootstrap.cc:654] T f0bf545f70ae4cd6a1ad2540c7158ecc P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:38.206918 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9744b9814b69470396ce039348fe5e45" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:38.219316 11085 raft_consensus.cc:374] T b61444c31dbf4882ac748f861ea66818 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260925 09:57:38.211633 10775 leader_election.cc:341] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Not found: Tablet not found: a21909e2e5794f57971f4e42795235c7
I20260925 09:57:38.219583 11085 raft_consensus.cc:729] T b61444c31dbf4882ac748f861ea66818 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:38.220194 11085 consensus_queue.cc:260] T b61444c31dbf4882ac748f861ea66818 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:38.220559 10775 leader_election.cc:341] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Not found: Tablet not found: a21909e2e5794f57971f4e42795235c7
I20260925 09:57:38.222471 11085 ts_tablet_manager.cc:1428] T b61444c31dbf4882ac748f861ea66818 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.021s user 0.006s sys 0.000s
I20260925 09:57:38.223268 11085 tablet_bootstrap.cc:492] T ab6bdba5ca5d41ae921a90c1f77006c5 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:38.224555 12519 raft_consensus.cc:482] T 93e1b5837aff48d7907890c40dfcd0d0 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b8850818e2ee42a082802d0095c08d21)
I20260925 09:57:38.225008 12519 raft_consensus.cc:504] T 93e1b5837aff48d7907890c40dfcd0d0 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:38.227596 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93e1b5837aff48d7907890c40dfcd0d0" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:38.227903 12519 leader_election.cc:288] T 93e1b5837aff48d7907890c40dfcd0d0 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
W20260925 09:57:38.232633 10919 leader_election.cc:341] T 93e1b5837aff48d7907890c40dfcd0d0 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Not found: Tablet not found: 93e1b5837aff48d7907890c40dfcd0d0
I20260925 09:57:38.233037 10919 leader_election.cc:302] T 93e1b5837aff48d7907890c40dfcd0d0 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:38.233688 12519 raft_consensus.cc:2726] T 93e1b5837aff48d7907890c40dfcd0d0 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 09:57:38.154064 12492 raft_consensus.cc:482] T 1e76f9e0ba554e1cad0cf18863393608 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:38.234675 12492 raft_consensus.cc:504] T 1e76f9e0ba554e1cad0cf18863393608 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:38.172115 10679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92814437d53143a79f99199f8ee3a742" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:38.235679 10679 raft_consensus.cc:3037] T 92814437d53143a79f99199f8ee3a742 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:38.239938 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 348) took 5633 ms. Trace:
I20260925 09:57:38.240312 11727 rpcz_store.cc:276] 0925 09:57:32.606843 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:32.606945 (+ 102us) service_pool.cc:227] Handling call
0925 09:57:37.842344 (+5235399us) ts_tablet_manager.cc:1145] Deleting tablet 1e76f9e0ba554e1cad0cf18863393608
0925 09:57:37.842399 (+ 55us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:38.061631 (+219232us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:38.110402 (+ 48771us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:38.239866 (+129464us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5234769}
I20260925 09:57:38.240954 12492 leader_election.cc:288] T 1e76f9e0ba554e1cad0cf18863393608 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:38.174021 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17bd30416fe14c0e9ab2e0c0bf5f2fb5" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:38.241546 11727 tablet_replica.cc:330] T 350904aa5957496d90a2aa2e637b591c P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:38.242313 11727 raft_consensus.cc:2229] T 350904aa5957496d90a2aa2e637b591c P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:38.242753 11727 raft_consensus.cc:2258] T 350904aa5957496d90a2aa2e637b591c P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 09:57:38.243744 10919 leader_election.cc:341] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:38.244179 10919 leader_election.cc:302] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:38.244540 11727 ts_tablet_manager.cc:1905] T 350904aa5957496d90a2aa2e637b591c P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:38.244966 12519 raft_consensus.cc:2726] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:38.245749 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 1e76f9e0ba554e1cad0cf18863393608 (table default.Table_a2a96456c8804d1692ffbafab832340b [id=5dc3fd1666fb4431b1c3bbf27b02ba54]) successfully deleted
I20260925 09:57:38.247311 12519 raft_consensus.cc:482] T a58cf1dc81454710a9ad1a8baf155c9f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:38.247704 12519 raft_consensus.cc:504] T a58cf1dc81454710a9ad1a8baf155c9f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:38.249511 12519 leader_election.cc:288] T a58cf1dc81454710a9ad1a8baf155c9f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:38.251427 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a58cf1dc81454710a9ad1a8baf155c9f" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:38.251602 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a58cf1dc81454710a9ad1a8baf155c9f" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:38.252835 10918 leader_election.cc:341] T a58cf1dc81454710a9ad1a8baf155c9f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:38.254362 10679 raft_consensus.cc:2454] T 92814437d53143a79f99199f8ee3a742 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
W20260925 09:57:38.255450 10919 leader_election.cc:341] T a58cf1dc81454710a9ad1a8baf155c9f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:38.255849 10919 leader_election.cc:302] T a58cf1dc81454710a9ad1a8baf155c9f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:38.258041 12519 raft_consensus.cc:2726] T a58cf1dc81454710a9ad1a8baf155c9f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:38.259759 10919 consensus_peers.cc:487] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0 -> Peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Couldn't send request to peer b8850818e2ee42a082802d0095c08d21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cfa619b699004882a82976f0ef8204ad. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:38.184302 12379 ts_tablet_manager.cc:1918] T 417058493c6547c69e72347cef373292 P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:38.260385 12379 log.cc:1196] T 417058493c6547c69e72347cef373292 P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/417058493c6547c69e72347cef373292
I20260925 09:57:38.261811 11082 raft_consensus.cc:374] T 9a73a1f242e7430cbf52229c07b4dc82 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:38.269239 11082 raft_consensus.cc:729] T 9a73a1f242e7430cbf52229c07b4dc82 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:38.274776 11085 tablet_bootstrap.cc:654] T ab6bdba5ca5d41ae921a90c1f77006c5 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:38.276853 10806 tablet_service.cc:1467] Processing CreateTablet for tablet 8c42f36f2e7f4a62a97f57e960e9bafa (DEFAULT_TABLE table=default.Table_0ac0877feb3445159347c33d900cb3ac [id=4baf6a5034dc4659bf64357db5f52bd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:38.277871 10806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c42f36f2e7f4a62a97f57e960e9bafa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:38.280961 10956 tablet_service.cc:1467] Processing CreateTablet for tablet 8c42f36f2e7f4a62a97f57e960e9bafa (DEFAULT_TABLE table=default.Table_0ac0877feb3445159347c33d900cb3ac [id=4baf6a5034dc4659bf64357db5f52bd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:38.281921 10956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c42f36f2e7f4a62a97f57e960e9bafa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:38.282968 10388 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 2f07dc9548364870923165f4ea524310 (table default.Table_0db927e92a3b4970a7efea3fb765e65e [id=a988264fe43c4eb88a84f8b6359e77a6]). Expected version 1 got 0
I20260925 09:57:38.288007 10946 tablet_service.cc:1467] Processing CreateTablet for tablet 5f74496920854b5c8b00cb33bdb3e392 (DEFAULT_TABLE table=default.Table_91c768bb83ce44d88f0320b446d6d74a [id=e4fa48723024467db9c6ebffe5c96a5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:38.289029 10946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f74496920854b5c8b00cb33bdb3e392. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:38.292308 10945 tablet_service.cc:1467] Processing CreateTablet for tablet de2fd903aefd418aa6a865dd9ef17350 (DEFAULT_TABLE table=default.Table_35aa99e4c3c9430ab411483bad074740 [id=e5172b2725714086935e93ae5df0cc0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:38.292518 10809 tablet_service.cc:1467] Processing CreateTablet for tablet 5f74496920854b5c8b00cb33bdb3e392 (DEFAULT_TABLE table=default.Table_91c768bb83ce44d88f0320b446d6d74a [id=e4fa48723024467db9c6ebffe5c96a5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:38.293309 10945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de2fd903aefd418aa6a865dd9ef17350. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:38.293560 10809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f74496920854b5c8b00cb33bdb3e392. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:38.293331 11082 consensus_queue.cc:260] T 9a73a1f242e7430cbf52229c07b4dc82 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:38.294869 10388 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 6540aba5854a44419addc5d454e473b7 (table default.Table_8180842f664042b0a9073b7008438ff7 [id=416a4b989570457e919fb7e7b16a017d]). Expected version 1 got 0
I20260925 09:57:38.294881 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e76f9e0ba554e1cad0cf18863393608" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:38.295028 10813 tablet_service.cc:1467] Processing CreateTablet for tablet de2fd903aefd418aa6a865dd9ef17350 (DEFAULT_TABLE table=default.Table_35aa99e4c3c9430ab411483bad074740 [id=e5172b2725714086935e93ae5df0cc0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:38.295931 10813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de2fd903aefd418aa6a865dd9ef17350. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:38.295966 10817 tablet_service.cc:1467] Processing CreateTablet for tablet f535784bf0a74769a6d97115fb3b0c8a (DEFAULT_TABLE table=default.Table_94b396db46ba43d88015187a199247fc [id=78304acc330348238cdb35b6034337fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:38.296954 10817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f535784bf0a74769a6d97115fb3b0c8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:38.300115 10952 tablet_service.cc:1467] Processing CreateTablet for tablet f535784bf0a74769a6d97115fb3b0c8a (DEFAULT_TABLE table=default.Table_94b396db46ba43d88015187a199247fc [id=78304acc330348238cdb35b6034337fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:38.301100 10952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f535784bf0a74769a6d97115fb3b0c8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:38.312569 10775 leader_election.cc:302] T 92814437d53143a79f99199f8ee3a742 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters: be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:38.314611 12506 raft_consensus.cc:2781] T 92814437d53143a79f99199f8ee3a742 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:38.315057 12506 raft_consensus.cc:686] T 92814437d53143a79f99199f8ee3a742 P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Becoming Leader. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Running, Role: LEADER
I20260925 09:57:38.315973 12506 consensus_queue.cc:237] T 92814437d53143a79f99199f8ee3a742 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:38.321256 10810 tablet_service.cc:1467] Processing CreateTablet for tablet 7a92074fdf254af2b52b39e41ff51689 (DEFAULT_TABLE table=default.Table_8f56e9a1b3504017bfabda8fd9b359ba [id=6b9c75f4db1c4d15b23284a5a6987cfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:38.322455 10810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a92074fdf254af2b52b39e41ff51689. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:38.325736 10812 tablet_service.cc:1467] Processing CreateTablet for tablet 78c84fd4338d44ec937dd9445d523001 (DEFAULT_TABLE table=default.Table_22ca44b09d854c16beb77aad966419d0 [id=667704be08684fdc847cf5345cef4d89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:38.326682 10812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78c84fd4338d44ec937dd9445d523001. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:38.327171 10944 tablet_service.cc:1467] Processing CreateTablet for tablet 7a92074fdf254af2b52b39e41ff51689 (DEFAULT_TABLE table=default.Table_8f56e9a1b3504017bfabda8fd9b359ba [id=6b9c75f4db1c4d15b23284a5a6987cfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:38.328289 10944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a92074fdf254af2b52b39e41ff51689. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:38.338476 12027 ts_tablet_manager.cc:1918] T 417058493c6547c69e72347cef373292 P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:38.338804 12027 log.cc:1196] T 417058493c6547c69e72347cef373292 P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/417058493c6547c69e72347cef373292
I20260925 09:57:38.346098 10960 tablet_service.cc:1467] Processing CreateTablet for tablet 78c84fd4338d44ec937dd9445d523001 (DEFAULT_TABLE table=default.Table_22ca44b09d854c16beb77aad966419d0 [id=667704be08684fdc847cf5345cef4d89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:38.346742 12464 raft_consensus.cc:482] T 2d79538d6474433fbda26c91645a0619 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:38.347347 10960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78c84fd4338d44ec937dd9445d523001. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:38.347157 12464 raft_consensus.cc:504] T 2d79538d6474433fbda26c91645a0619 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:38.349083 12464 leader_election.cc:288] T 2d79538d6474433fbda26c91645a0619 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:38.354233 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d79538d6474433fbda26c91645a0619" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:38.354611 11082 ts_tablet_manager.cc:1428] T 9a73a1f242e7430cbf52229c07b4dc82 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.231s user 0.004s sys 0.000s
I20260925 09:57:38.185796 12491 leader_election.cc:288] T 9744b9814b69470396ce039348fe5e45 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:38.269948 12379 ts_tablet_manager.cc:1939] T 417058493c6547c69e72347cef373292 P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:38.280503 10653 tablet_service.cc:1467] Processing CreateTablet for tablet 8c42f36f2e7f4a62a97f57e960e9bafa (DEFAULT_TABLE table=default.Table_0ac0877feb3445159347c33d900cb3ac [id=4baf6a5034dc4659bf64357db5f52bd7]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 09:57:38.355509 10918 leader_election.cc:341] T 2d79538d6474433fbda26c91645a0619 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:38.262141 10619 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 49745.4us
I20260925 09:57:38.300060 10668 tablet_service.cc:1467] Processing CreateTablet for tablet 5f74496920854b5c8b00cb33bdb3e392 (DEFAULT_TABLE table=default.Table_91c768bb83ce44d88f0320b446d6d74a [id=e4fa48723024467db9c6ebffe5c96a5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:38.301070 10658 tablet_service.cc:1467] Processing CreateTablet for tablet de2fd903aefd418aa6a865dd9ef17350 (DEFAULT_TABLE table=default.Table_35aa99e4c3c9430ab411483bad074740 [id=e5172b2725714086935e93ae5df0cc0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:38.302058 10664 tablet_service.cc:1467] Processing CreateTablet for tablet f535784bf0a74769a6d97115fb3b0c8a (DEFAULT_TABLE table=default.Table_94b396db46ba43d88015187a199247fc [id=78304acc330348238cdb35b6034337fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:38.304289 10665 tablet_service.cc:1467] Processing CreateTablet for tablet 7a92074fdf254af2b52b39e41ff51689 (DEFAULT_TABLE table=default.Table_8f56e9a1b3504017bfabda8fd9b359ba [id=6b9c75f4db1c4d15b23284a5a6987cfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:38.364971 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9744b9814b69470396ce039348fe5e45" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:38.365708 10976 raft_consensus.cc:3037] T 9744b9814b69470396ce039348fe5e45 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:38.368633 10679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d79538d6474433fbda26c91645a0619" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:38.370116 10919 leader_election.cc:341] T 2d79538d6474433fbda26c91645a0619 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:38.370563 10919 leader_election.cc:302] T 2d79538d6474433fbda26c91645a0619 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:38.372900 10664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f535784bf0a74769a6d97115fb3b0c8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:38.376984 11082 tablet_bootstrap.cc:492] T a81530200cf14717bc7cbf67bb73f4e5 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:38.377907 10653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c42f36f2e7f4a62a97f57e960e9bafa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:38.380147 12519 raft_consensus.cc:2726] T 2d79538d6474433fbda26c91645a0619 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:38.383445 10665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a92074fdf254af2b52b39e41ff51689. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:38.383930 10668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f74496920854b5c8b00cb33bdb3e392. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:38.387648 10658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de2fd903aefd418aa6a865dd9ef17350. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:38.407711 12379 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 379) took 2672 ms. Trace:
I20260925 09:57:38.408193 12379 rpcz_store.cc:276] 0925 09:57:35.735634 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:35.735759 (+ 125us) service_pool.cc:227] Handling call
0925 09:57:37.671511 (+1935752us) ts_tablet_manager.cc:1145] Deleting tablet 417058493c6547c69e72347cef373292
0925 09:57:37.672746 (+ 1235us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:38.004792 (+332046us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:38.184056 (+179264us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:38.407643 (+223587us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1935076}
I20260925 09:57:38.409665 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 417058493c6547c69e72347cef373292 (table default.Table_d1b168dcbc624b88980354edd6eff43a [id=8165aba086104ad4ab318b0128fdf3a2]) successfully deleted
I20260925 09:57:38.410696 12379 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 380) took 2652 ms. Trace:
I20260925 09:57:38.411100 12379 rpcz_store.cc:276] 0925 09:57:35.757936 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:35.758050 (+ 114us) service_pool.cc:227] Handling call
0925 09:57:38.410426 (+2652376us) ts_tablet_manager.cc:1145] Deleting tablet 3ff4cf556eee494d8cd590884466f09f
0925 09:57:38.410470 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:38.410640 (+ 170us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2651884}
W20260925 09:57:38.416039 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 3ff4cf556eee494d8cd590884466f09f because the tablet was not found. No further retry: Not found: Tablet not found: 3ff4cf556eee494d8cd590884466f09f
I20260925 09:57:38.418915 11727 ts_tablet_manager.cc:1918] T 350904aa5957496d90a2aa2e637b591c P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:38.419270 11727 log.cc:1196] T 350904aa5957496d90a2aa2e637b591c P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/350904aa5957496d90a2aa2e637b591c
I20260925 09:57:38.425936 11727 ts_tablet_manager.cc:1939] T 350904aa5957496d90a2aa2e637b591c P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:38.427932 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 349) took 5595 ms. Trace:
I20260925 09:57:38.428220 11727 rpcz_store.cc:276] 0925 09:57:32.832706 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:32.832824 (+ 118us) service_pool.cc:227] Handling call
0925 09:57:38.241372 (+5408548us) ts_tablet_manager.cc:1145] Deleting tablet 350904aa5957496d90a2aa2e637b591c
0925 09:57:38.241432 (+ 60us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:38.268625 (+ 27193us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:38.418677 (+150052us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:38.427879 (+ 9202us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5407995}
I20260925 09:57:38.429425 11727 tablet_replica.cc:330] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
W20260925 09:57:38.318840 10621 leader_election.cc:341] T 9744b9814b69470396ce039348fe5e45 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:38.334105 10654 tablet_service.cc:1467] Processing CreateTablet for tablet 78c84fd4338d44ec937dd9445d523001 (DEFAULT_TABLE table=default.Table_22ca44b09d854c16beb77aad966419d0 [id=667704be08684fdc847cf5345cef4d89]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 09:57:38.430442 10621 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 111626us
I20260925 09:57:38.430856 11082 tablet_bootstrap.cc:654] T a81530200cf14717bc7cbf67bb73f4e5 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:38.437713 10391 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 6540aba5854a44419addc5d454e473b7 (table default.Table_8180842f664042b0a9073b7008438ff7 [id=416a4b989570457e919fb7e7b16a017d]). Expected version 1 got 0
I20260925 09:57:38.438411 10654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78c84fd4338d44ec937dd9445d523001. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:38.439239 10919 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 69143.1us
I20260925 09:57:38.440083 10391 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet f62c62715c9b42de826c1a66fd431eab (table default.Table_bcbc82519cf043738f61029181f29418 [id=b3f7f21ab2214e4cb637bea27ce62abe]). Expected version 1 got 0
I20260925 09:57:38.442384 10391 catalog_manager.cc:5495] T ab6bdba5ca5d41ae921a90c1f77006c5 P be97ab5d75934a648d4ab76341193ec0 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } }
I20260925 09:57:38.444564 10391 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 2f07dc9548364870923165f4ea524310 (table default.Table_0db927e92a3b4970a7efea3fb765e65e [id=a988264fe43c4eb88a84f8b6359e77a6]). Expected version 1 got 0
I20260925 09:57:38.445298 10391 catalog_manager.cc:5495] T defaef4d6b754ca5a2dfc21b7a596a5b P be97ab5d75934a648d4ab76341193ec0 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } } }
W20260925 09:57:38.454983 10619 leader_election.cc:341] T 1e76f9e0ba554e1cad0cf18863393608 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 1e76f9e0ba554e1cad0cf18863393608
W20260925 09:57:38.457386 10621 leader_election.cc:341] T 1e76f9e0ba554e1cad0cf18863393608 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:38.457731 10621 leader_election.cc:302] T 1e76f9e0ba554e1cad0cf18863393608 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8850818e2ee42a082802d0095c08d21; no voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:38.458366 12492 raft_consensus.cc:2726] T 1e76f9e0ba554e1cad0cf18863393608 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:38.460044 12379 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 382) took 2419 ms. Trace:
I20260925 09:57:38.460443 12379 rpcz_store.cc:276] 0925 09:57:36.040837 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:36.040960 (+ 123us) service_pool.cc:227] Handling call
0925 09:57:38.459658 (+2418698us) ts_tablet_manager.cc:1145] Deleting tablet 4c3e66f6ad23475f9fb831afc1d579c4
0925 09:57:38.459716 (+ 58us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:38.459982 (+ 266us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2418070}
I20260925 09:57:38.461532 12027 ts_tablet_manager.cc:1939] T 417058493c6547c69e72347cef373292 P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:38.462455 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 343) took 5506 ms. Trace:
I20260925 09:57:38.462752 12027 rpcz_store.cc:276] 0925 09:57:32.956290 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:32.959726 (+ 3436us) service_pool.cc:227] Handling call
0925 09:57:37.793947 (+4834221us) ts_tablet_manager.cc:1145] Deleting tablet 417058493c6547c69e72347cef373292
0925 09:57:37.793985 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:38.016039 (+222054us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:38.338281 (+322242us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:38.462401 (+124120us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":66887,"tablet-delete.queue_time_us":4833481}
I20260925 09:57:38.462998 10386 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: default.Table_be0f658aa41f4188bc16db37e96f4ad1 [id=73083c1560cb4db5a12964387cb49b73] alter complete (version 1)
I20260925 09:57:38.475674 10976 raft_consensus.cc:2454] T 9744b9814b69470396ce039348fe5e45 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:38.484854 12379 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 386) took 1919 ms. Trace:
I20260925 09:57:38.485179 12379 rpcz_store.cc:276] 0925 09:57:36.565897 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:36.566011 (+ 114us) service_pool.cc:227] Handling call
0925 09:57:38.484505 (+1918494us) ts_tablet_manager.cc:1145] Deleting tablet d822e8ac13d9479aa9ce4f74ace55dd5
0925 09:57:38.484550 (+ 45us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:38.484799 (+ 249us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1918000}
I20260925 09:57:38.486259 12379 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 388) took 1917 ms. Trace:
I20260925 09:57:38.486588 12379 rpcz_store.cc:276] 0925 09:57:36.569138 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:36.569268 (+ 130us) service_pool.cc:227] Handling call
0925 09:57:38.485949 (+1916681us) ts_tablet_manager.cc:1145] Deleting tablet 2b1b2d66397b44f38056077d79941ac4
0925 09:57:38.485986 (+ 37us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:38.486199 (+ 213us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1916160}
W20260925 09:57:38.488477 10619 consensus_peers.cc:487] T 350904aa5957496d90a2aa2e637b591c P b8850818e2ee42a082802d0095c08d21 -> Peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Couldn't send request to peer be97ab5d75934a648d4ab76341193ec0. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:38.493456 10944 tablet_service.cc:1514] Processing DeleteTablet for tablet 69028aa7908c45b896da4ba8deef07b1 with delete_type TABLET_DATA_DELETED (replaced by tablet 2015c10f503e4751ad9ff77404fca50b) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:38.494060 10960 tablet_service.cc:1514] Processing DeleteTablet for tablet fb9c51f7e7fd4c029be2a710cb9462eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:31 UTC) from {username='slave'} at 127.0.0.1:53560
W20260925 09:57:38.495271 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 4c3e66f6ad23475f9fb831afc1d579c4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4c3e66f6ad23475f9fb831afc1d579c4 already in progress: creating tablet
W20260925 09:57:38.496095 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet d822e8ac13d9479aa9ce4f74ace55dd5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d822e8ac13d9479aa9ce4f74ace55dd5 already in progress: creating tablet
I20260925 09:57:38.496611 10956 tablet_service.cc:1514] Processing DeleteTablet for tablet c7cd1d6a603642afa2c8c737079961b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:29 UTC) from {username='slave'} at 127.0.0.1:53560
W20260925 09:57:38.496788 10380 catalog_manager.cc:4821] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 2b1b2d66397b44f38056077d79941ac4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2b1b2d66397b44f38056077d79941ac4 already in progress: creating tablet
I20260925 09:57:38.501281 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 350904aa5957496d90a2aa2e637b591c (table default.Table_836d63b3fbed4250b0757ef2d432a704 [id=f58c7edcfeb44c90968b95873346f919]) successfully deleted
I20260925 09:57:38.504166 10390 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 5608c5f9b65b4da7b2fbb56417c92d88
I20260925 09:57:38.505501 10390 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:35140 (request call id 373) took 2941 ms. Trace:
I20260925 09:57:38.511404 11727 raft_consensus.cc:2229] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0 [term 2 LEADER]: Raft consensus shutting down.
I20260925 09:57:38.512315 11727 raft_consensus.cc:2258] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:38.512074 10619 leader_election.cc:302] T 9744b9814b69470396ce039348fe5e45 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:38.512899 12390 raft_consensus.cc:2781] T 9744b9814b69470396ce039348fe5e45 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:38.513566 12390 raft_consensus.cc:686] T 9744b9814b69470396ce039348fe5e45 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:38.514292 11727 ts_tablet_manager.cc:1905] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:38.514418 12390 consensus_queue.cc:237] T 9744b9814b69470396ce039348fe5e45 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:38.521884 11085 tablet_bootstrap.cc:492] T ab6bdba5ca5d41ae921a90c1f77006c5 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:38.522320 11085 ts_tablet_manager.cc:1397] T ab6bdba5ca5d41ae921a90c1f77006c5 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.299s user 0.008s sys 0.000s
I20260925 09:57:38.505785 10390 rpcz_store.cc:276] 0925 09:57:35.563941 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:36.398866 (+834925us) service_pool.cc:227] Handling call
0925 09:57:36.404424 (+ 5558us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:57:36.405209 (+ 785us) catalog_manager.cc:2752] Modifying in-memory table state
0925 09:57:36.405356 (+ 147us) catalog_manager.cc:2761] Locking tablets
0925 09:57:36.406042 (+ 686us) catalog_manager.cc:2775] Removing table and tablets from system table
0925 09:57:36.437408 (+ 31366us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:36.437435 (+ 27us) write_op.cc:454] Released schema lock
0925 09:57:36.437658 (+ 223us) catalog_manager.cc:2793] Removing table from by-name map
0925 09:57:36.437866 (+ 208us) catalog_manager.cc:2809] Committing in-memory state
0925 09:57:38.504074 (+2066208us) catalog_manager.cc:2813] Aborting table tasks
0925 09:57:38.505411 (+ 1337us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:36.419637 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:36.420157 (+ 520us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:36.420280 (+ 123us) write_op.cc:435] Acquired schema lock
0925 09:57:36.420306 (+ 26us) tablet.cc:660] Decoding operations
0925 09:57:36.420705 (+ 399us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:36.420751 (+ 46us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:36.420777 (+ 26us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:36.421105 (+ 328us) tablet.cc:699] Row locks acquired
0925 09:57:36.421124 (+ 19us) write_op.cc:260] PREPARE: finished
0925 09:57:36.421262 (+ 138us) write_op.cc:270] Start()
0925 09:57:36.421364 (+ 102us) write_op.cc:276] Timestamp: P: 1790330256421197 usec, L: 0
0925 09:57:36.421381 (+ 17us) op_driver.cc:331] REPLICATION: starting
0925 09:57:36.422105 (+ 724us) log.cc:842] Serialized 787 byte log entry
0925 09:57:36.430495 (+ 8390us) op_driver.cc:444] REPLICATION: finished
0925 09:57:36.430895 (+ 400us) write_op.cc:301] APPLY: starting
0925 09:57:36.430975 (+ 80us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:36.431072 (+ 97us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:36.431186 (+ 114us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:36.431625 (+ 439us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:36.431807 (+ 182us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 09:57:36.431871 (+ 64us) write_op.cc:312] APPLY: finished
0925 09:57:36.432067 (+ 196us) log.cc:842] Serialized 29 byte log entry
0925 09:57:36.432260 (+ 193us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:36.432326 (+ 66us) write_op.cc:454] Released schema lock
0925 09:57:36.432354 (+ 28us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":198,"apply.run_cpu_time_us":1697,"apply.run_wall_time_us":1697,"num_ops":2,"prepare.queue_time_us":4736,"prepare.run_cpu_time_us":2840,"prepare.run_wall_time_us":2840,"raft.queue_time_us":185,"raft.run_cpu_time_us":689,"raft.run_wall_time_us":689,"replication_time_us":9034}]]}
W20260925 09:57:38.533363 10390 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.1:56563 (request call id 44) took 1236 ms (client timeout 1000 ms). Trace:
W20260925 09:57:38.533990 10390 rpcz_store.cc:269] 0925 09:57:37.297433 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.533199 (+1235766us) service_pool.cc:214] Skipping call since client already timed out
0925 09:57:38.533307 (+ 108us) inbound_call.cc:173] Queueing failure response
Metrics: {}
I20260925 09:57:38.546569 10950 tablet_service.cc:1514] Processing DeleteTablet for tablet dba14a44a9de492886d629441606767e with delete_type TABLET_DATA_DELETED (replaced by tablet 080c76ce032848c4af26ec5c8566f300) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:38.547215 10948 tablet_service.cc:1514] Processing DeleteTablet for tablet 3039ec3febf848dba7c4e6fd38bb8053 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:26 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:38.547875 10944 tablet_service.cc:1514] Processing DeleteTablet for tablet a81530200cf14717bc7cbf67bb73f4e5 with delete_type TABLET_DATA_DELETED (replaced by tablet e9faf70be8ac43c79305980d685a28eb) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:38.548511 10947 tablet_service.cc:1514] Processing DeleteTablet for tablet 197f61525b0c4f9da441d98701d231a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:30 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:38.549072 10960 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d087d3e35814c90a03eccd90eeea86c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:30 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:38.549664 10948 tablet_service.cc:1514] Processing DeleteTablet for tablet 350904aa5957496d90a2aa2e637b591c with delete_type TABLET_DATA_DELETED (replaced by tablet 154083d32e0f474f9e3a02e6cca940ef) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:38.553509 11085 raft_consensus.cc:348] T ab6bdba5ca5d41ae921a90c1f77006c5 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:38.576159 11085 raft_consensus.cc:374] T ab6bdba5ca5d41ae921a90c1f77006c5 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:38.576774 10948 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e76f9e0ba554e1cad0cf18863393608 with delete_type TABLET_DATA_DELETED (replaced by tablet 963e5bf2ac2848b4967074d28acdae52) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:38.579499 10956 tablet_service.cc:1514] Processing DeleteTablet for tablet 5608c5f9b65b4da7b2fbb56417c92d88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:36 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:38.586829 11727 ts_tablet_manager.cc:1918] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260925 09:57:38.587222 11727 log.cc:1196] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/cfa619b699004882a82976f0ef8204ad
I20260925 09:57:38.587921 11727 ts_tablet_manager.cc:1939] T cfa619b699004882a82976f0ef8204ad P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:38.589810 10391 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: default.Table_dfb73f4afe7c464d8dc98356500054e2 [id=ab3c143dc00c43e78531514a114bfd16] alter complete (version 1)
I20260925 09:57:38.597630 12548 tablet.cc:1722] T 2f07dc9548364870923165f4ea524310 P be97ab5d75934a648d4ab76341193ec0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:38.605621 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 350) took 5105 ms. Trace:
I20260925 09:57:38.607173 11727 rpcz_store.cc:276] 0925 09:57:33.499877 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:33.500025 (+ 148us) service_pool.cc:227] Handling call
0925 09:57:38.429279 (+4929254us) ts_tablet_manager.cc:1145] Deleting tablet cfa619b699004882a82976f0ef8204ad
0925 09:57:38.429327 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:38.521574 (+ 92247us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:38.586569 (+ 64995us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:38.605566 (+ 18997us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":81130,"tablet-delete.queue_time_us":4928697}
I20260925 09:57:38.608275 11727 tablet_replica.cc:330] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:38.609014 11727 raft_consensus.cc:2229] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:38.609467 11727 raft_consensus.cc:2258] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:38.611109 11727 ts_tablet_manager.cc:1905] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 09:57:38.611796 10385 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.3:41469 (request call id 29) took 5293 ms (client timeout 1000 ms). Trace:
I20260925 09:57:38.612974 10653 tablet_service.cc:1514] Processing DeleteTablet for tablet 5608c5f9b65b4da7b2fbb56417c92d88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:36 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:38.614777 12379 tablet_replica.cc:330] T 5608c5f9b65b4da7b2fbb56417c92d88 P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:38.615545 12379 raft_consensus.cc:2229] T 5608c5f9b65b4da7b2fbb56417c92d88 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Raft consensus shutting down.
W20260925 09:57:38.597874 10774 outbound_call.cc:322] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.10.9.62:34455, user_credentials={real_user=slave}} blocked reactor thread for 144625us
I20260925 09:57:38.616328 12379 raft_consensus.cc:2258] T 5608c5f9b65b4da7b2fbb56417c92d88 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:38.617414 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 417058493c6547c69e72347cef373292 (table default.Table_d1b168dcbc624b88980354edd6eff43a [id=8165aba086104ad4ab318b0128fdf3a2]) successfully deleted
I20260925 09:57:38.618019 12379 ts_tablet_manager.cc:1905] T 5608c5f9b65b4da7b2fbb56417c92d88 P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:38.619168 10813 tablet_service.cc:1514] Processing DeleteTablet for tablet 5608c5f9b65b4da7b2fbb56417c92d88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:36 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:38.609112 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 352) took 5649 ms. Trace:
I20260925 09:57:38.622591 12027 rpcz_store.cc:276] 0925 09:57:32.962759 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:32.962863 (+ 104us) service_pool.cc:227] Handling call
0925 09:57:38.490435 (+5527572us) ts_tablet_manager.cc:1145] Deleting tablet 1e76f9e0ba554e1cad0cf18863393608
0925 09:57:38.608766 (+118331us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:38.609052 (+ 286us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5512497}
W20260925 09:57:38.625144 10389 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.2:40761 (request call id 42) took 3411 ms (client timeout 1000 ms). Trace:
I20260925 09:57:38.632745 11085 raft_consensus.cc:729] T ab6bdba5ca5d41ae921a90c1f77006c5 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
W20260925 09:57:38.633416 10388 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.2:40761 (request call id 43) took 2132 ms (client timeout 1000 ms). Trace:
I20260925 09:57:38.633469 11085 consensus_queue.cc:260] T ab6bdba5ca5d41ae921a90c1f77006c5 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:38.633735 10388 rpcz_store.cc:269] 0925 09:57:36.501117 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:36.570012 (+ 68895us) service_pool.cc:227] Handling call
0925 09:57:36.714127 (+144115us) spinlock_profiling.cc:243] Waited 44.7 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2ad9c3e 00007fabf0610fb4 00007fabf0610f42 00007fabf0610e71 00007fabf0610dec 00007fabf06100a2 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:38.633353 (+1919226us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":59629,"reported_tablets":21,"spinlock_wait_cycles":102750592}
W20260925 09:57:38.625538 10389 rpcz_store.cc:269] 0925 09:57:35.213338 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:35.368855 (+155517us) service_pool.cc:227] Handling call
0925 09:57:38.625079 (+3256224us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":261745,"reported_tablets":19}
I20260925 09:57:38.635699 11085 ts_tablet_manager.cc:1428] T ab6bdba5ca5d41ae921a90c1f77006c5 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.113s user 0.006s sys 0.002s
I20260925 09:57:38.636831 10388 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_d3efd9bb6b174cdfbdbd1af72107b558" } new_table_name: "default.Table_82a8d1df379d442e9333fd4ab294738c" modify_external_catalogs: true
I20260925 09:57:38.637383 10664 tablet_service.cc:1514] Processing DeleteTablet for tablet 6161774f61ae48ab9764a05f2188a80b with delete_type TABLET_DATA_DELETED (replaced by tablet 10657a81058241fc988d37e5ea7ff8cf) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:38.638036 10665 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a31b999f0fd447eacff13de71aa3fff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:23 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:38.638558 10667 tablet_service.cc:1514] Processing DeleteTablet for tablet fb9c51f7e7fd4c029be2a710cb9462eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:31 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:38.639067 10655 tablet_service.cc:1514] Processing DeleteTablet for tablet 417058493c6547c69e72347cef373292 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:25 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:38.643829 10653 tablet_service.cc:1514] Processing DeleteTablet for tablet 93e1b5837aff48d7907890c40dfcd0d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:32 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:38.646111 10655 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e3120347c8e4a3eab57d6562ae40e8d with delete_type TABLET_DATA_DELETED (replaced by tablet c52745a8cdee461aba7485b0531245d6) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:38.646781 10655 tablet_service.cc:1514] Processing DeleteTablet for tablet 37bab60afe7a4c758a08ef7c9c24aba0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:26 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:38.647475 10655 tablet_service.cc:1514] Processing DeleteTablet for tablet dba14a44a9de492886d629441606767e with delete_type TABLET_DATA_DELETED (replaced by tablet 080c76ce032848c4af26ec5c8566f300) from {username='slave'} at 127.0.0.1:37876
W20260925 09:57:38.612386 10385 rpcz_store.cc:269] 0925 09:57:33.318442 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:33.325624 (+ 7182us) service_pool.cc:227] Handling call
0925 09:57:38.611694 (+5286070us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":66}
I20260925 09:57:38.648164 10653 tablet_service.cc:1514] Processing DeleteTablet for tablet 545ac025f2cb469da847979011a0b3ce with delete_type TABLET_DATA_DELETED (replaced by tablet ab6bdba5ca5d41ae921a90c1f77006c5) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:38.651162 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet cfa619b699004882a82976f0ef8204ad (table default.Table_2a6203820e0d44aa993080049923de10 [id=c90c671ed4f043c2b56a1f84ff9a42cd]) successfully deleted
W20260925 09:57:38.665112 10387 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.1:56563 (request call id 43) took 2771 ms (client timeout 1000 ms). Trace:
I20260925 09:57:38.668948 10655 tablet_service.cc:1514] Processing DeleteTablet for tablet 10657a81058241fc988d37e5ea7ff8cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:25 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:38.669641 10655 tablet_service.cc:1514] Processing DeleteTablet for tablet cf775c9d9b114ddcb97dc6c6eca8e0fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:26 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:38.671409 10655 tablet_service.cc:1514] Processing DeleteTablet for tablet 92814437d53143a79f99199f8ee3a742 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:29 UTC) from {username='slave'} at 127.0.0.1:37876
W20260925 09:57:38.672107 10621 consensus_peers.cc:487] T 350904aa5957496d90a2aa2e637b591c P b8850818e2ee42a082802d0095c08d21 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:38.669658 10653 tablet_service.cc:1514] Processing DeleteTablet for tablet a21909e2e5794f57971f4e42795235c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:32 UTC) from {username='slave'} at 127.0.0.1:37876
W20260925 09:57:38.672672 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 1e76f9e0ba554e1cad0cf18863393608 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1e76f9e0ba554e1cad0cf18863393608 already in progress: creating tablet
I20260925 09:57:38.686365 12551 tablet.cc:1722] T 2f07dc9548364870923165f4ea524310 P c43461c4ee854ae1ac52b873ccada318: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260925 09:57:38.687378 12507 lock_manager.cc:586] Waited 1 seconds to obtain row lock on key '<redacted>' (key hash 12926952885456789040) tablet 00000000000000000000000000000000 cur holder 0x7b5800044700
I20260925 09:57:38.695735 12379 ts_tablet_manager.cc:1918] T 5608c5f9b65b4da7b2fbb56417c92d88 P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:57:38.696089 12379 log.cc:1196] T 5608c5f9b65b4da7b2fbb56417c92d88 P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/5608c5f9b65b4da7b2fbb56417c92d88
I20260925 09:57:38.697962 10809 tablet_service.cc:1514] Processing DeleteTablet for tablet 417058493c6547c69e72347cef373292 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:25 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:38.698637 12547 tablet.cc:1722] T 2f07dc9548364870923165f4ea524310 P b8850818e2ee42a082802d0095c08d21: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:38.699651 10813 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e3120347c8e4a3eab57d6562ae40e8d with delete_type TABLET_DATA_DELETED (replaced by tablet c52745a8cdee461aba7485b0531245d6) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:38.701675 10813 tablet_service.cc:1514] Processing DeleteTablet for tablet 6161774f61ae48ab9764a05f2188a80b with delete_type TABLET_DATA_DELETED (replaced by tablet 10657a81058241fc988d37e5ea7ff8cf) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:38.708652 11084 tablet_bootstrap.cc:492] T f0bf545f70ae4cd6a1ad2540c7158ecc P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:38.709095 11084 ts_tablet_manager.cc:1397] T f0bf545f70ae4cd6a1ad2540c7158ecc P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.689s user 0.007s sys 0.026s
I20260925 09:57:38.712896 12322 raft_consensus.cc:482] T 9a73a1f242e7430cbf52229c07b4dc82 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:38.713371 12322 raft_consensus.cc:504] T 9a73a1f242e7430cbf52229c07b4dc82 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:38.715627 12322 leader_election.cc:288] T 9a73a1f242e7430cbf52229c07b4dc82 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
W20260925 09:57:38.721251 10619 leader_election.cc:341] T 9a73a1f242e7430cbf52229c07b4dc82 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 9a73a1f242e7430cbf52229c07b4dc82
I20260925 09:57:38.723600 10813 tablet_service.cc:1514] Processing DeleteTablet for tablet 545ac025f2cb469da847979011a0b3ce with delete_type TABLET_DATA_DELETED (replaced by tablet ab6bdba5ca5d41ae921a90c1f77006c5) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:38.724690 12445 raft_consensus.cc:482] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b8850818e2ee42a082802d0095c08d21)
I20260925 09:57:38.725167 12445 raft_consensus.cc:504] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:38.726454 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 353) took 5747 ms. Trace:
I20260925 09:57:38.730157 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93e1b5837aff48d7907890c40dfcd0d0" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:38.731141 10976 raft_consensus.cc:2454] T 93e1b5837aff48d7907890c40dfcd0d0 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:38.726738 12027 rpcz_store.cc:276] 0925 09:57:32.978685 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:32.979459 (+ 774us) service_pool.cc:227] Handling call
0925 09:57:38.623364 (+5643905us) ts_tablet_manager.cc:1145] Deleting tablet 350904aa5957496d90a2aa2e637b591c
0925 09:57:38.726134 (+102770us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:38.726400 (+ 266us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5643434}
I20260925 09:57:38.733652 10806 tablet_service.cc:1514] Processing DeleteTablet for tablet fb9c51f7e7fd4c029be2a710cb9462eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:31 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:38.734131 12027 tablet_replica.cc:330] T cfa619b699004882a82976f0ef8204ad P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:38.734897 12027 raft_consensus.cc:2229] T cfa619b699004882a82976f0ef8204ad P c43461c4ee854ae1ac52b873ccada318 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:38.735343 12027 raft_consensus.cc:2258] T cfa619b699004882a82976f0ef8204ad P c43461c4ee854ae1ac52b873ccada318 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:38.737246 10945 tablet_service.cc:1514] Processing DeleteTablet for tablet 10657a81058241fc988d37e5ea7ff8cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:25 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:38.737603 12027 ts_tablet_manager.cc:1905] T cfa619b699004882a82976f0ef8204ad P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 09:57:38.738658 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 350904aa5957496d90a2aa2e637b591c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 350904aa5957496d90a2aa2e637b591c already in progress: creating tablet
I20260925 09:57:38.727221 12445 leader_election.cc:288] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:38.729260 11085 tablet_bootstrap.cc:492] T a436fe287a0d48a1821191adcd031d74 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:38.732039 10772 leader_election.cc:302] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:38.732951 10813 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a31b999f0fd447eacff13de71aa3fff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:23 UTC) from {username='slave'} at 127.0.0.1:33344
W20260925 09:57:38.743860 10775 leader_election.cc:341] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Not found: Tablet not found: 93e1b5837aff48d7907890c40dfcd0d0
I20260925 09:57:38.745915 12445 raft_consensus.cc:2781] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 09:57:38.746228 12445 raft_consensus.cc:482] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b8850818e2ee42a082802d0095c08d21)
I20260925 09:57:38.746539 12445 raft_consensus.cc:3037] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:57:38.749892 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a73a1f242e7430cbf52229c07b4dc82" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:38.752097 10621 leader_election.cc:341] T 9a73a1f242e7430cbf52229c07b4dc82 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:38.752516 11082 tablet_bootstrap.cc:492] T a81530200cf14717bc7cbf67bb73f4e5 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:38.752475 10621 leader_election.cc:302] T 9a73a1f242e7430cbf52229c07b4dc82 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8850818e2ee42a082802d0095c08d21; no voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:38.752980 11082 ts_tablet_manager.cc:1397] T a81530200cf14717bc7cbf67bb73f4e5 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.376s user 0.037s sys 0.032s
I20260925 09:57:38.753610 12492 raft_consensus.cc:2726] T 9a73a1f242e7430cbf52229c07b4dc82 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:38.756939 12445 raft_consensus.cc:504] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:38.760195 10775 leader_election.cc:341] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Not found: Tablet not found: 93e1b5837aff48d7907890c40dfcd0d0
I20260925 09:57:38.763214 10668 tablet_service.cc:1514] Processing DeleteTablet for tablet cfa619b699004882a82976f0ef8204ad with delete_type TABLET_DATA_DELETED (replaced by tablet e83d6dc8657d4a1f89ab0893f435abf6) from {username='slave'} at 127.0.0.1:37876
W20260925 09:57:38.764779 10380 catalog_manager.cc:4914] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): alter failed for tablet 5608c5f9b65b4da7b2fbb56417c92d88 (table default.Table_f37bcee5db2044f2bea9c353982b30cd [id=e46d114e5d484b66bb15eb0416b5ce0d]),no further retry: Illegal state: Tablet not RUNNING: STOPPED
I20260925 09:57:38.767881 10945 tablet_service.cc:1514] Processing DeleteTablet for tablet 3be0eccc65564bfba3d6e5c398c61fb6 with delete_type TABLET_DATA_DELETED (replaced by tablet cc295f7a9169475f9a9235e657be629f) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:38.769989 10945 tablet_service.cc:1514] Processing DeleteTablet for tablet 67ce2e3325214aa6bdc3347e9e8f82b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:23 UTC) from {username='slave'} at 127.0.0.1:53560
W20260925 09:57:38.665504 10387 rpcz_store.cc:269] 0925 09:57:35.893323 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:36.081666 (+188343us) service_pool.cc:227] Handling call
0925 09:57:36.262618 (+180952us) spinlock_profiling.cc:243] Waited 21.6 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2ad9c3e 00007fabf0610fb4 00007fabf0610f42 00007fabf0610e71 00007fabf0610dec 00007fabf06100a2 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:38.665039 (+2402421us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:37.068721 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:37.069482 (+ 761us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:37.069512 (+ 30us) write_op.cc:435] Acquired schema lock
0925 09:57:37.069524 (+ 12us) tablet.cc:660] Decoding operations
0925 09:57:37.070262 (+ 738us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:37.070315 (+ 53us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:37.070350 (+ 35us) tablet.cc:683] Acquiring locks for 5 operations
0925 09:57:37.070946 (+ 596us) tablet.cc:699] Row locks acquired
0925 09:57:37.070964 (+ 18us) write_op.cc:260] PREPARE: finished
0925 09:57:37.071090 (+ 126us) write_op.cc:270] Start()
0925 09:57:37.071185 (+ 95us) write_op.cc:276] Timestamp: P: 1790330257071012 usec, L: 0
0925 09:57:37.071201 (+ 16us) op_driver.cc:331] REPLICATION: starting
0925 09:57:37.071861 (+ 660us) log.cc:842] Serialized 2009 byte log entry
0925 09:57:37.244063 (+172202us) op_driver.cc:444] REPLICATION: finished
0925 09:57:37.244391 (+ 328us) write_op.cc:301] APPLY: starting
0925 09:57:37.244444 (+ 53us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:37.244591 (+ 147us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:37.244688 (+ 97us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:37.245209 (+ 521us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:37.245352 (+ 143us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0925 09:57:37.245408 (+ 56us) write_op.cc:312] APPLY: finished
0925 09:57:37.245607 (+ 199us) log.cc:842] Serialized 47 byte log entry
0925 09:57:37.245746 (+ 139us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:37.245823 (+ 77us) write_op.cc:454] Released schema lock
0925 09:57:37.245857 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:37.246024 (+ 167us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:37.246057 (+ 33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":9347,"reported_tablets":45,"spinlock_wait_cycles":49682688,"child_traces":[["op",{"apply.queue_time_us":183,"apply.run_cpu_time_us":1905,"apply.run_wall_time_us":1903,"num_ops":5,"prepare.queue_time_us":95,"prepare.run_cpu_time_us":3213,"prepare.run_wall_time_us":6046,"raft.queue_time_us":116,"raft.run_cpu_time_us":758,"raft.run_wall_time_us":760,"replication_time_us":172782}]]}
I20260925 09:57:38.772133 10387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_41a628305794438b90de11e177341305"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:38.773599 10387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_41a628305794438b90de11e177341305 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:38.774158 12537 raft_consensus.cc:482] T b39f9e7102ff4a9c96a2cc5d2676f03d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:38.774577 12537 raft_consensus.cc:504] T b39f9e7102ff4a9c96a2cc5d2676f03d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:38.775985 12379 ts_tablet_manager.cc:1939] T 5608c5f9b65b4da7b2fbb56417c92d88 P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:38.776484 12537 leader_election.cc:288] T b39f9e7102ff4a9c96a2cc5d2676f03d P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:38.776995 10679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b39f9e7102ff4a9c96a2cc5d2676f03d" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:38.777763 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b39f9e7102ff4a9c96a2cc5d2676f03d" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:38.778110 10919 leader_election.cc:341] T b39f9e7102ff4a9c96a2cc5d2676f03d P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:38.779224 10918 leader_election.cc:341] T b39f9e7102ff4a9c96a2cc5d2676f03d P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:38.779609 10918 leader_election.cc:302] T b39f9e7102ff4a9c96a2cc5d2676f03d P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:38.780565 12537 raft_consensus.cc:2726] T b39f9e7102ff4a9c96a2cc5d2676f03d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:38.781558 10380 catalog_manager.cc:4914] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): alter failed for tablet 5608c5f9b65b4da7b2fbb56417c92d88 (table default.Table_f37bcee5db2044f2bea9c353982b30cd [id=e46d114e5d484b66bb15eb0416b5ce0d]),no further retry: Illegal state: Tablet not RUNNING: STOPPED
I20260925 09:57:38.788764 10945 tablet_service.cc:1514] Processing DeleteTablet for tablet cfa619b699004882a82976f0ef8204ad with delete_type TABLET_DATA_DELETED (replaced by tablet e83d6dc8657d4a1f89ab0893f435abf6) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:38.790045 10945 tablet_service.cc:1514] Processing DeleteTablet for tablet 17bd30416fe14c0e9ab2e0c0bf5f2fb5 with delete_type TABLET_DATA_DELETED (replaced by tablet 4a6b1284cddc4b2581931d8895212ca6) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:38.791270 12379 tablet_replica.cc:330] T fb9c51f7e7fd4c029be2a710cb9462eb P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:38.792116 12379 raft_consensus.cc:2229] T fb9c51f7e7fd4c029be2a710cb9462eb P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:38.792552 12379 raft_consensus.cc:2258] T fb9c51f7e7fd4c029be2a710cb9462eb P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:38.794108 12379 ts_tablet_manager.cc:1905] T fb9c51f7e7fd4c029be2a710cb9462eb P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:38.711210 11084 raft_consensus.cc:348] T f0bf545f70ae4cd6a1ad2540c7158ecc P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:38.795816 11084 raft_consensus.cc:374] T f0bf545f70ae4cd6a1ad2540c7158ecc P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:38.796060 11084 raft_consensus.cc:729] T f0bf545f70ae4cd6a1ad2540c7158ecc P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:38.796710 11084 consensus_queue.cc:260] T f0bf545f70ae4cd6a1ad2540c7158ecc P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:38.798101 11084 ts_tablet_manager.cc:1428] T f0bf545f70ae4cd6a1ad2540c7158ecc P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.089s user 0.003s sys 0.000s
I20260925 09:57:38.798913 11084 tablet_bootstrap.cc:492] T 139fe14ba82a4137b59115b6d592a7c4 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:38.801738 10952 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e3120347c8e4a3eab57d6562ae40e8d with delete_type TABLET_DATA_DELETED (replaced by tablet c52745a8cdee461aba7485b0531245d6) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:38.809170 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 5608c5f9b65b4da7b2fbb56417c92d88 (table default.Table_f37bcee5db2044f2bea9c353982b30cd [id=e46d114e5d484b66bb15eb0416b5ce0d]) successfully deleted
W20260925 09:57:38.809783 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 6161774f61ae48ab9764a05f2188a80b because the tablet was not found. No further retry: Not found: Tablet not found: 6161774f61ae48ab9764a05f2188a80b
W20260925 09:57:38.810325 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 7a31b999f0fd447eacff13de71aa3fff because the tablet was not found. No further retry: Not found: Tablet not found: 7a31b999f0fd447eacff13de71aa3fff
W20260925 09:57:38.814850 10386 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.1:56563 (request call id 41) took 5322 ms (client timeout 1000 ms). Trace:
I20260925 09:57:38.755081 11082 raft_consensus.cc:348] T a81530200cf14717bc7cbf67bb73f4e5 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:38.819813 11082 raft_consensus.cc:374] T a81530200cf14717bc7cbf67bb73f4e5 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:38.820084 11082 raft_consensus.cc:729] T a81530200cf14717bc7cbf67bb73f4e5 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:38.820649 11082 consensus_queue.cc:260] T a81530200cf14717bc7cbf67bb73f4e5 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:38.821090 12027 ts_tablet_manager.cc:1918] T cfa619b699004882a82976f0ef8204ad P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260925 09:57:38.822703 11082 ts_tablet_manager.cc:1428] T a81530200cf14717bc7cbf67bb73f4e5 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.069s user 0.004s sys 0.000s
I20260925 09:57:38.824074 12027 log.cc:1196] T cfa619b699004882a82976f0ef8204ad P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/cfa619b699004882a82976f0ef8204ad
I20260925 09:57:38.824832 12027 ts_tablet_manager.cc:1939] T cfa619b699004882a82976f0ef8204ad P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:38.842084 11082 tablet_bootstrap.cc:492] T 3039ec3febf848dba7c4e6fd38bb8053 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:38.846170 10952 tablet_service.cc:1514] Processing DeleteTablet for tablet abfc8f390d434c47a8c4990f8406c270 with delete_type TABLET_DATA_DELETED (replaced by tablet b025f8ba34694ba09cb01ca959d3629c) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:38.846787 10945 tablet_service.cc:1514] Processing DeleteTablet for tablet b3d9a3cf5bda4cf6bc1fa3fae1cb9288 with delete_type TABLET_DATA_DELETED (replaced by tablet f46095351ea64812a7fdcfc79cd92c1f) from {username='slave'} at 127.0.0.1:53560
W20260925 09:57:38.815136 10386 rpcz_store.cc:269] 0925 09:57:33.492135 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:33.498782 (+ 6647us) service_pool.cc:227] Handling call
0925 09:57:38.814787 (+5316005us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:34.757108 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:34.757563 (+ 455us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:34.757600 (+ 37us) write_op.cc:435] Acquired schema lock
0925 09:57:34.757623 (+ 23us) tablet.cc:660] Decoding operations
0925 09:57:34.757933 (+ 310us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:34.757981 (+ 48us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:34.758006 (+ 25us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:34.758166 (+ 160us) tablet.cc:699] Row locks acquired
0925 09:57:34.758177 (+ 11us) write_op.cc:260] PREPARE: finished
0925 09:57:34.758291 (+ 114us) write_op.cc:270] Start()
0925 09:57:34.758379 (+ 88us) write_op.cc:276] Timestamp: P: 1790330254758242 usec, L: 0
0925 09:57:34.758398 (+ 19us) op_driver.cc:331] REPLICATION: starting
0925 09:57:34.758907 (+ 509us) log.cc:842] Serialized 524 byte log entry
0925 09:57:35.068329 (+309422us) op_driver.cc:444] REPLICATION: finished
0925 09:57:35.068975 (+ 646us) write_op.cc:301] APPLY: starting
0925 09:57:35.069045 (+ 70us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:35.069160 (+ 115us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:35.069273 (+ 113us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:35.069543 (+ 270us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:35.069706 (+ 163us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:35.069755 (+ 49us) write_op.cc:312] APPLY: finished
0925 09:57:35.069937 (+ 182us) log.cc:842] Serialized 23 byte log entry
0925 09:57:35.070118 (+ 181us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:35.070206 (+ 88us) write_op.cc:454] Released schema lock
0925 09:57:35.070259 (+ 53us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:35.070513 (+ 254us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:35.070545 (+ 32us) write_op.cc:454] Released schema lock
Related trace 'op':
0925 09:57:35.452690 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:35.525081 (+ 72391us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:35.525109 (+ 28us) write_op.cc:435] Acquired schema lock
0925 09:57:35.525128 (+ 19us) tablet.cc:660] Decoding operations
0925 09:57:35.525440 (+ 312us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:35.525499 (+ 59us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:35.525530 (+ 31us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:35.525681 (+ 151us) tablet.cc:699] Row locks acquired
0925 09:57:35.525699 (+ 18us) write_op.cc:260] PREPARE: finished
0925 09:57:35.525825 (+ 126us) write_op.cc:270] Start()
0925 09:57:35.615010 (+ 89185us) write_op.cc:276] Timestamp: P: 1790330255525753 usec, L: 0
0925 09:57:35.615051 (+ 41us) op_driver.cc:331] REPLICATION: starting
0925 09:57:35.615849 (+ 798us) log.cc:842] Serialized 356 byte log entry
0925 09:57:35.804057 (+188208us) op_driver.cc:444] REPLICATION: finished
0925 09:57:35.870907 (+ 66850us) write_op.cc:301] APPLY: starting
0925 09:57:35.870964 (+ 57us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:35.871053 (+ 89us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:35.871157 (+ 104us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:35.871381 (+ 224us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:35.871543 (+ 162us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:35.871595 (+ 52us) write_op.cc:312] APPLY: finished
0925 09:57:35.871786 (+ 191us) log.cc:842] Serialized 23 byte log entry
0925 09:57:35.871939 (+ 153us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:35.871995 (+ 56us) write_op.cc:454] Released schema lock
0925 09:57:35.872023 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:35.922636 (+ 50613us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:35.922664 (+ 28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":68300,"reported_tablets":34,"child_traces":[["op",{"apply.queue_time_us":444,"apply.run_cpu_time_us":1777,"apply.run_wall_time_us":1777,"mutex_wait_us":6435,"num_ops":1,"prepare.queue_time_us":140,"prepare.run_cpu_time_us":2265,"prepare.run_wall_time_us":8675,"raft.queue_time_us":6689,"raft.run_cpu_time_us":738,"raft.run_wall_time_us":738,"replication_time_us":309844,"spinlock_wait_cycles":29056}],["op",{"apply.queue_time_us":66666,"apply.run_cpu_time_us":1554,"apply.run_wall_time_us":51991,"num_ops":1,"prepare.queue_time_us":87358,"prepare.run_cpu_time_us":2714,"prepare.run_wall_time_us":235564,"raft.queue_time_us":92,"raft.run_cpu_time_us":662,"raft.run_wall_time_us":662,"replication_time_us":188916,"thread_start_us":157802,"threads_started":2}]]}
I20260925 09:57:38.852972 11085 tablet_bootstrap.cc:700] T a436fe287a0d48a1821191adcd031d74 P c43461c4ee854ae1ac52b873ccada318: Time spent opening tablet: real 0.102s user 0.004s sys 0.000s
I20260925 09:57:38.868676 11082 tablet_bootstrap.cc:654] T 3039ec3febf848dba7c4e6fd38bb8053 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:38.870532 12445 leader_election.cc:288] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:38.870590 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 354) took 5105 ms. Trace:
I20260925 09:57:38.871351 10978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93e1b5837aff48d7907890c40dfcd0d0" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:38.871507 12027 rpcz_store.cc:276] 0925 09:57:33.765390 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:33.765494 (+ 104us) service_pool.cc:227] Handling call
0925 09:57:38.733988 (+4968494us) ts_tablet_manager.cc:1145] Deleting tablet cfa619b699004882a82976f0ef8204ad
0925 09:57:38.734029 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:38.801081 (+ 67052us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:38.820811 (+ 19730us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:38.870530 (+ 49719us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":33657,"tablet-delete.queue_time_us":4967885}
I20260925 09:57:38.871965 10978 raft_consensus.cc:3037] T 93e1b5837aff48d7907890c40dfcd0d0 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:57:38.872574 12027 tablet_replica.cc:330] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:38.873281 12027 raft_consensus.cc:2229] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318 [term 2 LEADER]: Raft consensus shutting down.
I20260925 09:57:38.873875 12027 pending_rounds.cc:62] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318: Trying to abort 1 pending ops.
I20260925 09:57:38.874061 12027 pending_rounds.cc:68] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7333192737069596672 op_type: NO_OP noop_request { }
I20260925 09:57:38.874393 12027 raft_consensus.cc:2866] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:57:38.874684 12027 raft_consensus.cc:2258] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260925 09:57:38.870646 10391 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.3:41469 (request call id 33) took 1753 ms (client timeout 1000 ms). Trace:
I20260925 09:57:38.879330 11085 tablet_bootstrap.cc:654] T a436fe287a0d48a1821191adcd031d74 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:38.883344 10978 raft_consensus.cc:2454] T 93e1b5837aff48d7907890c40dfcd0d0 P be97ab5d75934a648d4ab76341193ec0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 2.
I20260925 09:57:38.885186 10772 leader_election.cc:302] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21
I20260925 09:57:38.886940 12523 raft_consensus.cc:2781] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 09:57:38.887356 12523 raft_consensus.cc:686] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318 [term 2 LEADER]: Becoming Leader. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Running, Role: LEADER
I20260925 09:57:38.888092 12523 consensus_queue.cc:237] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:38.889571 10393 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.3:41469 (request call id 32) took 3414 ms (client timeout 1000 ms). Trace:
W20260925 09:57:38.889861 10393 rpcz_store.cc:269] 0925 09:57:35.475058 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:35.829164 (+354106us) service_pool.cc:227] Handling call
0925 09:57:35.995812 (+166648us) spinlock_profiling.cc:243] Waited 84.5 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2ad9c3e 00007fabf0610fb4 00007fabf0610f42 00007fabf0610e71 00007fabf0610dec 00007fabf06100a2 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:38.889508 (+2893696us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":135704,"reported_tablets":67,"spinlock_wait_cycles":194339712}
W20260925 09:57:38.875720 10391 rpcz_store.cc:269] 0925 09:57:37.117448 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:37.273370 (+155922us) service_pool.cc:227] Handling call
0925 09:57:38.870574 (+1597204us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:38.472344 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:38.473227 (+ 883us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:38.473257 (+ 30us) write_op.cc:435] Acquired schema lock
0925 09:57:38.473274 (+ 17us) tablet.cc:660] Decoding operations
0925 09:57:38.473636 (+ 362us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:38.473698 (+ 62us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:38.473737 (+ 39us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:38.474035 (+ 298us) tablet.cc:699] Row locks acquired
0925 09:57:38.474053 (+ 18us) write_op.cc:260] PREPARE: finished
0925 09:57:38.474186 (+ 133us) write_op.cc:270] Start()
0925 09:57:38.474279 (+ 93us) write_op.cc:276] Timestamp: P: 1790330258474118 usec, L: 0
0925 09:57:38.474295 (+ 16us) op_driver.cc:331] REPLICATION: starting
0925 09:57:38.476662 (+ 2367us) log.cc:842] Serialized 895 byte log entry
0925 09:57:38.499485 (+ 22823us) op_driver.cc:444] REPLICATION: finished
0925 09:57:38.526394 (+ 26909us) write_op.cc:301] APPLY: starting
0925 09:57:38.526453 (+ 59us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:38.526558 (+ 105us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:38.526642 (+ 84us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:38.526891 (+ 249us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:38.527022 (+ 131us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 09:57:38.527063 (+ 41us) write_op.cc:312] APPLY: finished
0925 09:57:38.527243 (+ 180us) log.cc:842] Serialized 29 byte log entry
0925 09:57:38.527859 (+ 616us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:38.527931 (+ 72us) write_op.cc:454] Released schema lock
0925 09:57:38.527969 (+ 38us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:38.528134 (+ 165us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:38.528161 (+ 27us) write_op.cc:454] Released schema lock
Related trace 'op':
0925 09:57:38.569925 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:38.570349 (+ 424us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:38.570389 (+ 40us) write_op.cc:435] Acquired schema lock
0925 09:57:38.570407 (+ 18us) tablet.cc:660] Decoding operations
0925 09:57:38.570753 (+ 346us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:38.570795 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:38.570826 (+ 31us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:38.570987 (+ 161us) tablet.cc:699] Row locks acquired
0925 09:57:38.571002 (+ 15us) write_op.cc:260] PREPARE: finished
0925 09:57:38.571129 (+ 127us) write_op.cc:270] Start()
0925 09:57:38.571216 (+ 87us) write_op.cc:276] Timestamp: P: 1790330258571050 usec, L: 0
0925 09:57:38.571250 (+ 34us) op_driver.cc:331] REPLICATION: starting
0925 09:57:38.571833 (+ 583us) log.cc:842] Serialized 356 byte log entry
0925 09:57:38.583211 (+ 11378us) op_driver.cc:444] REPLICATION: finished
0925 09:57:38.588018 (+ 4807us) write_op.cc:301] APPLY: starting
0925 09:57:38.588071 (+ 53us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:38.588161 (+ 90us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:38.588259 (+ 98us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:38.588442 (+ 183us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:38.588574 (+ 132us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:38.588613 (+ 39us) write_op.cc:312] APPLY: finished
0925 09:57:38.588765 (+ 152us) log.cc:842] Serialized 23 byte log entry
0925 09:57:38.589375 (+ 610us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:38.589442 (+ 67us) write_op.cc:454] Released schema lock
0925 09:57:38.589476 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:38.589652 (+ 176us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:38.589675 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":261,"reported_tablets":70,"child_traces":[["op",{"apply.queue_time_us":26726,"apply.run_cpu_time_us":1522,"apply.run_wall_time_us":1967,"num_ops":2,"prepare.queue_time_us":5654,"prepare.run_cpu_time_us":2514,"prepare.run_wall_time_us":6598,"raft.queue_time_us":838,"raft.run_cpu_time_us":667,"raft.run_wall_time_us":667,"replication_time_us":25070,"spinlock_wait_cycles":28416,"thread_start_us":4982,"threads_started":1}],["op",{"apply.queue_time_us":4642,"apply.run_cpu_time_us":1337,"apply.run_wall_time_us":2307,"num_ops":1,"prepare.queue_time_us":117,"prepare.run_cpu_time_us":2215,"prepare.run_wall_time_us":2223,"raft.queue_time_us":879,"raft.run_cpu_time_us":630,"raft.run_wall_time_us":630,"replication_time_us":11880}]]}
I20260925 09:57:38.897217 12537 raft_consensus.cc:482] T 3d84c717b6ac462eb331e25d80c918e4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:38.897722 12537 raft_consensus.cc:504] T 3d84c717b6ac462eb331e25d80c918e4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:38.902525 12537 leader_election.cc:288] T 3d84c717b6ac462eb331e25d80c918e4 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:38.903178 10679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d84c717b6ac462eb331e25d80c918e4" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:38.903676 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d84c717b6ac462eb331e25d80c918e4" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:38.904546 10919 leader_election.cc:341] T 3d84c717b6ac462eb331e25d80c918e4 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:38.905035 10918 leader_election.cc:341] T 3d84c717b6ac462eb331e25d80c918e4 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:38.905362 10918 leader_election.cc:302] T 3d84c717b6ac462eb331e25d80c918e4 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
W20260925 09:57:38.906306 10384 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.3:41469 (request call id 30) took 4524 ms (client timeout 1000 ms). Trace:
I20260925 09:57:38.906384 12537 raft_consensus.cc:2726] T 3d84c717b6ac462eb331e25d80c918e4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:38.891088 10391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_b5303e844fd045a0b85dc5b75c932806"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:38.906797 10384 rpcz_store.cc:269] 0925 09:57:34.382136 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:35.339873 (+957737us) service_pool.cc:227] Handling call
0925 09:57:35.741497 (+401624us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2ad9c3e 00007fabf0610fb4 00007fabf0610f42 00007fabf0610e71 00007fabf0610dec 00007fabf06100a2 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:38.906238 (+3164741us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":5939,"reported_tablets":65,"spinlock_wait_cycles":274177408}
W20260925 09:57:38.912542 10391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b5303e844fd045a0b85dc5b75c932806 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 09:57:38.915628 10392 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.3:41469 (request call id 27) took 8038 ms (client timeout 1000 ms). Trace:
W20260925 09:57:38.915978 10392 rpcz_store.cc:269] 0925 09:57:30.876990 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:30.877130 (+ 140us) service_pool.cc:227] Handling call
0925 09:57:30.901215 (+ 24085us) spinlock_profiling.cc:243] Waited 9.97 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2ad9c3e 00007fabf0610fb4 00007fabf0610f42 00007fabf0610e71 00007fabf0610dec 00007fabf06100a2 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:38.735914 (+7834699us) spinlock_profiling.cc:243] Waited 4.69 ms on lock 0x7b4c000040c0. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabee853210 00007fabee863fb2 00007fabee864512 00007fabee82979b 00007fabee85673c 00007fabee856bfc 00007fabf01e7f1b 00007fabf2a2c91b 00007fabf29d1330 00007fabf29d7083 00007fabf2ada4fe
0925 09:57:38.915481 (+179567us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":70900,"reported_tablets":65,"spinlock_wait_cycles":33721344}
I20260925 09:57:38.920504 11082 tablet_bootstrap.cc:492] T 3039ec3febf848dba7c4e6fd38bb8053 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:38.920914 11082 ts_tablet_manager.cc:1397] T 3039ec3febf848dba7c4e6fd38bb8053 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.098s user 0.012s sys 0.000s
I20260925 09:57:38.923936 11082 raft_consensus.cc:348] T 3039ec3febf848dba7c4e6fd38bb8053 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:38.924558 11082 raft_consensus.cc:374] T 3039ec3febf848dba7c4e6fd38bb8053 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:38.924814 11082 raft_consensus.cc:729] T 3039ec3febf848dba7c4e6fd38bb8053 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:38.927263 12379 ts_tablet_manager.cc:1918] T fb9c51f7e7fd4c029be2a710cb9462eb P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:38.928952 12379 log.cc:1196] T fb9c51f7e7fd4c029be2a710cb9462eb P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/fb9c51f7e7fd4c029be2a710cb9462eb
I20260925 09:57:38.929736 12379 ts_tablet_manager.cc:1939] T fb9c51f7e7fd4c029be2a710cb9462eb P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:38.931783 12537 raft_consensus.cc:482] T f9c259867cf44c4c90e50864475a4970 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:38.932214 12537 raft_consensus.cc:504] T f9c259867cf44c4c90e50864475a4970 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:38.931417 11082 consensus_queue.cc:260] T 3039ec3febf848dba7c4e6fd38bb8053 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:38.942541 12537 leader_election.cc:288] T f9c259867cf44c4c90e50864475a4970 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:38.943428 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c259867cf44c4c90e50864475a4970" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:38.945500 10919 leader_election.cc:341] T f9c259867cf44c4c90e50864475a4970 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:38.948805 11082 ts_tablet_manager.cc:1428] T 3039ec3febf848dba7c4e6fd38bb8053 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.028s user 0.009s sys 0.000s
I20260925 09:57:38.949715 11082 tablet_bootstrap.cc:492] T 05e2c7f9820f41679c4d58343ff3f829 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:38.959582 12379 tablet_replica.cc:330] T 37bab60afe7a4c758a08ef7c9c24aba0 P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:38.965704 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c259867cf44c4c90e50864475a4970" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:38.967185 10918 leader_election.cc:341] T f9c259867cf44c4c90e50864475a4970 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:38.967649 10918 leader_election.cc:302] T f9c259867cf44c4c90e50864475a4970 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:38.969066 12537 raft_consensus.cc:2726] T f9c259867cf44c4c90e50864475a4970 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:38.972357 10392 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_a2a96456c8804d1692ffbafab832340b" } new_table_name: "default.Table_257e35b439d7481fbcfcd5d50b6c6c08" modify_external_catalogs: true
I20260925 09:57:38.921262 10387 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35140 (request call id 378) took 2357 ms. Trace:
I20260925 09:57:38.921993 10388 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:35140 (request call id 377) took 2373 ms. Trace:
I20260925 09:57:38.945182 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet fb9c51f7e7fd4c029be2a710cb9462eb (table default.Table_44ff944ba4b84cb7a677f7dd03f8f7dd [id=80ab4d354310414eb9374b8ac6a3a946]) successfully deleted
W20260925 09:57:38.975196 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 417058493c6547c69e72347cef373292 because the tablet was not found. No further retry: Not found: Tablet not found: 417058493c6547c69e72347cef373292
W20260925 09:57:38.975855 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 93e1b5837aff48d7907890c40dfcd0d0 because the tablet was not found. No further retry: Not found: Tablet not found: 93e1b5837aff48d7907890c40dfcd0d0
W20260925 09:57:38.976428 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 4e3120347c8e4a3eab57d6562ae40e8d because the tablet was not found. No further retry: Not found: Tablet not found: 4e3120347c8e4a3eab57d6562ae40e8d
W20260925 09:57:38.977725 10772 consensus_peers.cc:487] T 10657a81058241fc988d37e5ea7ff8cf P c43461c4ee854ae1ac52b873ccada318 -> Peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Couldn't send request to peer be97ab5d75934a648d4ab76341193ec0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 10657a81058241fc988d37e5ea7ff8cf. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:38.981014 12027 ts_tablet_manager.cc:1905] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:38.973090 10387 rpcz_store.cc:276] 0925 09:57:36.563714 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.771707 (+2207993us) service_pool.cc:227] Handling call
0925 09:57:38.831876 (+ 60169us) catalog_manager.cc:2075] Acquired catalog manager lock
0925 09:57:38.833323 (+ 1447us) catalog_manager.cc:2121] Created new table and tablet info
0925 09:57:38.903504 (+ 70181us) catalog_manager.cc:2181] Wrote table and tablets to system table
0925 09:57:38.920893 (+ 17389us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0925 09:57:38.921196 (+ 303us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:38.834455 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:38.834884 (+ 429us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:38.834916 (+ 32us) write_op.cc:435] Acquired schema lock
0925 09:57:38.834934 (+ 18us) tablet.cc:660] Decoding operations
0925 09:57:38.835238 (+ 304us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:38.835294 (+ 56us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:38.835321 (+ 27us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:38.835566 (+ 245us) tablet.cc:699] Row locks acquired
0925 09:57:38.835584 (+ 18us) write_op.cc:260] PREPARE: finished
0925 09:57:38.835710 (+ 126us) write_op.cc:270] Start()
0925 09:57:38.835800 (+ 90us) write_op.cc:276] Timestamp: P: 1790330258835651 usec, L: 0
0925 09:57:38.835818 (+ 18us) op_driver.cc:331] REPLICATION: starting
0925 09:57:38.836297 (+ 479us) log.cc:842] Serialized 433 byte log entry
0925 09:57:38.844438 (+ 8141us) op_driver.cc:444] REPLICATION: finished
0925 09:57:38.844720 (+ 282us) write_op.cc:301] APPLY: starting
0925 09:57:38.844774 (+ 54us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:38.844870 (+ 96us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:38.844975 (+ 105us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:38.845353 (+ 378us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:38.845478 (+ 125us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 09:57:38.845526 (+ 48us) write_op.cc:312] APPLY: finished
0925 09:57:38.845711 (+ 185us) log.cc:842] Serialized 29 byte log entry
0925 09:57:38.845850 (+ 139us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:38.845922 (+ 72us) write_op.cc:454] Released schema lock
0925 09:57:38.845965 (+ 43us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:38.846141 (+ 176us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:38.846168 (+ 27us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":140,"apply.run_cpu_time_us":1612,"apply.run_wall_time_us":1612,"num_ops":2,"prepare.queue_time_us":84,"prepare.run_cpu_time_us":2113,"prepare.run_wall_time_us":2120,"raft.queue_time_us":834,"raft.run_cpu_time_us":623,"raft.run_wall_time_us":623,"replication_time_us":8544}]]}
I20260925 09:57:38.986313 10387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_3a4caa0a9b2044689a5696cf420f9bcd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:38.974767 10388 rpcz_store.cc:276] 0925 09:57:36.548750 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.636458 (+2087708us) service_pool.cc:227] Handling call
0925 09:57:38.637411 (+ 953us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:57:38.638877 (+ 1466us) catalog_manager.cc:3616] Apply alter schema
0925 09:57:38.805570 (+166693us) catalog_manager.cc:3639] Acquired catalog manager lock
0925 09:57:38.805780 (+ 210us) catalog_manager.cc:3786] Updating metadata on disk
0925 09:57:38.824118 (+ 18338us) catalog_manager.cc:3817] Committing alterations to in-memory state
0925 09:57:38.921897 (+ 97779us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:38.806940 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:38.807366 (+ 426us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:38.807392 (+ 26us) write_op.cc:435] Acquired schema lock
0925 09:57:38.807408 (+ 16us) tablet.cc:660] Decoding operations
0925 09:57:38.807754 (+ 346us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:38.807795 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:38.807819 (+ 24us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:38.807995 (+ 176us) tablet.cc:699] Row locks acquired
0925 09:57:38.808010 (+ 15us) write_op.cc:260] PREPARE: finished
0925 09:57:38.808124 (+ 114us) write_op.cc:270] Start()
0925 09:57:38.808208 (+ 84us) write_op.cc:276] Timestamp: P: 1790330258808071 usec, L: 0
0925 09:57:38.808226 (+ 18us) op_driver.cc:331] REPLICATION: starting
0925 09:57:38.808713 (+ 487us) log.cc:842] Serialized 405 byte log entry
0925 09:57:38.817392 (+ 8679us) op_driver.cc:444] REPLICATION: finished
0925 09:57:38.817915 (+ 523us) write_op.cc:301] APPLY: starting
0925 09:57:38.817969 (+ 54us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:38.818053 (+ 84us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:38.818141 (+ 88us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:38.818327 (+ 186us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:38.818445 (+ 118us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:38.818485 (+ 40us) write_op.cc:312] APPLY: finished
0925 09:57:38.818648 (+ 163us) log.cc:842] Serialized 23 byte log entry
0925 09:57:38.818782 (+ 134us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:38.818838 (+ 56us) write_op.cc:454] Released schema lock
0925 09:57:38.818872 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:38.819021 (+ 149us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:38.819044 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":281,"apply.run_cpu_time_us":1270,"apply.run_wall_time_us":1270,"num_ops":1,"prepare.queue_time_us":94,"prepare.run_cpu_time_us":2080,"prepare.run_wall_time_us":2079,"raft.queue_time_us":863,"raft.run_cpu_time_us":592,"raft.run_wall_time_us":593,"replication_time_us":9066}]]}
I20260925 09:57:38.987707 10393 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_b13e058d7a264358a7b057cb23f04d18" } modify_external_catalogs: true
I20260925 09:57:38.910472 10390 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 2f07dc9548364870923165f4ea524310 (table default.Table_0db927e92a3b4970a7efea3fb765e65e [id=a988264fe43c4eb88a84f8b6359e77a6]). Expected version 1 got 0
I20260925 09:57:38.993630 10388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_f323f1cef50d4ecfbedb14f38b396eac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:57:38.997404 12379 raft_consensus.cc:2229] T 37bab60afe7a4c758a08ef7c9c24aba0 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:38.999269 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet cfa619b699004882a82976f0ef8204ad (table default.Table_2a6203820e0d44aa993080049923de10 [id=c90c671ed4f043c2b56a1f84ff9a42cd]) successfully deleted
I20260925 09:57:39.000392 12390 raft_consensus.cc:482] T cf775c9d9b114ddcb97dc6c6eca8e0fc P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:39.000895 12390 raft_consensus.cc:504] T cf775c9d9b114ddcb97dc6c6eca8e0fc P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:39.002982 12390 leader_election.cc:288] T cf775c9d9b114ddcb97dc6c6eca8e0fc P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:39.006026 12379 raft_consensus.cc:2258] T 37bab60afe7a4c758a08ef7c9c24aba0 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 09:57:39.006397 10619 leader_election.cc:341] T cf775c9d9b114ddcb97dc6c6eca8e0fc P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: cf775c9d9b114ddcb97dc6c6eca8e0fc
I20260925 09:57:39.008903 12379 ts_tablet_manager.cc:1905] T 37bab60afe7a4c758a08ef7c9c24aba0 P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:39.012028 10384 master_service.cc:946] ReplaceTablet: received request to replace tablet 05e2c7f9820f41679c4d58343ff3f829 from {username='slave'} at 127.0.0.1:35140
I20260925 09:57:39.014518 10390 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 6540aba5854a44419addc5d454e473b7 (table default.Table_8180842f664042b0a9073b7008438ff7 [id=416a4b989570457e919fb7e7b16a017d]). Expected version 1 got 0
I20260925 09:57:39.021078 11082 tablet_bootstrap.cc:654] T 05e2c7f9820f41679c4d58343ff3f829 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:39.023936 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf775c9d9b114ddcb97dc6c6eca8e0fc" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:39.027729 12460 consensus_queue.cc:1035] T 10657a81058241fc988d37e5ea7ff8cf P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:57:39.031049 11085 tablet_bootstrap.cc:492] T a436fe287a0d48a1821191adcd031d74 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:39.032184 11085 ts_tablet_manager.cc:1397] T a436fe287a0d48a1821191adcd031d74 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.303s user 0.010s sys 0.000s
I20260925 09:57:39.034750 11085 raft_consensus.cc:348] T a436fe287a0d48a1821191adcd031d74 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:39.035665 11085 raft_consensus.cc:374] T a436fe287a0d48a1821191adcd031d74 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:39.036397 11085 raft_consensus.cc:729] T a436fe287a0d48a1821191adcd031d74 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
W20260925 09:57:39.039608 10621 leader_election.cc:341] T cf775c9d9b114ddcb97dc6c6eca8e0fc P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:39.040000 10621 leader_election.cc:302] T cf775c9d9b114ddcb97dc6c6eca8e0fc P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8850818e2ee42a082802d0095c08d21; no voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:39.040612 12491 raft_consensus.cc:2726] T cf775c9d9b114ddcb97dc6c6eca8e0fc P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:39.043944 11727 ts_tablet_manager.cc:1918] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:39.044401 11727 log.cc:1196] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/a21909e2e5794f57971f4e42795235c7
I20260925 09:57:39.045181 11727 ts_tablet_manager.cc:1939] T a21909e2e5794f57971f4e42795235c7 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
W20260925 09:57:39.033725 10379 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet cf7b2d3f3b3949a0b81283bc68e7b131 (table default.Table_f261a668854045ea93bc3cd825d6a685 [id=e499d19b87184375b75fe2fb263d92b5], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet cf7b2d3f3b3949a0b81283bc68e7b131 consensus state has no leader
W20260925 09:57:39.011291 10388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f323f1cef50d4ecfbedb14f38b396eac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 09:57:39.073689 10387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3a4caa0a9b2044689a5696cf420f9bcd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:57:39.075820 11082 tablet_bootstrap.cc:492] T 05e2c7f9820f41679c4d58343ff3f829 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:39.079854 11082 ts_tablet_manager.cc:1397] T 05e2c7f9820f41679c4d58343ff3f829 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.130s user 0.015s sys 0.001s
I20260925 09:57:39.089955 12551 tablet.cc:1722] T 3c31682bf400494f82ee3ce5e6eec441 P c43461c4ee854ae1ac52b873ccada318: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:39.091935 11085 consensus_queue.cc:260] T a436fe287a0d48a1821191adcd031d74 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:39.099200 12506 raft_consensus.cc:482] T b61444c31dbf4882ac748f861ea66818 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:39.099670 12506 raft_consensus.cc:504] T b61444c31dbf4882ac748f861ea66818 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:39.101610 12506 leader_election.cc:288] T b61444c31dbf4882ac748f861ea66818 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:39.103314 10978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b61444c31dbf4882ac748f861ea66818" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:39.115911 12547 tablet.cc:1722] T 3c31682bf400494f82ee3ce5e6eec441 P b8850818e2ee42a082802d0095c08d21: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:39.118332 10830 raft_consensus.cc:3037] T b61444c31dbf4882ac748f861ea66818 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:39.127467 12548 tablet.cc:1722] T 3c31682bf400494f82ee3ce5e6eec441 P be97ab5d75934a648d4ab76341193ec0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:39.129546 10679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b61444c31dbf4882ac748f861ea66818" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:39.131073 10775 leader_election.cc:302] T b61444c31dbf4882ac748f861ea66818 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:39.132460 10391 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35140 (request call id 379) took 2528 ms. Trace:
I20260925 09:57:39.143481 12379 ts_tablet_manager.cc:1918] T 37bab60afe7a4c758a08ef7c9c24aba0 P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:39.145776 12537 raft_consensus.cc:482] T c9fccada64d1411eb1f75480c8001c49 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:39.146251 12537 raft_consensus.cc:504] T c9fccada64d1411eb1f75480c8001c49 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:39.149238 12537 leader_election.cc:288] T c9fccada64d1411eb1f75480c8001c49 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:39.154134 10388 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35140 (request call id 383) took 1671 ms. Trace:
I20260925 09:57:39.154248 12027 ts_tablet_manager.cc:1918] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260925 09:57:39.154527 10388 rpcz_store.cc:276] 0925 09:57:37.482451 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.990491 (+1508040us) service_pool.cc:227] Handling call
0925 09:57:39.120285 (+129794us) catalog_manager.cc:2075] Acquired catalog manager lock
0925 09:57:39.121890 (+ 1605us) catalog_manager.cc:2121] Created new table and tablet info
0925 09:57:39.153486 (+ 31596us) catalog_manager.cc:2181] Wrote table and tablets to system table
0925 09:57:39.153815 (+ 329us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0925 09:57:39.154076 (+ 261us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:39.123182 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:39.123649 (+ 467us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:39.123677 (+ 28us) write_op.cc:435] Acquired schema lock
0925 09:57:39.123696 (+ 19us) tablet.cc:660] Decoding operations
0925 09:57:39.124043 (+ 347us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:39.124090 (+ 47us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:39.124120 (+ 30us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:39.124363 (+ 243us) tablet.cc:699] Row locks acquired
0925 09:57:39.124383 (+ 20us) write_op.cc:260] PREPARE: finished
0925 09:57:39.124505 (+ 122us) write_op.cc:270] Start()
0925 09:57:39.124588 (+ 83us) write_op.cc:276] Timestamp: P: 1790330259124422 usec, L: 0
0925 09:57:39.124611 (+ 23us) op_driver.cc:331] REPLICATION: starting
0925 09:57:39.125129 (+ 518us) log.cc:842] Serialized 433 byte log entry
0925 09:57:39.149919 (+ 24790us) op_driver.cc:444] REPLICATION: finished
0925 09:57:39.151617 (+ 1698us) write_op.cc:301] APPLY: starting
0925 09:57:39.151701 (+ 84us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:39.151877 (+ 176us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:39.152016 (+ 139us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:39.152565 (+ 549us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:39.152793 (+ 228us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 09:57:39.152841 (+ 48us) write_op.cc:312] APPLY: finished
0925 09:57:39.153029 (+ 188us) log.cc:842] Serialized 29 byte log entry
0925 09:57:39.153162 (+ 133us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:39.153232 (+ 70us) write_op.cc:454] Released schema lock
0925 09:57:39.153275 (+ 43us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:39.153449 (+ 174us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:39.153476 (+ 27us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1495,"apply.run_cpu_time_us":1993,"apply.run_wall_time_us":2043,"num_ops":2,"prepare.queue_time_us":88,"prepare.run_cpu_time_us":2137,"prepare.run_wall_time_us":2136,"replication_time_us":25227}]]}
I20260925 09:57:39.162000 10388 master_service.cc:946] ReplaceTablet: received request to replace tablet f62c62715c9b42de826c1a66fd431eab from {username='slave'} at 127.0.0.1:35140
I20260925 09:57:39.165683 10393 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_b13e058d7a264358a7b057cb23f04d18" } modify_external_catalogs: true
I20260925 09:57:39.173130 12555 raft_consensus.cc:2726] T b61444c31dbf4882ac748f861ea66818 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:39.175068 12027 log.cc:1196] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/a21909e2e5794f57971f4e42795235c7
I20260925 09:57:39.175977 12027 ts_tablet_manager.cc:1939] T a21909e2e5794f57971f4e42795235c7 P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:39.176376 12379 log.cc:1196] T 37bab60afe7a4c758a08ef7c9c24aba0 P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/37bab60afe7a4c758a08ef7c9c24aba0
I20260925 09:57:39.178081 12379 ts_tablet_manager.cc:1939] T 37bab60afe7a4c758a08ef7c9c24aba0 P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:39.178987 10654 tablet_service.cc:1467] Processing CreateTablet for tablet ca85372248ec4795909339d9765edf04 (DEFAULT_TABLE table=default.Table_891a8679f0f040fcbeff83cb61afed65 [id=def1015251f0495cb2fb42ac4c980379]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:39.184211 10654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca85372248ec4795909339d9765edf04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:39.091223 10385 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 2f07dc9548364870923165f4ea524310 (table default.Table_0db927e92a3b4970a7efea3fb765e65e [id=a988264fe43c4eb88a84f8b6359e77a6]). Expected version 1 got 0
I20260925 09:57:39.159852 10679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9fccada64d1411eb1f75480c8001c49" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:39.177861 10806 tablet_service.cc:1467] Processing CreateTablet for tablet ca85372248ec4795909339d9765edf04 (DEFAULT_TABLE table=default.Table_891a8679f0f040fcbeff83cb61afed65 [id=def1015251f0495cb2fb42ac4c980379]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:39.180333 10813 tablet_service.cc:1467] Processing CreateTablet for tablet a8e2435327c147798c7f4e27a196fff5 (DEFAULT_TABLE table=default.Table_2e0bff2adcb743479f616bb108fe4497 [id=4ff49368ad084b12a85ac678a682c543]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 09:57:39.193646 10919 leader_election.cc:341] T c9fccada64d1411eb1f75480c8001c49 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:39.200706 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 37bab60afe7a4c758a08ef7c9c24aba0 (table default.Table_8b45818055ec40e386d43e2641ec6ed0 [id=38b800f8e1e948909d40cc976272fdd6]) successfully deleted
I20260925 09:57:39.201459 11085 ts_tablet_manager.cc:1428] T a436fe287a0d48a1821191adcd031d74 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.169s user 0.005s sys 0.002s
I20260925 09:57:39.206422 10388 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet f62c62715c9b42de826c1a66fd431eab
I20260925 09:57:39.208169 10812 tablet_service.cc:1514] Processing DeleteTablet for tablet f62c62715c9b42de826c1a66fd431eab with delete_type TABLET_DATA_DELETED (replaced by tablet 4e0df1d8cf81469692b946b3c2a50caf) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:39.212143 10668 tablet_service.cc:1514] Processing DeleteTablet for tablet f62c62715c9b42de826c1a66fd431eab with delete_type TABLET_DATA_DELETED (replaced by tablet 4e0df1d8cf81469692b946b3c2a50caf) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:39.215852 10952 tablet_service.cc:1514] Processing DeleteTablet for tablet f62c62715c9b42de826c1a66fd431eab with delete_type TABLET_DATA_DELETED (replaced by tablet 4e0df1d8cf81469692b946b3c2a50caf) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:39.219313 10388 catalog_manager.cc:6565] ReplaceTablet: tablet f62c62715c9b42de826c1a66fd431eab deleted and replaced by tablet 4e0df1d8cf81469692b946b3c2a50caf
I20260925 09:57:39.221626 11082 raft_consensus.cc:348] T 05e2c7f9820f41679c4d58343ff3f829 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:39.222404 11082 raft_consensus.cc:374] T 05e2c7f9820f41679c4d58343ff3f829 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:39.222666 11082 raft_consensus.cc:729] T 05e2c7f9820f41679c4d58343ff3f829 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
W20260925 09:57:39.226325 12529 log.cc:924] Time spent T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2: Append to log took a long time: real 0.736s user 0.036s sys 0.007s
I20260925 09:57:39.235805 10945 tablet_service.cc:1467] Processing CreateTablet for tablet ca85372248ec4795909339d9765edf04 (DEFAULT_TABLE table=default.Table_891a8679f0f040fcbeff83cb61afed65 [id=def1015251f0495cb2fb42ac4c980379]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:39.237071 10945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca85372248ec4795909339d9765edf04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:39.237936 11084 tablet_bootstrap.cc:654] T 139fe14ba82a4137b59115b6d592a7c4 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:39.238809 10668 tablet_service.cc:1467] Processing CreateTablet for tablet a8e2435327c147798c7f4e27a196fff5 (DEFAULT_TABLE table=default.Table_2e0bff2adcb743479f616bb108fe4497 [id=4ff49368ad084b12a85ac678a682c543]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:39.240556 10952 tablet_service.cc:1467] Processing CreateTablet for tablet a8e2435327c147798c7f4e27a196fff5 (DEFAULT_TABLE table=default.Table_2e0bff2adcb743479f616bb108fe4497 [id=4ff49368ad084b12a85ac678a682c543]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:39.241478 10952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8e2435327c147798c7f4e27a196fff5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:39.252731 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 361) took 3863 ms. Trace:
I20260925 09:57:39.253146 11727 rpcz_store.cc:276] 0925 09:57:35.389288 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:35.389382 (+ 94us) service_pool.cc:227] Handling call
0925 09:57:38.608121 (+3218739us) ts_tablet_manager.cc:1145] Deleting tablet a21909e2e5794f57971f4e42795235c7
0925 09:57:38.608161 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:38.827671 (+219510us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:39.043691 (+216020us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:39.252631 (+208940us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3218233}
I20260925 09:57:39.254508 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet a21909e2e5794f57971f4e42795235c7 (table default.Table_48f8cb39156845c8bffbb2f9f17a900b [id=093bf6e622054652ad406255d557d0c8]) successfully deleted
I20260925 09:57:39.258631 12379 tablet_replica.cc:330] T dba14a44a9de492886d629441606767e P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:39.136198 10391 rpcz_store.cc:276] 0925 09:57:36.600458 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.890683 (+2290225us) service_pool.cc:227] Handling call
0925 09:57:39.107882 (+217199us) catalog_manager.cc:2075] Acquired catalog manager lock
0925 09:57:39.109529 (+ 1647us) catalog_manager.cc:2121] Created new table and tablet info
0925 09:57:39.127762 (+ 18233us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:39.127959 (+ 197us) write_op.cc:454] Released schema lock
0925 09:57:39.128162 (+ 203us) catalog_manager.cc:2181] Wrote table and tablets to system table
0925 09:57:39.128472 (+ 310us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0925 09:57:39.128714 (+ 242us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:39.111174 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:39.112262 (+ 1088us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:39.112317 (+ 55us) write_op.cc:435] Acquired schema lock
0925 09:57:39.112364 (+ 47us) tablet.cc:660] Decoding operations
0925 09:57:39.112779 (+ 415us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:39.112828 (+ 49us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:39.112858 (+ 30us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:39.113098 (+ 240us) tablet.cc:699] Row locks acquired
0925 09:57:39.113117 (+ 19us) write_op.cc:260] PREPARE: finished
0925 09:57:39.113246 (+ 129us) write_op.cc:270] Start()
0925 09:57:39.113346 (+ 100us) write_op.cc:276] Timestamp: P: 1790330259113161 usec, L: 0
0925 09:57:39.113369 (+ 23us) op_driver.cc:331] REPLICATION: starting
0925 09:57:39.113911 (+ 542us) log.cc:842] Serialized 433 byte log entry
0925 09:57:39.123164 (+ 9253us) op_driver.cc:444] REPLICATION: finished
0925 09:57:39.123522 (+ 358us) write_op.cc:301] APPLY: starting
0925 09:57:39.123582 (+ 60us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:39.123696 (+ 114us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:39.123803 (+ 107us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:39.124239 (+ 436us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:39.124451 (+ 212us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 09:57:39.124493 (+ 42us) write_op.cc:312] APPLY: finished
0925 09:57:39.124689 (+ 196us) log.cc:842] Serialized 29 byte log entry
0925 09:57:39.124830 (+ 141us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:39.124896 (+ 66us) write_op.cc:454] Released schema lock
0925 09:57:39.124927 (+ 31us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":213,"apply.run_cpu_time_us":1594,"apply.run_wall_time_us":1594,"num_ops":2,"prepare.queue_time_us":140,"prepare.run_cpu_time_us":3085,"prepare.run_wall_time_us":3086,"raft.queue_time_us":953,"raft.run_cpu_time_us":711,"raft.run_wall_time_us":719,"replication_time_us":9724}]]}
I20260925 09:57:39.223315 11082 consensus_queue.cc:260] T 05e2c7f9820f41679c4d58343ff3f829 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:39.264814 11082 ts_tablet_manager.cc:1428] T 05e2c7f9820f41679c4d58343ff3f829 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.182s user 0.005s sys 0.000s
I20260925 09:57:39.267261 11727 tablet_replica.cc:330] T 93e1b5837aff48d7907890c40dfcd0d0 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:39.270267 10806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca85372248ec4795909339d9765edf04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:39.275738 12491 raft_consensus.cc:482] T a81530200cf14717bc7cbf67bb73f4e5 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:39.276194 12491 raft_consensus.cc:504] T a81530200cf14717bc7cbf67bb73f4e5 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:39.278066 12491 leader_election.cc:288] T a81530200cf14717bc7cbf67bb73f4e5 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:39.280089 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a81530200cf14717bc7cbf67bb73f4e5" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:39.282022 10621 leader_election.cc:341] T a81530200cf14717bc7cbf67bb73f4e5 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:39.282629 10619 leader_election.cc:341] T a81530200cf14717bc7cbf67bb73f4e5 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: a81530200cf14717bc7cbf67bb73f4e5
I20260925 09:57:39.283121 10619 leader_election.cc:302] T a81530200cf14717bc7cbf67bb73f4e5 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8850818e2ee42a082802d0095c08d21; no voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:39.284121 12491 raft_consensus.cc:2726] T a81530200cf14717bc7cbf67bb73f4e5 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:39.284472 10385 catalog_manager.cc:5587] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257) does not have the latest schema for tablet 6540aba5854a44419addc5d454e473b7 (table default.Table_8180842f664042b0a9073b7008438ff7 [id=416a4b989570457e919fb7e7b16a017d]). Expected version 1 got 0
I20260925 09:57:39.285439 10276 master-stress-test.cc:578] Killing test thread 1/10
I20260925 09:57:39.289327 10276 master-stress-test.cc:580] Killed test thread 1/10
I20260925 09:57:39.289570 10276 master-stress-test.cc:578] Killing test thread 2/10
I20260925 09:57:39.290123 10391 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_0011387b45014de4b24b466f4cb907f7" } modify_external_catalogs: true
I20260925 09:57:39.300925 10668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8e2435327c147798c7f4e27a196fff5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:39.302713 11082 tablet_bootstrap.cc:492] T d54ab15dd054489ab50b7f0b26b79a50 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:39.305239 10813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8e2435327c147798c7f4e27a196fff5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:39.305660 11085 tablet_bootstrap.cc:492] T 96a80382bb7d4395a14ad9ae972f4a25 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:39.309017 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 365) took 4019 ms. Trace:
I20260925 09:57:39.309372 12027 rpcz_store.cc:276] 0925 09:57:35.289847 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:35.289981 (+ 134us) service_pool.cc:227] Handling call
0925 09:57:38.872428 (+3582447us) ts_tablet_manager.cc:1145] Deleting tablet a21909e2e5794f57971f4e42795235c7
0925 09:57:38.872471 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:39.078341 (+205870us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:39.154037 (+ 75696us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:39.308959 (+154922us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3581714}
I20260925 09:57:39.316991 10388 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:35140 (request call id 385) took 1203 ms. Trace:
I20260925 09:57:39.319816 12027 tablet_replica.cc:330] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:39.320663 12027 raft_consensus.cc:2229] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318 [term 2 LEADER]: Raft consensus shutting down.
I20260925 09:57:39.321431 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet a21909e2e5794f57971f4e42795235c7 (table default.Table_48f8cb39156845c8bffbb2f9f17a900b [id=093bf6e622054652ad406255d557d0c8]) successfully deleted
I20260925 09:57:39.322515 12027 pending_rounds.cc:62] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318: Trying to abort 1 pending ops.
I20260925 09:57:39.322927 12027 pending_rounds.cc:68] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318: Aborting op as it isn't in flight: id { term: 2 index: 3 } timestamp: 7333192740443226112 op_type: NO_OP noop_request { }
I20260925 09:57:39.323367 12027 raft_consensus.cc:2866] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:57:39.323758 12027 raft_consensus.cc:2258] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:39.326246 12027 ts_tablet_manager.cc:1905] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 09:57:39.329835 10918 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 184429us
I20260925 09:57:39.331157 10830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9fccada64d1411eb1f75480c8001c49" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:39.340296 12578 raft_consensus.cc:482] T 54157e776efa4c6c8e2bd4c58321fcda P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:39.345010 12547 tablet.cc:1722] T 6540aba5854a44419addc5d454e473b7 P b8850818e2ee42a082802d0095c08d21: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:39.347458 12551 tablet.cc:1722] T 6540aba5854a44419addc5d454e473b7 P c43461c4ee854ae1ac52b873ccada318: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:39.351650 12379 raft_consensus.cc:2229] T dba14a44a9de492886d629441606767e P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:39.353214 12379 raft_consensus.cc:2258] T dba14a44a9de492886d629441606767e P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:39.355283 12379 ts_tablet_manager.cc:1905] T dba14a44a9de492886d629441606767e P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:39.318046 10388 rpcz_store.cc:276] 0925 09:57:38.023413 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:39.161852 (+1138439us) service_pool.cc:227] Handling call
0925 09:57:39.226287 (+ 64435us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:39.176158 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:39.176666 (+ 508us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:39.176701 (+ 35us) write_op.cc:435] Acquired schema lock
0925 09:57:39.176718 (+ 17us) tablet.cc:660] Decoding operations
0925 09:57:39.177074 (+ 356us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:39.177116 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:39.177143 (+ 27us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:39.177368 (+ 225us) tablet.cc:699] Row locks acquired
0925 09:57:39.177384 (+ 16us) write_op.cc:260] PREPARE: finished
0925 09:57:39.180181 (+ 2797us) write_op.cc:270] Start()
0925 09:57:39.180275 (+ 94us) write_op.cc:276] Timestamp: P: 1790330259180084 usec, L: 0
0925 09:57:39.180292 (+ 17us) op_driver.cc:331] REPLICATION: starting
0925 09:57:39.180967 (+ 675us) log.cc:842] Serialized 619 byte log entry
0925 09:57:39.200232 (+ 19265us) op_driver.cc:444] REPLICATION: finished
0925 09:57:39.204367 (+ 4135us) write_op.cc:301] APPLY: starting
0925 09:57:39.204427 (+ 60us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:39.204536 (+ 109us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:39.204650 (+ 114us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:39.205016 (+ 366us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:39.205226 (+ 210us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:39.205284 (+ 58us) write_op.cc:312] APPLY: finished
0925 09:57:39.205457 (+ 173us) log.cc:842] Serialized 29 byte log entry
0925 09:57:39.205598 (+ 141us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:39.205668 (+ 70us) write_op.cc:454] Released schema lock
0925 09:57:39.205700 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:39.205880 (+ 180us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:39.205911 (+ 31us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":3937,"apply.run_cpu_time_us":1732,"apply.run_wall_time_us":1731,"num_ops":2,"prepare.queue_time_us":243,"prepare.run_cpu_time_us":2354,"prepare.run_wall_time_us":5054,"replication_time_us":19845}]]}
W20260925 09:57:39.368461 10621 consensus_peers.cc:487] T 9744b9814b69470396ce039348fe5e45 P b8850818e2ee42a082802d0095c08d21 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:39.374297 11085 tablet_bootstrap.cc:654] T 96a80382bb7d4395a14ad9ae972f4a25 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:39.371065 10388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35140:
name: "default.Table_c8324101e1254bf2ad787feab7261dae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:57:39.380609 10388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c8324101e1254bf2ad787feab7261dae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 09:57:39.345794 10918 leader_election.cc:341] T c9fccada64d1411eb1f75480c8001c49 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:39.408520 10918 leader_election.cc:302] T c9fccada64d1411eb1f75480c8001c49 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
W20260925 09:57:39.409284 10918 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 63525.4us
I20260925 09:57:39.410444 12571 raft_consensus.cc:2726] T c9fccada64d1411eb1f75480c8001c49 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:39.381059 11084 tablet_bootstrap.cc:492] T 139fe14ba82a4137b59115b6d592a7c4 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:39.412861 11084 ts_tablet_manager.cc:1397] T 139fe14ba82a4137b59115b6d592a7c4 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.614s user 0.015s sys 0.008s
I20260925 09:57:39.415357 11084 raft_consensus.cc:348] T 139fe14ba82a4137b59115b6d592a7c4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:39.416299 11084 raft_consensus.cc:374] T 139fe14ba82a4137b59115b6d592a7c4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:39.416589 11084 raft_consensus.cc:729] T 139fe14ba82a4137b59115b6d592a7c4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:39.417735 11084 consensus_queue.cc:260] T 139fe14ba82a4137b59115b6d592a7c4 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:39.433924 12379 ts_tablet_manager.cc:1918] T dba14a44a9de492886d629441606767e P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:39.442654 10806 tablet_service.cc:1514] Processing DeleteTablet for tablet 37bab60afe7a4c758a08ef7c9c24aba0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:26 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:39.443197 12379 log.cc:1196] T dba14a44a9de492886d629441606767e P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/dba14a44a9de492886d629441606767e
I20260925 09:57:39.444139 12379 ts_tablet_manager.cc:1939] T dba14a44a9de492886d629441606767e P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:39.446779 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet dba14a44a9de492886d629441606767e (table default.Table_bb742c618d96453aaabe897d54fc657b [id=68683893b91c4f0190a0050ecdc6d396]) successfully deleted
I20260925 09:57:39.447505 11082 tablet_bootstrap.cc:654] T d54ab15dd054489ab50b7f0b26b79a50 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:39.448362 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet 545ac025f2cb469da847979011a0b3ce because the tablet was not found. No further retry: Not found: Tablet not found: 545ac025f2cb469da847979011a0b3ce
I20260925 09:57:39.449203 12379 tablet_replica.cc:330] T 10657a81058241fc988d37e5ea7ff8cf P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:39.450084 12379 raft_consensus.cc:2229] T 10657a81058241fc988d37e5ea7ff8cf P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:39.454440 10806 tablet_service.cc:1514] Processing DeleteTablet for tablet dba14a44a9de492886d629441606767e with delete_type TABLET_DATA_DELETED (replaced by tablet 080c76ce032848c4af26ec5c8566f300) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:39.455006 12379 raft_consensus.cc:2258] T 10657a81058241fc988d37e5ea7ff8cf P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:39.457760 12379 ts_tablet_manager.cc:1905] T 10657a81058241fc988d37e5ea7ff8cf P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 09:57:39.458148 10772 consensus_peers.cc:487] T 92814437d53143a79f99199f8ee3a742 P c43461c4ee854ae1ac52b873ccada318 -> Peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Couldn't send request to peer be97ab5d75934a648d4ab76341193ec0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 92814437d53143a79f99199f8ee3a742. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:39.462378 12537 raft_consensus.cc:482] T b3d9a3cf5bda4cf6bc1fa3fae1cb9288 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:39.462826 12537 raft_consensus.cc:504] T b3d9a3cf5bda4cf6bc1fa3fae1cb9288 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:39.464727 11085 tablet_bootstrap.cc:492] T 96a80382bb7d4395a14ad9ae972f4a25 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:39.464725 12537 leader_election.cc:288] T b3d9a3cf5bda4cf6bc1fa3fae1cb9288 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:39.465139 11085 ts_tablet_manager.cc:1397] T 96a80382bb7d4395a14ad9ae972f4a25 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.160s user 0.014s sys 0.000s
I20260925 09:57:39.466581 12506 raft_consensus.cc:482] T ab6bdba5ca5d41ae921a90c1f77006c5 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:39.466629 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3d9a3cf5bda4cf6bc1fa3fae1cb9288" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:39.467020 12506 raft_consensus.cc:504] T ab6bdba5ca5d41ae921a90c1f77006c5 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:39.467305 11085 raft_consensus.cc:348] T 96a80382bb7d4395a14ad9ae972f4a25 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:39.467418 10679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3d9a3cf5bda4cf6bc1fa3fae1cb9288" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:39.467959 11085 raft_consensus.cc:374] T 96a80382bb7d4395a14ad9ae972f4a25 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:39.468178 11085 raft_consensus.cc:729] T 96a80382bb7d4395a14ad9ae972f4a25 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:39.468852 11085 consensus_queue.cc:260] T 96a80382bb7d4395a14ad9ae972f4a25 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:39.470263 12506 leader_election.cc:288] T ab6bdba5ca5d41ae921a90c1f77006c5 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
W20260925 09:57:39.470527 10919 leader_election.cc:341] T b3d9a3cf5bda4cf6bc1fa3fae1cb9288 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:39.471354 10918 leader_election.cc:341] T b3d9a3cf5bda4cf6bc1fa3fae1cb9288 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:39.471163 10390 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.2:40761 (request call id 44) took 1887 ms (client timeout 1000 ms). Trace:
I20260925 09:57:39.471740 10918 leader_election.cc:302] T b3d9a3cf5bda4cf6bc1fa3fae1cb9288 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
W20260925 09:57:39.471866 10390 rpcz_store.cc:269] 0925 09:57:37.584128 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.534722 (+950594us) service_pool.cc:227] Handling call
0925 09:57:39.471090 (+936368us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":24650,"reported_tablets":23}
I20260925 09:57:39.472440 10679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab6bdba5ca5d41ae921a90c1f77006c5" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:39.473256 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab6bdba5ca5d41ae921a90c1f77006c5" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:39.473416 10832 raft_consensus.cc:3037] T a436fe287a0d48a1821191adcd031d74 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:39.473621 10390 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_a0c3cadc623f413bb77d0ddb87cafa8c" } new_table_name: "default.Table_36cd59e3f299423fa9f1a2714df4454c" modify_external_catalogs: true
I20260925 09:57:39.475785 10775 leader_election.cc:302] T ab6bdba5ca5d41ae921a90c1f77006c5 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:39.475298 12537 raft_consensus.cc:2726] T b3d9a3cf5bda4cf6bc1fa3fae1cb9288 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:39.477056 12523 raft_consensus.cc:2726] T ab6bdba5ca5d41ae921a90c1f77006c5 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:39.480152 12537 raft_consensus.cc:482] T f0bf545f70ae4cd6a1ad2540c7158ecc P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:39.480592 12537 raft_consensus.cc:504] T f0bf545f70ae4cd6a1ad2540c7158ecc P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:39.483250 10679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0bf545f70ae4cd6a1ad2540c7158ecc" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:39.484378 10919 leader_election.cc:341] T f0bf545f70ae4cd6a1ad2540c7158ecc P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:39.485805 12537 leader_election.cc:288] T f0bf545f70ae4cd6a1ad2540c7158ecc P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:39.486931 10830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0bf545f70ae4cd6a1ad2540c7158ecc" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:39.488145 10918 leader_election.cc:341] T f0bf545f70ae4cd6a1ad2540c7158ecc P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:39.488533 10918 leader_election.cc:302] T f0bf545f70ae4cd6a1ad2540c7158ecc P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:39.489406 12537 raft_consensus.cc:2726] T f0bf545f70ae4cd6a1ad2540c7158ecc P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:39.495117 11727 raft_consensus.cc:2229] T 93e1b5837aff48d7907890c40dfcd0d0 P be97ab5d75934a648d4ab76341193ec0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:39.495680 11727 raft_consensus.cc:2258] T 93e1b5837aff48d7907890c40dfcd0d0 P be97ab5d75934a648d4ab76341193ec0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:39.497517 11727 ts_tablet_manager.cc:1905] T 93e1b5837aff48d7907890c40dfcd0d0 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:39.503049 12555 consensus_queue.cc:1035] T 92814437d53143a79f99199f8ee3a742 P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:39.515651 12548 tablet.cc:1722] T 6540aba5854a44419addc5d454e473b7 P be97ab5d75934a648d4ab76341193ec0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:39.515674 10386 catalog_manager.cc:5495] T a436fe287a0d48a1821191adcd031d74 P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: HEALTHY } } }
I20260925 09:57:39.517928 10384 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 05e2c7f9820f41679c4d58343ff3f829
I20260925 09:57:39.519733 10384 catalog_manager.cc:6565] ReplaceTablet: tablet 05e2c7f9820f41679c4d58343ff3f829 deleted and replaced by tablet 4637237acb184a6aaae94a04218f896b
I20260925 09:57:39.519755 10668 tablet_service.cc:1514] Processing DeleteTablet for tablet 05e2c7f9820f41679c4d58343ff3f829 with delete_type TABLET_DATA_DELETED (replaced by tablet 4637237acb184a6aaae94a04218f896b) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:39.520126 10384 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:35140 (request call id 384) took 1809 ms. Trace:
I20260925 09:57:39.522668 10945 tablet_service.cc:1514] Processing DeleteTablet for tablet 05e2c7f9820f41679c4d58343ff3f829 with delete_type TABLET_DATA_DELETED (replaced by tablet 4637237acb184a6aaae94a04218f896b) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:39.522449 10384 rpcz_store.cc:276] 0925 09:57:37.710414 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:39.011898 (+1301484us) service_pool.cc:227] Handling call
0925 09:57:39.520077 (+508179us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:39.227884 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:39.228334 (+ 450us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:39.228363 (+ 29us) write_op.cc:435] Acquired schema lock
0925 09:57:39.228378 (+ 15us) tablet.cc:660] Decoding operations
0925 09:57:39.228813 (+ 435us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:39.228855 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:39.228885 (+ 30us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:39.229117 (+ 232us) tablet.cc:699] Row locks acquired
0925 09:57:39.229137 (+ 20us) write_op.cc:260] PREPARE: finished
0925 09:57:39.229257 (+ 120us) write_op.cc:270] Start()
0925 09:57:39.229356 (+ 99us) write_op.cc:276] Timestamp: P: 1790330259229204 usec, L: 0
0925 09:57:39.229382 (+ 26us) op_driver.cc:331] REPLICATION: starting
0925 09:57:39.230011 (+ 629us) log.cc:842] Serialized 619 byte log entry
0925 09:57:39.256398 (+ 26387us) op_driver.cc:444] REPLICATION: finished
0925 09:57:39.267046 (+ 10648us) write_op.cc:301] APPLY: starting
0925 09:57:39.267102 (+ 56us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:39.268630 (+ 1528us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:39.268725 (+ 95us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:39.269072 (+ 347us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:39.269287 (+ 215us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:39.269330 (+ 43us) write_op.cc:312] APPLY: finished
0925 09:57:39.269498 (+ 168us) log.cc:842] Serialized 29 byte log entry
0925 09:57:39.269638 (+ 140us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:39.269701 (+ 63us) write_op.cc:454] Released schema lock
0925 09:57:39.269731 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:39.269885 (+ 154us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:39.269908 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":7045,"apply.run_cpu_time_us":1718,"apply.run_wall_time_us":16895,"num_ops":2,"prepare.queue_time_us":98,"prepare.run_cpu_time_us":2213,"prepare.run_wall_time_us":2337,"replication_time_us":26942}]]}
I20260925 09:57:39.526176 10806 tablet_service.cc:1514] Processing DeleteTablet for tablet cfa619b699004882a82976f0ef8204ad with delete_type TABLET_DATA_DELETED (replaced by tablet e83d6dc8657d4a1f89ab0893f435abf6) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:39.526503 10813 tablet_service.cc:1514] Processing DeleteTablet for tablet 05e2c7f9820f41679c4d58343ff3f829 with delete_type TABLET_DATA_DELETED (replaced by tablet 4637237acb184a6aaae94a04218f896b) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:39.528995 11082 tablet_bootstrap.cc:492] T d54ab15dd054489ab50b7f0b26b79a50 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:39.529460 12379 ts_tablet_manager.cc:1918] T 10657a81058241fc988d37e5ea7ff8cf P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:39.408521 12578 raft_consensus.cc:504] T 54157e776efa4c6c8e2bd4c58321fcda P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:39.534718 12578 leader_election.cc:288] T 54157e776efa4c6c8e2bd4c58321fcda P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:39.535993 10830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54157e776efa4c6c8e2bd4c58321fcda" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:39.536556 12379 log.cc:1196] T 10657a81058241fc988d37e5ea7ff8cf P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/10657a81058241fc988d37e5ea7ff8cf
I20260925 09:57:39.535987 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54157e776efa4c6c8e2bd4c58321fcda" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:39.537079 10918 leader_election.cc:341] T 54157e776efa4c6c8e2bd4c58321fcda P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:39.529883 11082 ts_tablet_manager.cc:1397] T d54ab15dd054489ab50b7f0b26b79a50 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.227s user 0.010s sys 0.000s
W20260925 09:57:39.537758 10919 leader_election.cc:341] T 54157e776efa4c6c8e2bd4c58321fcda P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:39.538092 10919 leader_election.cc:302] T 54157e776efa4c6c8e2bd4c58321fcda P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:39.538398 12379 ts_tablet_manager.cc:1939] T 10657a81058241fc988d37e5ea7ff8cf P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:39.544883 10813 tablet_service.cc:1514] Processing DeleteTablet for tablet a21909e2e5794f57971f4e42795235c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:32 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:39.545543 11084 ts_tablet_manager.cc:1428] T 139fe14ba82a4137b59115b6d592a7c4 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.132s user 0.003s sys 0.005s
I20260925 09:57:39.546418 11084 tablet_bootstrap.cc:492] T bbcfc4dca399496fa41105eff0c71c3b P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:39.546489 10813 tablet_service.cc:1514] Processing DeleteTablet for tablet 10657a81058241fc988d37e5ea7ff8cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:25 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:39.525415 10391 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35140:
table { table_name: "default.Table_0011387b45014de4b24b466f4cb907f7" } modify_external_catalogs: true
I20260925 09:57:39.530150 10387 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35140 (request call id 382) took 2306 ms. Trace:
I20260925 09:57:39.548910 10387 rpcz_store.cc:276] 0925 09:57:37.224106 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.985857 (+1761751us) service_pool.cc:227] Handling call
0925 09:57:39.213623 (+227766us) catalog_manager.cc:2075] Acquired catalog manager lock
0925 09:57:39.215220 (+ 1597us) catalog_manager.cc:2121] Created new table and tablet info
0925 09:57:39.261686 (+ 46466us) catalog_manager.cc:2181] Wrote table and tablets to system table
0925 09:57:39.529844 (+268158us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0925 09:57:39.530099 (+ 255us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:39.216669 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:39.217122 (+ 453us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:39.217152 (+ 30us) write_op.cc:435] Acquired schema lock
0925 09:57:39.217171 (+ 19us) tablet.cc:660] Decoding operations
0925 09:57:39.217497 (+ 326us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:39.217547 (+ 50us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:39.217575 (+ 28us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:39.217811 (+ 236us) tablet.cc:699] Row locks acquired
0925 09:57:39.217830 (+ 19us) write_op.cc:260] PREPARE: finished
0925 09:57:39.217958 (+ 128us) write_op.cc:270] Start()
0925 09:57:39.218043 (+ 85us) write_op.cc:276] Timestamp: P: 1790330259217915 usec, L: 0
0925 09:57:39.218066 (+ 23us) op_driver.cc:331] REPLICATION: starting
0925 09:57:39.218600 (+ 534us) log.cc:842] Serialized 433 byte log entry
0925 09:57:39.242994 (+ 24394us) op_driver.cc:444] REPLICATION: finished
0925 09:57:39.244608 (+ 1614us) write_op.cc:301] APPLY: starting
0925 09:57:39.244662 (+ 54us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:39.244788 (+ 126us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:39.244901 (+ 113us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:39.245313 (+ 412us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:39.245452 (+ 139us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 09:57:39.245507 (+ 55us) write_op.cc:312] APPLY: finished
0925 09:57:39.245699 (+ 192us) log.cc:842] Serialized 29 byte log entry
0925 09:57:39.245867 (+ 168us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:39.245937 (+ 70us) write_op.cc:454] Released schema lock
0925 09:57:39.245968 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:39.246148 (+ 180us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:39.246174 (+ 26us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1458,"apply.run_cpu_time_us":1744,"apply.run_wall_time_us":3608,"num_ops":2,"prepare.queue_time_us":94,"prepare.run_cpu_time_us":2231,"prepare.run_wall_time_us":2232,"raft.queue_time_us":1102,"raft.run_cpu_time_us":649,"raft.run_wall_time_us":649,"replication_time_us":24846}]]}
I20260925 09:57:39.551744 11085 ts_tablet_manager.cc:1428] T 96a80382bb7d4395a14ad9ae972f4a25 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.086s user 0.006s sys 0.001s
I20260925 09:57:39.557560 10384 rpcz_store.cc:275] Call kudu.master.MasterService.GetMasterRegistration from 127.0.0.1:47424 (request call id 0) took 1792 ms. Trace:
I20260925 09:57:39.557772 11085 tablet_bootstrap.cc:492] T cf775c9d9b114ddcb97dc6c6eca8e0fc P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:39.559964 10276 master-stress-test.cc:580] Killed test thread 2/10
I20260925 09:57:39.560227 10276 master-stress-test.cc:578] Killing test thread 3/10
I20260925 09:57:39.563422 12537 raft_consensus.cc:2726] T 54157e776efa4c6c8e2bd4c58321fcda P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:39.563959 12537 raft_consensus.cc:482] T 8a3149fb2325435699af36338e77d042 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:39.564289 12537 raft_consensus.cc:504] T 8a3149fb2325435699af36338e77d042 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:39.566046 12537 leader_election.cc:288] T 8a3149fb2325435699af36338e77d042 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:39.567555 10384 rpcz_store.cc:276] 0925 09:57:37.764945 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:39.557027 (+1792082us) service_pool.cc:227] Handling call
0925 09:57:39.557486 (+ 459us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260925 09:57:39.568266 11082 raft_consensus.cc:348] T d54ab15dd054489ab50b7f0b26b79a50 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:39.568925 11082 raft_consensus.cc:374] T d54ab15dd054489ab50b7f0b26b79a50 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:39.569167 11082 raft_consensus.cc:729] T d54ab15dd054489ab50b7f0b26b79a50 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:39.572566 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a3149fb2325435699af36338e77d042" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:39.574105 10919 leader_election.cc:341] T 8a3149fb2325435699af36338e77d042 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:39.579653 11082 consensus_queue.cc:260] T d54ab15dd054489ab50b7f0b26b79a50 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:39.581984 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 10657a81058241fc988d37e5ea7ff8cf (table default.Table_64ba123898174e72a6884dbb9ad07de3 [id=7ee526720b2442119e0fc487b7c4fcd4]) successfully deleted
I20260925 09:57:39.582893 12379 tablet_replica.cc:330] T cf775c9d9b114ddcb97dc6c6eca8e0fc P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:39.590267 12379 raft_consensus.cc:2229] T cf775c9d9b114ddcb97dc6c6eca8e0fc P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:39.604099 10386 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 2f07dc9548364870923165f4ea524310 (table default.Table_0db927e92a3b4970a7efea3fb765e65e [id=a988264fe43c4eb88a84f8b6359e77a6]). Expected version 1 got 0
I20260925 09:57:39.605250 10386 catalog_manager.cc:5587] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197) does not have the latest schema for tablet 6540aba5854a44419addc5d454e473b7 (table default.Table_8180842f664042b0a9073b7008438ff7 [id=416a4b989570457e919fb7e7b16a017d]). Expected version 1 got 0
I20260925 09:57:39.606144 10386 catalog_manager.cc:5495] T aca5987b14b64152887ac5ee584b257a P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: HEALTHY } } }
I20260925 09:57:39.611267 10276 master-stress-test.cc:580] Killed test thread 3/10
I20260925 09:57:39.611536 10276 master-stress-test.cc:578] Killing test thread 4/10
I20260925 09:57:39.611763 10276 master-stress-test.cc:580] Killed test thread 4/10
I20260925 09:57:39.611953 10276 master-stress-test.cc:578] Killing test thread 5/10
I20260925 09:57:39.612799 11727 ts_tablet_manager.cc:1918] T 93e1b5837aff48d7907890c40dfcd0d0 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:57:39.613209 11727 log.cc:1196] T 93e1b5837aff48d7907890c40dfcd0d0 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/93e1b5837aff48d7907890c40dfcd0d0
I20260925 09:57:39.613827 11082 ts_tablet_manager.cc:1428] T d54ab15dd054489ab50b7f0b26b79a50 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.076s user 0.005s sys 0.000s
I20260925 09:57:39.614197 11727 ts_tablet_manager.cc:1939] T 93e1b5837aff48d7907890c40dfcd0d0 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:39.614777 11082 tablet_bootstrap.cc:492] T b39f9e7102ff4a9c96a2cc5d2676f03d P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:39.619791 12379 raft_consensus.cc:2258] T cf775c9d9b114ddcb97dc6c6eca8e0fc P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:39.621423 12379 ts_tablet_manager.cc:1905] T cf775c9d9b114ddcb97dc6c6eca8e0fc P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 09:57:39.599046 10385 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.2:40761 (request call id 45) took 951 ms (client timeout 1000 ms). Trace:
W20260925 09:57:39.623850 10385 rpcz_store.cc:269] 0925 09:57:38.647680 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.648055 (+ 375us) service_pool.cc:227] Handling call
0925 09:57:39.598976 (+950921us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":166573,"reported_tablets":26}
I20260925 09:57:39.637269 12027 ts_tablet_manager.cc:1918] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.3
I20260925 09:57:39.640066 12027 log.cc:1196] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/93e1b5837aff48d7907890c40dfcd0d0
I20260925 09:57:39.640872 12027 ts_tablet_manager.cc:1939] T 93e1b5837aff48d7907890c40dfcd0d0 P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:39.642732 12503 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8af3f111e8da4d2dad4e4a98ef0fd5d3. Latest consensus state: current_term: 4 leader_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } } }
I20260925 09:57:39.643529 12503 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:57:39.647522 12462 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8af3f111e8da4d2dad4e4a98ef0fd5d3)
I20260925 09:57:39.648046 12462 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } }
I20260925 09:57:39.650358 12537 raft_consensus.cc:482] T abfc8f390d434c47a8c4990f8406c270 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:39.650822 12537 raft_consensus.cc:504] T abfc8f390d434c47a8c4990f8406c270 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:39.653162 12537 leader_election.cc:288] T abfc8f390d434c47a8c4990f8406c270 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:39.654827 10813 tablet_service.cc:1514] Processing DeleteTablet for tablet 92814437d53143a79f99199f8ee3a742 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:29 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:39.609899 10386 catalog_manager.cc:5495] T 96a80382bb7d4395a14ad9ae972f4a25 P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: HEALTHY } } }
I20260925 09:57:39.659044 12322 raft_consensus.cc:482] T 3039ec3febf848dba7c4e6fd38bb8053 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:39.659562 10679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abfc8f390d434c47a8c4990f8406c270" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:39.659505 12322 raft_consensus.cc:504] T 3039ec3febf848dba7c4e6fd38bb8053 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:39.661399 11085 tablet_bootstrap.cc:654] T cf775c9d9b114ddcb97dc6c6eca8e0fc P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:39.662518 12322 leader_election.cc:288] T 3039ec3febf848dba7c4e6fd38bb8053 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:39.663628 11084 tablet_bootstrap.cc:654] T bbcfc4dca399496fa41105eff0c71c3b P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:39.664187 10919 leader_election.cc:341] T abfc8f390d434c47a8c4990f8406c270 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:39.676478 12289 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" instance_seqno: 1790330224889889) as {username='slave'} at 127.10.9.3:33067; Asking this server to re-register.
W20260925 09:57:39.677197 12289 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.3:33067 (request call id 3) took 1126 ms (client timeout 1000 ms). Trace:
I20260925 09:57:39.677636 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 362) took 4288 ms. Trace:
I20260925 09:57:39.677610 10833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3039ec3febf848dba7c4e6fd38bb8053" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:39.677551 12289 rpcz_store.cc:269] 0925 09:57:38.550364 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.550503 (+ 139us) service_pool.cc:227] Handling call
0925 09:57:39.653609 (+1103106us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b5400030554. stack: 00007fc7bb981033 00007fc7bb980f30 00007fc7bb502534 00000000004db2b4 00000000004db249 00007fc7c288daf0 00007fc7c2910f2a 00007fc7c47dcd6c 00007fc7c48d6c3e 00007fc7c240dfb4 00007fc7c240df42 00007fc7c240de71 00007fc7c240ddec 00007fc7c240d0a2 00007fc7c06b794c 00007fc7c06b6d86
0925 09:57:39.677123 (+ 23514us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2501784960}
W20260925 09:57:39.678998 10621 leader_election.cc:341] T 3039ec3febf848dba7c4e6fd38bb8053 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:39.679783 12298 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" instance_seqno: 1790330222809768) as {username='slave'} at 127.10.9.2:52201; Asking this server to re-register.
W20260925 09:57:39.680365 12298 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.2:52201 (request call id 5) took 2479 ms (client timeout 1000 ms). Trace:
W20260925 09:57:39.680645 12298 rpcz_store.cc:269] 0925 09:57:37.200548 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:37.200854 (+ 306us) service_pool.cc:227] Handling call
0925 09:57:39.658655 (+2457801us) spinlock_profiling.cc:243] Waited 2.45 s on lock 0x7b5400030554. stack: 00007fc7bb981033 00007fc7bb980f30 00007fc7bb502534 00000000004db2b4 00000000004db249 00007fc7c288daf0 00007fc7c2910f2a 00007fc7c47dcd6c 00007fc7c48d6c3e 00007fc7c240dfb4 00007fc7c240df42 00007fc7c240de71 00007fc7c240ddec 00007fc7c240d0a2 00007fc7c06b794c 00007fc7c06b6d86
0925 09:57:39.680308 (+ 21653us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":5634412928}
I20260925 09:57:39.677939 11727 rpcz_store.cc:276] 0925 09:57:35.389553 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:35.389997 (+ 444us) service_pool.cc:227] Handling call
0925 09:57:39.267113 (+3877116us) ts_tablet_manager.cc:1145] Deleting tablet 93e1b5837aff48d7907890c40dfcd0d0
0925 09:57:39.267153 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:39.505326 (+238173us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:39.612593 (+107267us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:39.677577 (+ 64984us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":223897,"tablet-delete.queue_time_us":3876562}
I20260925 09:57:39.681205 10824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abfc8f390d434c47a8c4990f8406c270" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:39.682143 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 364) took 4011 ms. Trace:
I20260925 09:57:39.682488 12287 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" instance_seqno: 1790330222809768) as {username='slave'} at 127.10.9.2:52201; Asking this server to re-register.
I20260925 09:57:39.682427 11727 rpcz_store.cc:276] 0925 09:57:35.671165 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:35.671302 (+ 137us) service_pool.cc:227] Handling call
0925 09:57:39.681876 (+4010574us) ts_tablet_manager.cc:1145] Deleting tablet 61aa6380d30a4b21b70ee2d321247bd8
0925 09:57:39.681918 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:39.682094 (+ 176us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4009941}
I20260925 09:57:39.684191 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 365) took 4012 ms. Trace:
I20260925 09:57:39.684502 11727 rpcz_store.cc:276] 0925 09:57:35.672051 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:35.672166 (+ 115us) service_pool.cc:227] Handling call
0925 09:57:39.683940 (+4011774us) ts_tablet_manager.cc:1145] Deleting tablet 92814437d53143a79f99199f8ee3a742
0925 09:57:39.683978 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:39.684143 (+ 165us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4011251}
W20260925 09:57:39.684955 10918 leader_election.cc:341] T abfc8f390d434c47a8c4990f8406c270 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:39.685448 10918 leader_election.cc:302] T abfc8f390d434c47a8c4990f8406c270 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:39.686049 12578 raft_consensus.cc:2726] T abfc8f390d434c47a8c4990f8406c270 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:39.683418 12300 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" instance_seqno: 1790330224889889) as {username='slave'} at 127.10.9.3:33067; Asking this server to re-register.
W20260925 09:57:39.686617 12300 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.3:33067 (request call id 1) took 2537 ms (client timeout 1000 ms). Trace:
W20260925 09:57:39.686918 12300 rpcz_store.cc:269] 0925 09:57:37.149397 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:37.149571 (+ 174us) service_pool.cc:227] Handling call
0925 09:57:39.652024 (+2502453us) spinlock_profiling.cc:243] Waited 2.45 s on lock 0x7b5400030554. stack: 00007fc7bb981033 00007fc7bb980f30 00007fc7bb502534 00000000004db2b4 00000000004db249 00007fc7c288daf0 00007fc7c2910f2a 00007fc7c47dcd6c 00007fc7c48d6c3e 00007fc7c240dfb4 00007fc7c240df42 00007fc7c240de71 00007fc7c240ddec 00007fc7c240d0a2 00007fc7c06b794c 00007fc7c06b6d86
0925 09:57:39.686555 (+ 34531us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":5635610368}
I20260925 09:57:39.681880 12288 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "b8850818e2ee42a082802d0095c08d21" instance_seqno: 1790330220617316) as {username='slave'} at 127.10.9.1:44421; Asking this server to re-register.
W20260925 09:57:39.688400 12288 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.1:44421 (request call id 2) took 860 ms (client timeout 1000 ms). Trace:
I20260925 09:57:39.688983 10885 heartbeater.cc:464] Registering TS with master...
W20260925 09:57:39.688685 12288 rpcz_store.cc:269] 0925 09:57:38.828341 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.828536 (+ 195us) service_pool.cc:227] Handling call
0925 09:57:39.660010 (+831474us) spinlock_profiling.cc:243] Waited 738 ms on lock 0x7b5400030554. stack: 00007fc7bb981033 00007fc7bb980f30 00007fc7bb502534 00000000004db2b4 00000000004db249 00007fc7c288daf0 00007fc7c2910f2a 00007fc7c47dcd6c 00007fc7c48d6c3e 00007fc7c240dfb4 00007fc7c240df42 00007fc7c240de71 00007fc7c240ddec 00007fc7c240d0a2 00007fc7c06b794c 00007fc7c06b6d86
0925 09:57:39.688349 (+ 28339us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1698405632}
I20260925 09:57:39.692463 10326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" candidate_term: 5 candidate_status { last_received { term: 4 index: 456 } } ignore_live_leader: false dest_uuid: "54c8485a7dcc47a1a75d08ef57647ece" is_pre_election: true
I20260925 09:57:39.691017 12462 leader_election.cc:288] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 54c8485a7dcc47a1a75d08ef57647ece (127.10.9.62:34455), 8af3f111e8da4d2dad4e4a98ef0fd5d3 (127.10.9.61:33047)
I20260925 09:57:39.693980 12297 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "b8850818e2ee42a082802d0095c08d21" instance_seqno: 1790330220617316) as {username='slave'} at 127.10.9.1:44421; Asking this server to re-register.
W20260925 09:57:39.694555 12297 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.1:44421 (request call id 1) took 2416 ms (client timeout 1000 ms). Trace:
W20260925 09:57:39.694857 12297 rpcz_store.cc:269] 0925 09:57:37.278641 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:37.283626 (+ 4985us) service_pool.cc:227] Handling call
0925 09:57:39.646109 (+2362483us) spinlock_profiling.cc:243] Waited 2.32 s on lock 0x7b5400030554. stack: 00007fc7bb981033 00007fc7bb980f30 00007fc7bb502534 00000000004db2b4 00000000004db249 00007fc7c288daf0 00007fc7c2910f2a 00007fc7c47dcd6c 00007fc7c48d6c3e 00007fc7c240dfb4 00007fc7c240df42 00007fc7c240de71 00007fc7c240ddec 00007fc7c240d0a2 00007fc7c06b794c 00007fc7c06b6d86
0925 09:57:39.694505 (+ 48396us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":5328802048}
I20260925 09:57:39.695338 10734 heartbeater.cc:464] Registering TS with master...
I20260925 09:57:39.695571 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a3149fb2325435699af36338e77d042" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:39.700210 10918 leader_election.cc:341] T 8a3149fb2325435699af36338e77d042 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:39.700332 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 366) took 4337 ms. Trace:
I20260925 09:57:39.700805 12316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40752 (request call id 0) took 2870 ms. Trace:
I20260925 09:57:39.700697 10918 leader_election.cc:302] T 8a3149fb2325435699af36338e77d042 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:39.703716 12578 raft_consensus.cc:2726] T 8a3149fb2325435699af36338e77d042 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:39.705238 10403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" candidate_term: 5 candidate_status { last_received { term: 4 index: 456 } } ignore_live_leader: false dest_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" is_pre_election: true
I20260925 09:57:39.707048 12198 leader_election.cc:302] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8be057e0b6d40a3a4b90f2b1f4a0fa2; no voters: 54c8485a7dcc47a1a75d08ef57647ece, 8af3f111e8da4d2dad4e4a98ef0fd5d3
I20260925 09:57:39.708395 12462 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260925 09:57:39.709446 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 93e1b5837aff48d7907890c40dfcd0d0 (table default.Table_f8b067e4e4cf498bbf44d7bf24c29363 [id=23d14ed1d2d647fc8b3b5ef40c7dc0bb]) successfully deleted
W20260925 09:57:39.710067 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 61aa6380d30a4b21b70ee2d321247bd8 because the tablet was not found. No further retry: Not found: Tablet not found: 61aa6380d30a4b21b70ee2d321247bd8
W20260925 09:57:39.710577 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 92814437d53143a79f99199f8ee3a742 because the tablet was not found. No further retry: Not found: Tablet not found: 92814437d53143a79f99199f8ee3a742
I20260925 09:57:39.700631 12027 rpcz_store.cc:276] 0925 09:57:35.363013 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:35.363149 (+ 136us) service_pool.cc:227] Handling call
0925 09:57:39.319656 (+3956507us) ts_tablet_manager.cc:1145] Deleting tablet 93e1b5837aff48d7907890c40dfcd0d0
0925 09:57:39.319704 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:39.493480 (+173776us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:39.637041 (+143561us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:39.700076 (+ 63035us) spinlock_profiling.cc:243] Waited 28.6 ms on lock 0x7b5800001898. stack: 00007f0270186033 00007f0270185f30 00007f026fd07534 00000000004db2b4 00000000004db249 00007f026fffcf11 00007f0270180851 00007f0270180e82 00007f02790408b9 00007f02780745d8 00007f0278138e42 00007f0278168e10 00007f027814fe0d 00007f027814fdb9 00007f027814fd49 00007f027814fd11
0925 09:57:39.700271 (+ 195us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":65769216,"tablet-delete.queue_time_us":3955891}
I20260925 09:57:39.711931 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 93e1b5837aff48d7907890c40dfcd0d0 (table default.Table_f8b067e4e4cf498bbf44d7bf24c29363 [id=23d14ed1d2d647fc8b3b5ef40c7dc0bb]) successfully deleted
I20260925 09:57:39.722391 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 366) took 4049 ms. Trace:
I20260925 09:57:39.723155 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 367) took 4287 ms. Trace:
I20260925 09:57:39.723588 11727 rpcz_store.cc:276] 0925 09:57:35.672853 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:35.672984 (+ 131us) service_pool.cc:227] Handling call
0925 09:57:39.685232 (+4012248us) ts_tablet_manager.cc:1145] Deleting tablet 417058493c6547c69e72347cef373292
0925 09:57:39.685271 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:39.722315 (+ 37044us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4011754}
I20260925 09:57:39.725001 11082 tablet_bootstrap.cc:700] T b39f9e7102ff4a9c96a2cc5d2676f03d P b8850818e2ee42a082802d0095c08d21: Time spent opening tablet: real 0.106s user 0.004s sys 0.000s
I20260925 09:57:39.725693 11082 tablet_bootstrap.cc:654] T b39f9e7102ff4a9c96a2cc5d2676f03d P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:39.730718 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 417058493c6547c69e72347cef373292 because the tablet was not found. No further retry: Not found: Tablet not found: 417058493c6547c69e72347cef373292
W20260925 09:57:39.731881 10619 leader_election.cc:341] T 3039ec3febf848dba7c4e6fd38bb8053 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 3039ec3febf848dba7c4e6fd38bb8053
I20260925 09:57:39.732296 10619 leader_election.cc:302] T 3039ec3febf848dba7c4e6fd38bb8053 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8850818e2ee42a082802d0095c08d21; no voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:39.734741 12390 consensus_queue.cc:1035] T 9744b9814b69470396ce039348fe5e45 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:39.735034 11085 tablet_bootstrap.cc:492] T cf775c9d9b114ddcb97dc6c6eca8e0fc P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:39.735445 11085 ts_tablet_manager.cc:1397] T cf775c9d9b114ddcb97dc6c6eca8e0fc P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.178s user 0.011s sys 0.001s
I20260925 09:57:39.735731 12491 raft_consensus.cc:2726] T 3039ec3febf848dba7c4e6fd38bb8053 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:39.724814 12027 rpcz_store.cc:276] 0925 09:57:35.435893 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:35.437353 (+ 1460us) service_pool.cc:227] Handling call
0925 09:57:39.722901 (+4285548us) ts_tablet_manager.cc:1145] Deleting tablet c7cd1d6a603642afa2c8c737079961b5
0925 09:57:39.722945 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:39.723100 (+ 155us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4284983}
I20260925 09:57:39.737586 11085 raft_consensus.cc:348] T cf775c9d9b114ddcb97dc6c6eca8e0fc P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:39.738219 11085 raft_consensus.cc:374] T cf775c9d9b114ddcb97dc6c6eca8e0fc P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:39.738451 11085 raft_consensus.cc:729] T cf775c9d9b114ddcb97dc6c6eca8e0fc P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:39.739060 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 367) took 3994 ms. Trace:
W20260925 09:57:39.740016 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet c7cd1d6a603642afa2c8c737079961b5 because the tablet was not found. No further retry: Not found: Tablet not found: c7cd1d6a603642afa2c8c737079961b5
I20260925 09:57:39.739369 11727 rpcz_store.cc:276] 0925 09:57:35.744313 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:35.744449 (+ 136us) service_pool.cc:227] Handling call
0925 09:57:39.725651 (+3981202us) ts_tablet_manager.cc:1145] Deleting tablet b555eeba505740988c704df5c734e7c0
0925 09:57:39.725708 (+ 57us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:39.738998 (+ 13290us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3980553}
I20260925 09:57:39.740908 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 368) took 3932 ms. Trace:
I20260925 09:57:39.741009 10836 raft_consensus.cc:3037] T ab6bdba5ca5d41ae921a90c1f77006c5 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:39.741160 11727 rpcz_store.cc:276] 0925 09:57:35.808498 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:35.808647 (+ 149us) service_pool.cc:227] Handling call
0925 09:57:39.740635 (+3931988us) ts_tablet_manager.cc:1145] Deleting tablet aba253f22afa4200926f31a1b43e3970
0925 09:57:39.740687 (+ 52us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:39.740862 (+ 175us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3931357}
I20260925 09:57:39.742085 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 369) took 3933 ms. Trace:
I20260925 09:57:39.742333 11727 rpcz_store.cc:276] 0925 09:57:35.809180 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:35.809334 (+ 154us) service_pool.cc:227] Handling call
0925 09:57:39.741856 (+3932522us) ts_tablet_manager.cc:1145] Deleting tablet 3ff4cf556eee494d8cd590884466f09f
0925 09:57:39.741894 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:39.742046 (+ 152us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3931885}
I20260925 09:57:39.748489 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 369) took 3706 ms. Trace:
I20260925 09:57:39.748864 12027 rpcz_store.cc:276] 0925 09:57:36.042225 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:36.047524 (+ 5299us) service_pool.cc:227] Handling call
0925 09:57:39.739857 (+3692333us) ts_tablet_manager.cc:1145] Deleting tablet 4c3e66f6ad23475f9fb831afc1d579c4
0925 09:57:39.748170 (+ 8313us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:39.748434 (+ 264us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3691777}
I20260925 09:57:39.751264 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 373) took 3197 ms. Trace:
I20260925 09:57:39.751628 12027 rpcz_store.cc:276] 0925 09:57:36.554144 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:36.559487 (+ 5343us) service_pool.cc:227] Handling call
0925 09:57:39.749641 (+3190154us) ts_tablet_manager.cc:1145] Deleting tablet d822e8ac13d9479aa9ce4f74ace55dd5
0925 09:57:39.750961 (+ 1320us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:39.751214 (+ 253us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3189639}
W20260925 09:57:39.753731 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 4c3e66f6ad23475f9fb831afc1d579c4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4c3e66f6ad23475f9fb831afc1d579c4 already in progress: creating tablet
W20260925 09:57:39.757157 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet b555eeba505740988c704df5c734e7c0 because the tablet was not found. No further retry: Not found: Tablet not found: b555eeba505740988c704df5c734e7c0
W20260925 09:57:39.757750 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet aba253f22afa4200926f31a1b43e3970 because the tablet was not found. No further retry: Not found: Tablet not found: aba253f22afa4200926f31a1b43e3970
W20260925 09:57:39.758255 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 3ff4cf556eee494d8cd590884466f09f because the tablet was not found. No further retry: Not found: Tablet not found: 3ff4cf556eee494d8cd590884466f09f
W20260925 09:57:39.733505 12529 log.cc:924] Time spent T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2: Append to log took a long time: real 0.094s user 0.004s sys 0.000s
I20260925 09:57:39.785513 12288 ts_manager.cc:194] Registered new tserver with Master: c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:39.805581 12300 ts_manager.cc:194] Registered new tserver with Master: b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:39.807858 12483 catalog_manager.cc:1259] Loaded cluster ID: 189c454748024b14abd819435a918f04
I20260925 09:57:39.808248 12483 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2: loading cluster ID for follower catalog manager: success
I20260925 09:57:39.816045 12492 raft_consensus.cc:482] T 05e2c7f9820f41679c4d58343ff3f829 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260925 09:57:39.816148 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet d822e8ac13d9479aa9ce4f74ace55dd5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d822e8ac13d9479aa9ce4f74ace55dd5 already in progress: creating tablet
I20260925 09:57:39.816478 12492 raft_consensus.cc:504] T 05e2c7f9820f41679c4d58343ff3f829 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:39.818269 12492 leader_election.cc:288] T 05e2c7f9820f41679c4d58343ff3f829 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:39.743235 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 370) took 3932 ms. Trace:
I20260925 09:57:39.819042 11727 rpcz_store.cc:276] 0925 09:57:35.811297 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:35.811535 (+ 238us) service_pool.cc:227] Handling call
0925 09:57:39.743004 (+3931469us) ts_tablet_manager.cc:1145] Deleting tablet cf775c9d9b114ddcb97dc6c6eca8e0fc
0925 09:57:39.743039 (+ 35us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:39.743194 (+ 155us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3930848}
I20260925 09:57:39.739008 11085 consensus_queue.cc:260] T cf775c9d9b114ddcb97dc6c6eca8e0fc P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:39.820889 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05e2c7f9820f41679c4d58343ff3f829" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:39.824051 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 372) took 4011 ms. Trace:
I20260925 09:57:39.824903 11085 ts_tablet_manager.cc:1428] T cf775c9d9b114ddcb97dc6c6eca8e0fc P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.089s user 0.008s sys 0.000s
W20260925 09:57:39.824863 10621 leader_election.cc:341] T 05e2c7f9820f41679c4d58343ff3f829 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:39.824673 11727 rpcz_store.cc:276] 0925 09:57:35.812957 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:35.813074 (+ 117us) service_pool.cc:227] Handling call
0925 09:57:39.823608 (+4010534us) ts_tablet_manager.cc:1145] Deleting tablet 9a73a1f242e7430cbf52229c07b4dc82
0925 09:57:39.823651 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:39.823996 (+ 345us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4009829}
I20260925 09:57:39.825714 11085 tablet_bootstrap.cc:492] T defaef4d6b754ca5a2dfc21b7a596a5b P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
W20260925 09:57:39.829471 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet cf775c9d9b114ddcb97dc6c6eca8e0fc because the tablet was not found. No further retry: Not found: Tablet not found: cf775c9d9b114ddcb97dc6c6eca8e0fc
I20260925 09:57:39.829855 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 375) took 3261 ms. Trace:
W20260925 09:57:39.830252 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 9a73a1f242e7430cbf52229c07b4dc82 because the tablet was not found. No further retry: Not found: Tablet not found: 9a73a1f242e7430cbf52229c07b4dc82
I20260925 09:57:39.830245 12027 rpcz_store.cc:276] 0925 09:57:36.568114 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:36.568273 (+ 159us) service_pool.cc:227] Handling call
0925 09:57:39.829365 (+3261092us) ts_tablet_manager.cc:1145] Deleting tablet 2b1b2d66397b44f38056077d79941ac4
0925 09:57:39.829410 (+ 45us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:39.829802 (+ 392us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3246818}
I20260925 09:57:39.831425 11084 tablet_bootstrap.cc:492] T bbcfc4dca399496fa41105eff0c71c3b P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:39.831871 11084 ts_tablet_manager.cc:1397] T bbcfc4dca399496fa41105eff0c71c3b P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.286s user 0.005s sys 0.007s
I20260925 09:57:39.831769 12027 tablet_replica.cc:330] T 5608c5f9b65b4da7b2fbb56417c92d88 P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:39.832345 10978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05e2c7f9820f41679c4d58343ff3f829" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:39.832638 12027 raft_consensus.cc:2229] T 5608c5f9b65b4da7b2fbb56417c92d88 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:39.832978 10978 raft_consensus.cc:2454] T 05e2c7f9820f41679c4d58343ff3f829 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:39.833194 12027 raft_consensus.cc:2258] T 5608c5f9b65b4da7b2fbb56417c92d88 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 09:57:39.833511 10379 catalog_manager.cc:4821] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 2b1b2d66397b44f38056077d79941ac4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2b1b2d66397b44f38056077d79941ac4 already in progress: creating tablet
I20260925 09:57:39.834115 11084 raft_consensus.cc:348] T bbcfc4dca399496fa41105eff0c71c3b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:39.834787 11084 raft_consensus.cc:374] T bbcfc4dca399496fa41105eff0c71c3b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:39.835021 11084 raft_consensus.cc:729] T bbcfc4dca399496fa41105eff0c71c3b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:39.833567 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 371) took 4021 ms. Trace:
I20260925 09:57:39.835569 11084 consensus_queue.cc:260] T bbcfc4dca399496fa41105eff0c71c3b P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:39.837023 11084 ts_tablet_manager.cc:1428] T bbcfc4dca399496fa41105eff0c71c3b P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260925 09:57:39.835623 11727 rpcz_store.cc:276] 0925 09:57:35.812637 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:35.813100 (+ 463us) service_pool.cc:227] Handling call
0925 09:57:39.825690 (+4012590us) ts_tablet_manager.cc:1145] Deleting tablet 37bab60afe7a4c758a08ef7c9c24aba0
0925 09:57:39.825749 (+ 59us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:39.833517 (+ 7768us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4011924}
I20260925 09:57:39.837205 10619 leader_election.cc:302] T 05e2c7f9820f41679c4d58343ff3f829 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:39.837814 11084 tablet_bootstrap.cc:492] T 25ac35fb43d34c9b85b8e61ad6e386a6 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:39.838037 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 373) took 3795 ms. Trace:
I20260925 09:57:39.835326 12027 ts_tablet_manager.cc:1905] T 5608c5f9b65b4da7b2fbb56417c92d88 P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 09:57:39.840041 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 37bab60afe7a4c758a08ef7c9c24aba0 because the tablet was not found. No further retry: Not found: Tablet not found: 37bab60afe7a4c758a08ef7c9c24aba0
I20260925 09:57:39.840864 11727 rpcz_store.cc:276] 0925 09:57:36.042145 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:36.042253 (+ 108us) service_pool.cc:227] Handling call
0925 09:57:39.837650 (+3795397us) ts_tablet_manager.cc:1145] Deleting tablet 4c3e66f6ad23475f9fb831afc1d579c4
0925 09:57:39.837692 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:39.837938 (+ 246us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3747604}
I20260925 09:57:39.841475 11085 tablet_bootstrap.cc:654] T defaef4d6b754ca5a2dfc21b7a596a5b P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:39.848140 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 376) took 3277 ms. Trace:
I20260925 09:57:39.851532 12322 raft_consensus.cc:482] T d54ab15dd054489ab50b7f0b26b79a50 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:39.851994 12322 raft_consensus.cc:504] T d54ab15dd054489ab50b7f0b26b79a50 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:39.840732 12491 raft_consensus.cc:2781] T 05e2c7f9820f41679c4d58343ff3f829 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:39.853227 12491 raft_consensus.cc:482] T 05e2c7f9820f41679c4d58343ff3f829 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:39.853613 12491 raft_consensus.cc:3037] T 05e2c7f9820f41679c4d58343ff3f829 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:39.853987 12322 leader_election.cc:288] T d54ab15dd054489ab50b7f0b26b79a50 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:39.855939 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d54ab15dd054489ab50b7f0b26b79a50" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:39.857259 10621 leader_election.cc:341] T d54ab15dd054489ab50b7f0b26b79a50 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:39.861996 11082 tablet_bootstrap.cc:492] T b39f9e7102ff4a9c96a2cc5d2676f03d P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:39.862545 11082 ts_tablet_manager.cc:1397] T b39f9e7102ff4a9c96a2cc5d2676f03d P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.248s user 0.010s sys 0.002s
I20260925 09:57:39.863263 11727 rpcz_store.cc:276] 0925 09:57:36.570345 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:36.570501 (+ 156us) service_pool.cc:227] Handling call
0925 09:57:39.847737 (+3277236us) ts_tablet_manager.cc:1145] Deleting tablet d822e8ac13d9479aa9ce4f74ace55dd5
0925 09:57:39.847788 (+ 51us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:39.848090 (+ 302us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3276521}
I20260925 09:57:39.864496 11727 tablet_replica.cc:330] T 2b1b2d66397b44f38056077d79941ac4 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
W20260925 09:57:39.866209 10377 catalog_manager.cc:4821] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 4c3e66f6ad23475f9fb831afc1d579c4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4c3e66f6ad23475f9fb831afc1d579c4 already in progress: creating tablet
W20260925 09:57:39.866902 10377 catalog_manager.cc:4821] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet d822e8ac13d9479aa9ce4f74ace55dd5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d822e8ac13d9479aa9ce4f74ace55dd5 already in progress: creating tablet
I20260925 09:57:39.868916 11085 tablet_bootstrap.cc:492] T defaef4d6b754ca5a2dfc21b7a596a5b P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:39.869450 11085 ts_tablet_manager.cc:1397] T defaef4d6b754ca5a2dfc21b7a596a5b P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.044s user 0.005s sys 0.004s
I20260925 09:57:39.878715 11085 raft_consensus.cc:348] T defaef4d6b754ca5a2dfc21b7a596a5b P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:39.884038 12529 log.cc:924] Time spent T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2: Append to log took a long time: real 0.081s user 0.003s sys 0.000s
I20260925 09:57:39.891490 11085 raft_consensus.cc:374] T defaef4d6b754ca5a2dfc21b7a596a5b P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:39.891903 11085 raft_consensus.cc:729] T defaef4d6b754ca5a2dfc21b7a596a5b P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:39.892549 11085 consensus_queue.cc:260] T defaef4d6b754ca5a2dfc21b7a596a5b P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:39.894508 11085 ts_tablet_manager.cc:1428] T defaef4d6b754ca5a2dfc21b7a596a5b P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.025s user 0.003s sys 0.000s
I20260925 09:57:39.895529 11084 tablet_bootstrap.cc:654] T 25ac35fb43d34c9b85b8e61ad6e386a6 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:39.896328 12523 raft_consensus.cc:482] T cf775c9d9b114ddcb97dc6c6eca8e0fc P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:39.896782 12523 raft_consensus.cc:504] T cf775c9d9b114ddcb97dc6c6eca8e0fc P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:39.898741 12523 leader_election.cc:288] T cf775c9d9b114ddcb97dc6c6eca8e0fc P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:39.901567 10679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf775c9d9b114ddcb97dc6c6eca8e0fc" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:39.904374 10775 leader_election.cc:341] T cf775c9d9b114ddcb97dc6c6eca8e0fc P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: Tablet not RUNNING: STOPPED
I20260925 09:57:39.905504 12491 raft_consensus.cc:504] T 05e2c7f9820f41679c4d58343ff3f829 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:39.907815 11085 tablet_bootstrap.cc:492] T 61aa6380d30a4b21b70ee2d321247bd8 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:39.912717 12491 leader_election.cc:288] T 05e2c7f9820f41679c4d58343ff3f829 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:39.919492 11084 tablet_bootstrap.cc:492] T 25ac35fb43d34c9b85b8e61ad6e386a6 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:39.919862 11084 ts_tablet_manager.cc:1397] T 25ac35fb43d34c9b85b8e61ad6e386a6 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.082s user 0.010s sys 0.000s
I20260925 09:57:39.920315 11727 raft_consensus.cc:2229] T 2b1b2d66397b44f38056077d79941ac4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:39.920826 11727 raft_consensus.cc:2258] T 2b1b2d66397b44f38056077d79941ac4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:39.922060 11084 raft_consensus.cc:348] T 25ac35fb43d34c9b85b8e61ad6e386a6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:39.922894 11084 raft_consensus.cc:374] T 25ac35fb43d34c9b85b8e61ad6e386a6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:39.923643 11727 ts_tablet_manager.cc:1905] T 2b1b2d66397b44f38056077d79941ac4 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:39.924414 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05e2c7f9820f41679c4d58343ff3f829" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
W20260925 09:57:39.926774 10621 leader_election.cc:341] T 05e2c7f9820f41679c4d58343ff3f829 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:39.932868 11085 tablet_bootstrap.cc:654] T 61aa6380d30a4b21b70ee2d321247bd8 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:39.935757 10978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d54ab15dd054489ab50b7f0b26b79a50" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:39.936309 10978 raft_consensus.cc:2454] T d54ab15dd054489ab50b7f0b26b79a50 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:39.938388 10978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05e2c7f9820f41679c4d58343ff3f829" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:39.938895 10978 raft_consensus.cc:3037] T 05e2c7f9820f41679c4d58343ff3f829 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:39.947883 12379 ts_tablet_manager.cc:1918] T cf775c9d9b114ddcb97dc6c6eca8e0fc P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:39.954003 12297 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" instance_seqno: 1790330224889889) as {username='slave'} at 127.10.9.3:33067; Asking this server to re-register.
I20260925 09:57:39.923285 11084 raft_consensus.cc:729] T 25ac35fb43d34c9b85b8e61ad6e386a6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:39.966385 11084 consensus_queue.cc:260] T 25ac35fb43d34c9b85b8e61ad6e386a6 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:39.968621 11084 ts_tablet_manager.cc:1428] T 25ac35fb43d34c9b85b8e61ad6e386a6 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.049s user 0.006s sys 0.000s
I20260925 09:57:39.968756 12379 log.cc:1196] T cf775c9d9b114ddcb97dc6c6eca8e0fc P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/cf775c9d9b114ddcb97dc6c6eca8e0fc
I20260925 09:57:39.969444 12379 ts_tablet_manager.cc:1939] T cf775c9d9b114ddcb97dc6c6eca8e0fc P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:39.949674 11082 raft_consensus.cc:348] T b39f9e7102ff4a9c96a2cc5d2676f03d P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:39.974709 10389 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 6540aba5854a44419addc5d454e473b7 (table default.Table_8180842f664042b0a9073b7008438ff7 [id=416a4b989570457e919fb7e7b16a017d]). Expected version 1 got 0
I20260925 09:57:39.978422 11084 tablet_bootstrap.cc:492] T d289c2fb23df499a8118de92d82f0d19 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:39.984028 11029 heartbeater.cc:464] Registering TS with master...
I20260925 09:57:39.985638 11727 ts_tablet_manager.cc:1918] T 2b1b2d66397b44f38056077d79941ac4 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:39.954358 10619 leader_election.cc:302] T d54ab15dd054489ab50b7f0b26b79a50 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:39.992412 12491 raft_consensus.cc:2781] T d54ab15dd054489ab50b7f0b26b79a50 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:39.992776 12491 raft_consensus.cc:482] T d54ab15dd054489ab50b7f0b26b79a50 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:39.993103 12491 raft_consensus.cc:3037] T d54ab15dd054489ab50b7f0b26b79a50 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:39.994777 10389 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 3c31682bf400494f82ee3ce5e6eec441 (table default.Table_82a8d1df379d442e9333fd4ab294738c [id=0040224e69114e918cec485791935895]). Expected version 1 got 0
I20260925 09:57:39.999435 10389 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 2f07dc9548364870923165f4ea524310 (table default.Table_0db927e92a3b4970a7efea3fb765e65e [id=a988264fe43c4eb88a84f8b6359e77a6]). Expected version 1 got 0
I20260925 09:57:40.008242 10389 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet c52745a8cdee461aba7485b0531245d6 (table default.Table_8e186a99ccba425b9b588041d2364a40 [id=2d6bc0e2aaee4b1483b01fc678505bb0]). Expected version 2 got 0
I20260925 09:57:40.009948 10389 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet cc295f7a9169475f9a9235e657be629f (table default.Table_168840855e0746f7ab027c1900842751 [id=99ba81c4d0dd497398e048a05759419a]). Expected version 1 got 0
I20260925 09:57:39.997141 11084 tablet_bootstrap.cc:654] T d289c2fb23df499a8118de92d82f0d19 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:40.010488 11727 log.cc:1196] T 2b1b2d66397b44f38056077d79941ac4 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/2b1b2d66397b44f38056077d79941ac4
I20260925 09:57:40.013374 11727 ts_tablet_manager.cc:1939] T 2b1b2d66397b44f38056077d79941ac4 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
W20260925 09:57:40.015058 10619 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 60784us
I20260925 09:57:40.022218 11084 tablet_bootstrap.cc:492] T d289c2fb23df499a8118de92d82f0d19 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:40.022640 11084 ts_tablet_manager.cc:1397] T d289c2fb23df499a8118de92d82f0d19 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.053s user 0.007s sys 0.002s
I20260925 09:57:40.024173 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 378) took 3435 ms. Trace:
W20260925 09:57:40.025314 10392 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 963e5bf2ac2848b4967074d28acdae52 consensus state has no leader
I20260925 09:57:40.025250 11727 rpcz_store.cc:276] 0925 09:57:36.580073 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:36.580213 (+ 140us) service_pool.cc:227] Handling call
0925 09:57:39.864349 (+3284136us) ts_tablet_manager.cc:1145] Deleting tablet 2b1b2d66397b44f38056077d79941ac4
0925 09:57:39.864391 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:39.965828 (+101437us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:39.985415 (+ 19587us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:40.015365 (+ 29950us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":54925,"tablet-delete.queue_time_us":3283501}
I20260925 09:57:40.025806 10392 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:35140 (request call id 381) took 3041 ms. Trace:
I20260925 09:57:40.026512 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 392) took 1533 ms. Trace:
I20260925 09:57:40.026800 11727 rpcz_store.cc:276] 0925 09:57:38.493171 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.493303 (+ 132us) service_pool.cc:227] Handling call
0925 09:57:40.026249 (+1532946us) ts_tablet_manager.cc:1145] Deleting tablet 69028aa7908c45b896da4ba8deef07b1
0925 09:57:40.026292 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:40.026451 (+ 159us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1532370}
I20260925 09:57:40.026077 10392 rpcz_store.cc:276] 0925 09:57:36.984714 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.971992 (+1987278us) service_pool.cc:227] Handling call
0925 09:57:39.161976 (+189984us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:57:39.163621 (+ 1645us) catalog_manager.cc:3616] Apply alter schema
0925 09:57:39.164209 (+ 588us) catalog_manager.cc:3639] Acquired catalog manager lock
0925 09:57:39.164488 (+ 279us) catalog_manager.cc:3786] Updating metadata on disk
0925 09:57:39.239509 (+ 75021us) catalog_manager.cc:3817] Committing alterations to in-memory state
0925 09:57:40.025755 (+786246us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:39.165676 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:39.166113 (+ 437us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:39.166139 (+ 26us) write_op.cc:435] Acquired schema lock
0925 09:57:39.166155 (+ 16us) tablet.cc:660] Decoding operations
0925 09:57:39.166466 (+ 311us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:39.166572 (+ 106us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:39.166603 (+ 31us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:39.166811 (+ 208us) tablet.cc:699] Row locks acquired
0925 09:57:39.166831 (+ 20us) write_op.cc:260] PREPARE: finished
0925 09:57:39.167016 (+ 185us) write_op.cc:270] Start()
0925 09:57:39.167116 (+ 100us) write_op.cc:276] Timestamp: P: 1790330259166925 usec, L: 0
0925 09:57:39.167137 (+ 21us) op_driver.cc:331] REPLICATION: starting
0925 09:57:39.167806 (+ 669us) log.cc:842] Serialized 405 byte log entry
0925 09:57:39.186391 (+ 18585us) op_driver.cc:444] REPLICATION: finished
0925 09:57:39.186727 (+ 336us) write_op.cc:301] APPLY: starting
0925 09:57:39.186776 (+ 49us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:39.186868 (+ 92us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:39.186974 (+ 106us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:39.187142 (+ 168us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:39.187285 (+ 143us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:39.187330 (+ 45us) write_op.cc:312] APPLY: finished
0925 09:57:39.187533 (+ 203us) log.cc:842] Serialized 23 byte log entry
0925 09:57:39.187680 (+ 147us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:39.187733 (+ 53us) write_op.cc:454] Released schema lock
0925 09:57:39.187762 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:39.187930 (+ 168us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:39.187958 (+ 28us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":197,"apply.run_cpu_time_us":1388,"apply.run_wall_time_us":1388,"num_ops":1,"prepare.queue_time_us":106,"prepare.run_cpu_time_us":2341,"prepare.run_wall_time_us":2431,"raft.queue_time_us":1047,"raft.run_cpu_time_us":755,"raft.run_wall_time_us":755,"replication_time_us":19171}]]}
I20260925 09:57:40.030642 10276 master-stress-test.cc:580] Killed test thread 5/10
I20260925 09:57:40.030897 10276 master-stress-test.cc:578] Killing test thread 6/10
I20260925 09:57:40.039852 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 393) took 1546 ms. Trace:
I20260925 09:57:40.040563 11082 raft_consensus.cc:374] T b39f9e7102ff4a9c96a2cc5d2676f03d P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:40.040262 11727 rpcz_store.cc:276] 0925 09:57:38.493431 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.493953 (+ 522us) service_pool.cc:227] Handling call
0925 09:57:40.027600 (+1533647us) ts_tablet_manager.cc:1145] Deleting tablet fb9c51f7e7fd4c029be2a710cb9462eb
0925 09:57:40.039536 (+ 11936us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:40.039779 (+ 243us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1533156}
I20260925 09:57:40.041327 11082 raft_consensus.cc:729] T b39f9e7102ff4a9c96a2cc5d2676f03d P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:40.041342 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 394) took 1547 ms. Trace:
I20260925 09:57:40.041694 11727 rpcz_store.cc:276] 0925 09:57:38.493679 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.496505 (+ 2826us) service_pool.cc:227] Handling call
0925 09:57:40.041107 (+1544602us) ts_tablet_manager.cc:1145] Deleting tablet c7cd1d6a603642afa2c8c737079961b5
0925 09:57:40.041147 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:40.041292 (+ 145us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1544128}
I20260925 09:57:40.042847 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 400) took 1508 ms. Trace:
I20260925 09:57:40.043133 11727 rpcz_store.cc:276] 0925 09:57:38.534094 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.546414 (+ 12320us) service_pool.cc:227] Handling call
0925 09:57:40.042605 (+1496191us) ts_tablet_manager.cc:1145] Deleting tablet dba14a44a9de492886d629441606767e
0925 09:57:40.042642 (+ 37us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:40.042803 (+ 161us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1495571}
I20260925 09:57:40.044137 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 398) took 1510 ms. Trace:
I20260925 09:57:40.044430 11727 rpcz_store.cc:276] 0925 09:57:38.533645 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.547086 (+ 13441us) service_pool.cc:227] Handling call
0925 09:57:40.043898 (+1496812us) ts_tablet_manager.cc:1145] Deleting tablet 3039ec3febf848dba7c4e6fd38bb8053
0925 09:57:40.043934 (+ 36us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:40.044094 (+ 160us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1496267}
I20260925 09:57:40.045418 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 397) took 1512 ms. Trace:
I20260925 09:57:40.045696 11727 rpcz_store.cc:276] 0925 09:57:38.533411 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.547709 (+ 14298us) service_pool.cc:227] Handling call
0925 09:57:40.045192 (+1497483us) ts_tablet_manager.cc:1145] Deleting tablet a81530200cf14717bc7cbf67bb73f4e5
0925 09:57:40.045229 (+ 37us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:40.045375 (+ 146us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1496892}
I20260925 09:57:40.046669 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 399) took 1512 ms. Trace:
I20260925 09:57:40.048563 12483 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2: acquiring CA information for follower catalog manager: success
I20260925 09:57:40.039285 11084 raft_consensus.cc:348] T d289c2fb23df499a8118de92d82f0d19 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.049113 11084 raft_consensus.cc:374] T d289c2fb23df499a8118de92d82f0d19 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:40.049420 11084 raft_consensus.cc:729] T d289c2fb23df499a8118de92d82f0d19 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:40.050076 11084 consensus_queue.cc:260] T d289c2fb23df499a8118de92d82f0d19 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.056679 11084 ts_tablet_manager.cc:1428] T d289c2fb23df499a8118de92d82f0d19 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.034s user 0.005s sys 0.003s
I20260925 09:57:40.060649 11084 tablet_bootstrap.cc:492] T d7459730813f486e9cb9aeffe29509a2 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:40.072395 12537 raft_consensus.cc:482] T 139fe14ba82a4137b59115b6d592a7c4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:40.071808 11082 consensus_queue.cc:260] T b39f9e7102ff4a9c96a2cc5d2676f03d P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.072846 12537 raft_consensus.cc:504] T 139fe14ba82a4137b59115b6d592a7c4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.073632 11082 ts_tablet_manager.cc:1428] T b39f9e7102ff4a9c96a2cc5d2676f03d P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.211s user 0.004s sys 0.000s
I20260925 09:57:40.074663 12537 leader_election.cc:288] T 139fe14ba82a4137b59115b6d592a7c4 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:40.047003 11727 rpcz_store.cc:276] 0925 09:57:38.533873 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.548375 (+ 14502us) service_pool.cc:227] Handling call
0925 09:57:40.046443 (+1498068us) ts_tablet_manager.cc:1145] Deleting tablet 197f61525b0c4f9da441d98701d231a3
0925 09:57:40.046480 (+ 37us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:40.046629 (+ 149us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1497538}
I20260925 09:57:40.076982 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 396) took 1543 ms. Trace:
I20260925 09:57:40.077296 11727 rpcz_store.cc:276] 0925 09:57:38.533182 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.535548 (+ 2366us) service_pool.cc:227] Handling call
0925 09:57:40.076688 (+1541140us) ts_tablet_manager.cc:1145] Deleting tablet 2d087d3e35814c90a03eccd90eeea86c
0925 09:57:40.076734 (+ 46us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:40.076929 (+ 195us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1527259}
I20260925 09:57:40.078359 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 401) took 1530 ms. Trace:
W20260925 09:57:40.084244 10380 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 963e5bf2ac2848b4967074d28acdae52 (table default.Table_257e35b439d7481fbcfcd5d50b6c6c08 [id=5dc3fd1666fb4431b1c3bbf27b02ba54], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 963e5bf2ac2848b4967074d28acdae52 consensus state has no leader
I20260925 09:57:40.088274 11084 tablet_bootstrap.cc:654] T d7459730813f486e9cb9aeffe29509a2 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:40.088390 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 2b1b2d66397b44f38056077d79941ac4 (table default.Table_94b396db46ba43d88015187a199247fc [id=78304acc330348238cdb35b6034337fd]) successfully deleted
W20260925 09:57:40.089275 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 69028aa7908c45b896da4ba8deef07b1 because the tablet was not found. No further retry: Not found: Tablet not found: 69028aa7908c45b896da4ba8deef07b1
W20260925 09:57:40.091756 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet fb9c51f7e7fd4c029be2a710cb9462eb because the tablet was not found. No further retry: Not found: Tablet not found: fb9c51f7e7fd4c029be2a710cb9462eb
I20260925 09:57:40.078640 11727 rpcz_store.cc:276] 0925 09:57:38.547835 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.549512 (+ 1677us) service_pool.cc:227] Handling call
0925 09:57:40.078113 (+1528601us) ts_tablet_manager.cc:1145] Deleting tablet 350904aa5957496d90a2aa2e637b591c
0925 09:57:40.078152 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:40.078312 (+ 160us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1528045}
I20260925 09:57:40.094117 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 402) took 1546 ms. Trace:
I20260925 09:57:40.094432 11727 rpcz_store.cc:276] 0925 09:57:38.553926 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.576590 (+ 22664us) service_pool.cc:227] Handling call
0925 09:57:40.093854 (+1517264us) ts_tablet_manager.cc:1145] Deleting tablet 1e76f9e0ba554e1cad0cf18863393608
0925 09:57:40.093898 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:40.094066 (+ 168us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1516663}
I20260925 09:57:40.095324 11727 tablet_replica.cc:330] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:40.096093 11727 raft_consensus.cc:2229] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:40.100533 11727 raft_consensus.cc:2258] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:40.102401 11727 ts_tablet_manager.cc:1905] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:40.124290 11084 tablet_bootstrap.cc:492] T d7459730813f486e9cb9aeffe29509a2 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:40.124744 11084 ts_tablet_manager.cc:1397] T d7459730813f486e9cb9aeffe29509a2 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.064s user 0.006s sys 0.004s
W20260925 09:57:40.128530 10621 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 71625.5us
I20260925 09:57:40.127058 11084 raft_consensus.cc:348] T d7459730813f486e9cb9aeffe29509a2 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.134570 11084 raft_consensus.cc:374] T d7459730813f486e9cb9aeffe29509a2 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:40.134859 11084 raft_consensus.cc:729] T d7459730813f486e9cb9aeffe29509a2 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:40.135460 11084 consensus_queue.cc:260] T d7459730813f486e9cb9aeffe29509a2 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.144183 10679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "139fe14ba82a4137b59115b6d592a7c4" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:40.146670 10919 leader_election.cc:341] T 139fe14ba82a4137b59115b6d592a7c4 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:40.147882 11084 ts_tablet_manager.cc:1428] T d7459730813f486e9cb9aeffe29509a2 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.023s user 0.006s sys 0.000s
I20260925 09:57:40.148797 11084 tablet_bootstrap.cc:492] T e9faf70be8ac43c79305980d685a28eb P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:40.149737 12379 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 418) took 1480 ms. Trace:
I20260925 09:57:40.150138 12379 rpcz_store.cc:276] 0925 09:57:38.669413 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.669507 (+ 94us) service_pool.cc:227] Handling call
0925 09:57:39.582742 (+913235us) ts_tablet_manager.cc:1145] Deleting tablet cf775c9d9b114ddcb97dc6c6eca8e0fc
0925 09:57:39.582785 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:39.765566 (+182781us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:39.947584 (+182018us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:40.149680 (+202096us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":28934,"tablet-delete.queue_time_us":912691}
I20260925 09:57:40.153487 10978 raft_consensus.cc:2454] T 05e2c7f9820f41679c4d58343ff3f829 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:40.153520 12483 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260925 09:57:40.168916 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet c7cd1d6a603642afa2c8c737079961b5 because the tablet was not found. No further retry: Not found: Tablet not found: c7cd1d6a603642afa2c8c737079961b5
W20260925 09:57:40.170099 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet dba14a44a9de492886d629441606767e because the tablet was not found. No further retry: Not found: Tablet not found: dba14a44a9de492886d629441606767e
W20260925 09:57:40.170948 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 3039ec3febf848dba7c4e6fd38bb8053 because the tablet was not found. No further retry: Not found: Tablet not found: 3039ec3febf848dba7c4e6fd38bb8053
I20260925 09:57:40.171450 11727 ts_tablet_manager.cc:1918] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:57:40.171862 11727 log.cc:1196] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/5608c5f9b65b4da7b2fbb56417c92d88
W20260925 09:57:40.171875 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet a81530200cf14717bc7cbf67bb73f4e5 because the tablet was not found. No further retry: Not found: Tablet not found: a81530200cf14717bc7cbf67bb73f4e5
I20260925 09:57:40.172592 11727 ts_tablet_manager.cc:1939] T 5608c5f9b65b4da7b2fbb56417c92d88 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:40.174520 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 395) took 1641 ms. Trace:
W20260925 09:57:40.183869 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 197f61525b0c4f9da441d98701d231a3 because the tablet was not found. No further retry: Not found: Tablet not found: 197f61525b0c4f9da441d98701d231a3
W20260925 09:57:40.185134 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 2d087d3e35814c90a03eccd90eeea86c because the tablet was not found. No further retry: Not found: Tablet not found: 2d087d3e35814c90a03eccd90eeea86c
W20260925 09:57:40.187186 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 350904aa5957496d90a2aa2e637b591c because the tablet was not found. No further retry: Not found: Tablet not found: 350904aa5957496d90a2aa2e637b591c
W20260925 09:57:40.190127 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 1e76f9e0ba554e1cad0cf18863393608 because the tablet was not found. No further retry: Not found: Tablet not found: 1e76f9e0ba554e1cad0cf18863393608
I20260925 09:57:40.192440 12537 raft_consensus.cc:482] T 25ac35fb43d34c9b85b8e61ad6e386a6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:40.193240 12578 raft_consensus.cc:482] T bbcfc4dca399496fa41105eff0c71c3b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:40.192967 12537 raft_consensus.cc:504] T 25ac35fb43d34c9b85b8e61ad6e386a6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.193636 12578 raft_consensus.cc:504] T bbcfc4dca399496fa41105eff0c71c3b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.195333 12578 leader_election.cc:288] T bbcfc4dca399496fa41105eff0c71c3b P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:40.174839 11727 rpcz_store.cc:276] 0925 09:57:38.532940 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.536307 (+ 3367us) service_pool.cc:227] Handling call
0925 09:57:40.095184 (+1558877us) ts_tablet_manager.cc:1145] Deleting tablet 5608c5f9b65b4da7b2fbb56417c92d88
0925 09:57:40.095220 (+ 36us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:40.140007 (+ 44787us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:40.171193 (+ 31186us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:40.174463 (+ 3270us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1515121}
I20260925 09:57:40.197060 11084 tablet_bootstrap.cc:654] T e9faf70be8ac43c79305980d685a28eb P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:40.198136 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 5608c5f9b65b4da7b2fbb56417c92d88 (table default.Table_f37bcee5db2044f2bea9c353982b30cd [id=e46d114e5d484b66bb15eb0416b5ce0d]) successfully deleted
I20260925 09:57:40.198513 12537 leader_election.cc:288] T 25ac35fb43d34c9b85b8e61ad6e386a6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:40.197558 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 403) took 1460 ms. Trace:
I20260925 09:57:40.199774 11727 rpcz_store.cc:276] 0925 09:57:38.737016 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.737126 (+ 110us) service_pool.cc:227] Handling call
0925 09:57:40.197311 (+1460185us) ts_tablet_manager.cc:1145] Deleting tablet 10657a81058241fc988d37e5ea7ff8cf
0925 09:57:40.197355 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:40.197510 (+ 155us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1459008}
I20260925 09:57:40.200766 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 404) took 1433 ms. Trace:
I20260925 09:57:40.201035 11727 rpcz_store.cc:276] 0925 09:57:38.767241 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.767700 (+ 459us) service_pool.cc:227] Handling call
0925 09:57:40.200530 (+1432830us) ts_tablet_manager.cc:1145] Deleting tablet 3be0eccc65564bfba3d6e5c398c61fb6
0925 09:57:40.200567 (+ 37us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:40.200722 (+ 155us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1432249}
I20260925 09:57:40.201944 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 405) took 1433 ms. Trace:
I20260925 09:57:40.202209 11727 rpcz_store.cc:276] 0925 09:57:38.768936 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.769837 (+ 901us) service_pool.cc:227] Handling call
0925 09:57:40.201725 (+1431888us) ts_tablet_manager.cc:1145] Deleting tablet 67ce2e3325214aa6bdc3347e9e8f82b0
0925 09:57:40.201760 (+ 35us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:40.201897 (+ 137us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1431382}
I20260925 09:57:40.203084 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 406) took 1414 ms. Trace:
W20260925 09:57:40.203707 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 10657a81058241fc988d37e5ea7ff8cf because the tablet was not found. No further retry: Not found: Tablet not found: 10657a81058241fc988d37e5ea7ff8cf
W20260925 09:57:40.204596 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 3be0eccc65564bfba3d6e5c398c61fb6 because the tablet was not found. No further retry: Not found: Tablet not found: 3be0eccc65564bfba3d6e5c398c61fb6
W20260925 09:57:40.205564 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 67ce2e3325214aa6bdc3347e9e8f82b0 because the tablet was not found. No further retry: Not found: Tablet not found: 67ce2e3325214aa6bdc3347e9e8f82b0
W20260925 09:57:40.173265 10772 leader_election.cc:341] T cf775c9d9b114ddcb97dc6c6eca8e0fc P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: cf775c9d9b114ddcb97dc6c6eca8e0fc
I20260925 09:57:39.935331 12027 ts_tablet_manager.cc:1918] T 5608c5f9b65b4da7b2fbb56417c92d88 P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:57:40.215241 12027 log.cc:1196] T 5608c5f9b65b4da7b2fbb56417c92d88 P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/5608c5f9b65b4da7b2fbb56417c92d88
I20260925 09:57:40.215137 10772 leader_election.cc:302] T cf775c9d9b114ddcb97dc6c6eca8e0fc P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:39.993871 11085 tablet_bootstrap.cc:492] T 61aa6380d30a4b21b70ee2d321247bd8 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:40.215889 12523 raft_consensus.cc:2726] T cf775c9d9b114ddcb97dc6c6eca8e0fc P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:40.216068 12027 ts_tablet_manager.cc:1939] T 5608c5f9b65b4da7b2fbb56417c92d88 P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:40.216334 11085 ts_tablet_manager.cc:1397] T 61aa6380d30a4b21b70ee2d321247bd8 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.309s user 0.030s sys 0.018s
I20260925 09:57:40.061499 12506 raft_consensus.cc:482] T 545ac025f2cb469da847979011a0b3ce P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b8850818e2ee42a082802d0095c08d21)
W20260925 09:57:40.216820 10772 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 43600.3us
I20260925 09:57:40.217028 12506 raft_consensus.cc:504] T 545ac025f2cb469da847979011a0b3ce P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.218044 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25ac35fb43d34c9b85b8e61ad6e386a6" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:40.218835 11085 raft_consensus.cc:348] T 61aa6380d30a4b21b70ee2d321247bd8 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.219239 12506 leader_election.cc:288] T 545ac025f2cb469da847979011a0b3ce P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:40.219419 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbcfc4dca399496fa41105eff0c71c3b" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:40.219530 11085 raft_consensus.cc:374] T 61aa6380d30a4b21b70ee2d321247bd8 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:40.077054 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "139fe14ba82a4137b59115b6d592a7c4" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:40.112505 12555 raft_consensus.cc:482] T defaef4d6b754ca5a2dfc21b7a596a5b P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:40.221055 12288 ts_manager.cc:194] Registered new tserver with Master: be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:40.221081 12555 raft_consensus.cc:504] T defaef4d6b754ca5a2dfc21b7a596a5b P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.224126 12555 leader_election.cc:288] T defaef4d6b754ca5a2dfc21b7a596a5b P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:40.225265 12379 tablet_replica.cc:330] T 92814437d53143a79f99199f8ee3a742 P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:40.168916 11082 tablet_bootstrap.cc:492] T f9c259867cf44c4c90e50864475a4970 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:40.203337 11727 rpcz_store.cc:276] 0925 09:57:38.788474 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.788600 (+ 126us) service_pool.cc:227] Handling call
0925 09:57:40.202870 (+1414270us) ts_tablet_manager.cc:1145] Deleting tablet cfa619b699004882a82976f0ef8204ad
0925 09:57:40.202903 (+ 33us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:40.203042 (+ 139us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1413695}
W20260925 09:57:40.232116 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet cfa619b699004882a82976f0ef8204ad because the tablet was not found. No further retry: Not found: Tablet not found: cfa619b699004882a82976f0ef8204ad
W20260925 09:57:40.234237 10918 leader_election.cc:341] T 25ac35fb43d34c9b85b8e61ad6e386a6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:40.235162 10918 leader_election.cc:341] T 139fe14ba82a4137b59115b6d592a7c4 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:40.235512 10918 leader_election.cc:302] T 139fe14ba82a4137b59115b6d592a7c4 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:40.236562 12537 raft_consensus.cc:2726] T 139fe14ba82a4137b59115b6d592a7c4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:40.014572 12491 raft_consensus.cc:504] T d54ab15dd054489ab50b7f0b26b79a50 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.237771 11727 tablet_replica.cc:330] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:40.238617 12491 leader_election.cc:288] T d54ab15dd054489ab50b7f0b26b79a50 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:40.239487 10978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "defaef4d6b754ca5a2dfc21b7a596a5b" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:40.246134 10945 tablet_service.cc:1467] Processing CreateTablet for tablet 3a34a167bb944f03b35e2e2d86248aca (DEFAULT_TABLE table=default.Table_f323f1cef50d4ecfbedb14f38b396eac [id=4096f2f208b246a8ad535b868fcbb48a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:40.247198 10945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a34a167bb944f03b35e2e2d86248aca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:40.255678 12537 raft_consensus.cc:482] T dbc7c7b9a2ee437eabe10150bdbc2e4a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:40.256213 12537 raft_consensus.cc:504] T dbc7c7b9a2ee437eabe10150bdbc2e4a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.236887 10813 tablet_service.cc:1467] Processing CreateTablet for tablet 3a34a167bb944f03b35e2e2d86248aca (DEFAULT_TABLE table=default.Table_f323f1cef50d4ecfbedb14f38b396eac [id=4096f2f208b246a8ad535b868fcbb48a]), partition=RANGE (key) PARTITION UNBOUNDED
W20260925 09:57:40.238620 10772 leader_election.cc:341] T 545ac025f2cb469da847979011a0b3ce P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 545ac025f2cb469da847979011a0b3ce
I20260925 09:57:40.219857 11085 raft_consensus.cc:729] T 61aa6380d30a4b21b70ee2d321247bd8 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:40.259806 12537 leader_election.cc:288] T dbc7c7b9a2ee437eabe10150bdbc2e4a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:40.263171 10831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbc7c7b9a2ee437eabe10150bdbc2e4a" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:40.026163 10836 raft_consensus.cc:3037] T 96a80382bb7d4395a14ad9ae972f4a25 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:40.252111 10806 tablet_service.cc:1467] Processing CreateTablet for tablet a7ab16337cbd4a46a5e472d3741fb0db (DEFAULT_TABLE table=default.Table_41a628305794438b90de11e177341305 [id=cb6a70931c3a4d2588e9f19978a43188]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:40.263460 11085 consensus_queue.cc:260] T 61aa6380d30a4b21b70ee2d321247bd8 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.265820 11085 ts_tablet_manager.cc:1428] T 61aa6380d30a4b21b70ee2d321247bd8 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.049s user 0.006s sys 0.000s
I20260925 09:57:40.267272 11084 tablet_bootstrap.cc:492] T e9faf70be8ac43c79305980d685a28eb P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:40.267832 11084 ts_tablet_manager.cc:1397] T e9faf70be8ac43c79305980d685a28eb P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.119s user 0.008s sys 0.002s
W20260925 09:57:40.268400 10918 leader_election.cc:341] T dbc7c7b9a2ee437eabe10150bdbc2e4a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:40.270437 11084 raft_consensus.cc:348] T e9faf70be8ac43c79305980d685a28eb P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.271142 11084 raft_consensus.cc:374] T e9faf70be8ac43c79305980d685a28eb P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:40.271413 11084 raft_consensus.cc:729] T e9faf70be8ac43c79305980d685a28eb P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:40.272051 11084 consensus_queue.cc:260] T e9faf70be8ac43c79305980d685a28eb P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.273619 11084 ts_tablet_manager.cc:1428] T e9faf70be8ac43c79305980d685a28eb P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
W20260925 09:57:40.274140 10918 leader_election.cc:341] T bbcfc4dca399496fa41105eff0c71c3b P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:40.277138 10813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a34a167bb944f03b35e2e2d86248aca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:40.277726 10952 tablet_service.cc:1467] Processing CreateTablet for tablet a7ab16337cbd4a46a5e472d3741fb0db (DEFAULT_TABLE table=default.Table_41a628305794438b90de11e177341305 [id=cb6a70931c3a4d2588e9f19978a43188]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:40.279450 12379 raft_consensus.cc:2229] T 92814437d53143a79f99199f8ee3a742 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:40.280016 12379 raft_consensus.cc:2258] T 92814437d53143a79f99199f8ee3a742 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:40.282241 12379 ts_tablet_manager.cc:1905] T 92814437d53143a79f99199f8ee3a742 P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:40.289505 10812 tablet_service.cc:1467] Processing CreateTablet for tablet 5f531882bbaa4433954cd45ec98baaab (DEFAULT_TABLE table=default.Table_b5303e844fd045a0b85dc5b75c932806 [id=d9bdad50730e406aa7429d151031b313]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:40.291778 10946 tablet_service.cc:1467] Processing CreateTablet for tablet 5f531882bbaa4433954cd45ec98baaab (DEFAULT_TABLE table=default.Table_b5303e844fd045a0b85dc5b75c932806 [id=d9bdad50730e406aa7429d151031b313]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:40.292168 10806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7ab16337cbd4a46a5e472d3741fb0db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:40.292987 10809 tablet_service.cc:1467] Processing CreateTablet for tablet 4e0df1d8cf81469692b946b3c2a50caf (DEFAULT_TABLE table=default.Table_bcbc82519cf043738f61029181f29418 [id=b3f7f21ab2214e4cb637bea27ce62abe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:40.293849 10809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e0df1d8cf81469692b946b3c2a50caf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:40.301139 10812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f531882bbaa4433954cd45ec98baaab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:40.301290 11085 tablet_bootstrap.cc:492] T f62c62715c9b42de826c1a66fd431eab P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:40.304801 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 388) took 1686 ms. Trace:
I20260925 09:57:40.311484 12027 rpcz_store.cc:276] 0925 09:57:38.618390 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.619050 (+ 660us) service_pool.cc:227] Handling call
0925 09:57:39.831551 (+1212501us) ts_tablet_manager.cc:1145] Deleting tablet 5608c5f9b65b4da7b2fbb56417c92d88
0925 09:57:39.831600 (+ 49us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:39.928331 (+ 96731us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:39.935121 (+ 6790us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:40.304738 (+369617us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1212010}
I20260925 09:57:40.312901 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 5608c5f9b65b4da7b2fbb56417c92d88 (table default.Table_f37bcee5db2044f2bea9c353982b30cd [id=e46d114e5d484b66bb15eb0416b5ce0d]) successfully deleted
I20260925 09:57:40.313917 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 389) took 1616 ms. Trace:
I20260925 09:57:40.321307 11727 raft_consensus.cc:2229] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:40.321988 11727 raft_consensus.cc:2258] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:40.323918 11727 ts_tablet_manager.cc:1905] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:40.351967 11082 tablet_bootstrap.cc:654] T f9c259867cf44c4c90e50864475a4970 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:40.353688 10956 tablet_service.cc:1467] Processing CreateTablet for tablet 4e0df1d8cf81469692b946b3c2a50caf (DEFAULT_TABLE table=default.Table_bcbc82519cf043738f61029181f29418 [id=b3f7f21ab2214e4cb637bea27ce62abe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:40.354446 12537 raft_consensus.cc:482] T 88e53debe6de43e29b12e85e5171c8b5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:40.354612 11084 tablet_bootstrap.cc:492] T 086fc96ca247461db2ecd864139a228c P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:40.314268 12027 rpcz_store.cc:276] 0925 09:57:38.697501 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.697843 (+ 342us) service_pool.cc:227] Handling call
0925 09:57:40.313648 (+1615805us) ts_tablet_manager.cc:1145] Deleting tablet 417058493c6547c69e72347cef373292
0925 09:57:40.313689 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:40.313871 (+ 182us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1615238}
I20260925 09:57:40.354879 12537 raft_consensus.cc:504] T 88e53debe6de43e29b12e85e5171c8b5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.356290 12027 tablet_replica.cc:330] T 4e3120347c8e4a3eab57d6562ae40e8d P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:40.356797 12537 leader_election.cc:288] T 88e53debe6de43e29b12e85e5171c8b5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:40.357569 12027 raft_consensus.cc:2229] T 4e3120347c8e4a3eab57d6562ae40e8d P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Raft consensus shutting down.
W20260925 09:57:40.358178 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 417058493c6547c69e72347cef373292 because the tablet was not found. No further retry: Not found: Tablet not found: 417058493c6547c69e72347cef373292
I20260925 09:57:40.358304 12027 pending_rounds.cc:62] T 4e3120347c8e4a3eab57d6562ae40e8d P c43461c4ee854ae1ac52b873ccada318: Trying to abort 1 pending ops.
I20260925 09:57:40.358587 12027 pending_rounds.cc:68] T 4e3120347c8e4a3eab57d6562ae40e8d P c43461c4ee854ae1ac52b873ccada318: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192725521563648 op_type: NO_OP noop_request { }
I20260925 09:57:40.359262 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88e53debe6de43e29b12e85e5171c8b5" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:40.359540 10956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e0df1d8cf81469692b946b3c2a50caf. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:57:40.362207 10918 leader_election.cc:341] T 88e53debe6de43e29b12e85e5171c8b5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:40.362730 12027 raft_consensus.cc:2866] T 4e3120347c8e4a3eab57d6562ae40e8d P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:57:40.362948 10946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f531882bbaa4433954cd45ec98baaab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:40.398847 11084 tablet_bootstrap.cc:654] T 086fc96ca247461db2ecd864139a228c P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:40.307232 11085 tablet_bootstrap.cc:654] T f62c62715c9b42de826c1a66fd431eab P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:40.383723 12603 raft_consensus.cc:482] T 96a80382bb7d4395a14ad9ae972f4a25 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b8850818e2ee42a082802d0095c08d21)
I20260925 09:57:40.399886 12027 raft_consensus.cc:2258] T 4e3120347c8e4a3eab57d6562ae40e8d P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:40.402120 12027 ts_tablet_manager.cc:1905] T 4e3120347c8e4a3eab57d6562ae40e8d P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:40.400043 12603 raft_consensus.cc:504] T 96a80382bb7d4395a14ad9ae972f4a25 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.407791 10978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96a80382bb7d4395a14ad9ae972f4a25" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:40.408416 12603 leader_election.cc:288] T 96a80382bb7d4395a14ad9ae972f4a25 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:40.413403 11727 ts_tablet_manager.cc:1918] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:40.414464 12608 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8af3f111e8da4d2dad4e4a98ef0fd5d3)
I20260925 09:57:40.417085 11727 log.cc:1196] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/17bd30416fe14c0e9ab2e0c0bf5f2fb5
I20260925 09:57:40.417832 11727 ts_tablet_manager.cc:1939] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:40.420533 11085 tablet_bootstrap.cc:492] T f62c62715c9b42de826c1a66fd431eab P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:40.420994 11085 ts_tablet_manager.cc:1397] T f62c62715c9b42de826c1a66fd431eab P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.120s user 0.007s sys 0.002s
I20260925 09:57:40.424430 11085 raft_consensus.cc:348] T f62c62715c9b42de826c1a66fd431eab P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.425223 11085 raft_consensus.cc:374] T f62c62715c9b42de826c1a66fd431eab P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:40.440534 12608 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } }
I20260925 09:57:40.425490 11085 raft_consensus.cc:729] T f62c62715c9b42de826c1a66fd431eab P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:40.445006 11085 consensus_queue.cc:260] T f62c62715c9b42de826c1a66fd431eab P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.449105 11085 ts_tablet_manager.cc:1428] T f62c62715c9b42de826c1a66fd431eab P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.028s user 0.008s sys 0.000s
I20260925 09:57:40.449977 11085 tablet_bootstrap.cc:492] T aca5987b14b64152887ac5ee584b257a P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:40.456380 10326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" candidate_term: 5 candidate_status { last_received { term: 4 index: 456 } } ignore_live_leader: false dest_uuid: "54c8485a7dcc47a1a75d08ef57647ece" is_pre_election: true
I20260925 09:57:40.461585 11084 tablet_bootstrap.cc:492] T 086fc96ca247461db2ecd864139a228c P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:40.462097 11084 ts_tablet_manager.cc:1397] T 086fc96ca247461db2ecd864139a228c P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.108s user 0.011s sys 0.001s
I20260925 09:57:40.463519 12608 leader_election.cc:288] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 54c8485a7dcc47a1a75d08ef57647ece (127.10.9.62:34455), 8af3f111e8da4d2dad4e4a98ef0fd5d3 (127.10.9.61:33047)
I20260925 09:57:40.463686 10403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" candidate_term: 5 candidate_status { last_received { term: 4 index: 456 } } ignore_live_leader: false dest_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" is_pre_election: true
I20260925 09:57:40.464993 12198 leader_election.cc:302] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8be057e0b6d40a3a4b90f2b1f4a0fa2; no voters: 54c8485a7dcc47a1a75d08ef57647ece, 8af3f111e8da4d2dad4e4a98ef0fd5d3
I20260925 09:57:40.466419 11084 raft_consensus.cc:348] T 086fc96ca247461db2ecd864139a228c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.467090 11084 raft_consensus.cc:374] T 086fc96ca247461db2ecd864139a228c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:40.467329 11084 raft_consensus.cc:729] T 086fc96ca247461db2ecd864139a228c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:40.470826 10952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7ab16337cbd4a46a5e472d3741fb0db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:40.477762 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 407) took 1688 ms. Trace:
I20260925 09:57:40.478083 11727 rpcz_store.cc:276] 0925 09:57:38.789775 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.789908 (+ 133us) service_pool.cc:227] Handling call
0925 09:57:40.237628 (+1447720us) ts_tablet_manager.cc:1145] Deleting tablet 17bd30416fe14c0e9ab2e0c0bf5f2fb5
0925 09:57:40.237669 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:40.332277 (+ 94608us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:40.413168 (+ 80891us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:40.477708 (+ 64540us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":82536,"tablet-delete.queue_time_us":1447219}
I20260925 09:57:40.479167 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 17bd30416fe14c0e9ab2e0c0bf5f2fb5 (table default.Table_4688c2ec003a44218fedb9f17dc0f5e3 [id=0e904cb2f51c411aa5fbdde8907a695d]) successfully deleted
I20260925 09:57:40.479532 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 409) took 1679 ms. Trace:
I20260925 09:57:40.479787 11727 rpcz_store.cc:276] 0925 09:57:38.800589 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.801593 (+ 1004us) service_pool.cc:227] Handling call
0925 09:57:40.479265 (+1677672us) ts_tablet_manager.cc:1145] Deleting tablet 4e3120347c8e4a3eab57d6562ae40e8d
0925 09:57:40.479304 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:40.479489 (+ 185us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1677122}
W20260925 09:57:40.480713 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 4e3120347c8e4a3eab57d6562ae40e8d because the tablet was not found. No further retry: Not found: Tablet not found: 4e3120347c8e4a3eab57d6562ae40e8d
I20260925 09:57:40.480917 11727 tablet_replica.cc:330] T abfc8f390d434c47a8c4990f8406c270 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:40.481904 11727 raft_consensus.cc:2229] T abfc8f390d434c47a8c4990f8406c270 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:40.482548 11727 raft_consensus.cc:2258] T abfc8f390d434c47a8c4990f8406c270 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:40.489071 11084 consensus_queue.cc:260] T 086fc96ca247461db2ecd864139a228c P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.495347 11727 ts_tablet_manager.cc:1905] T abfc8f390d434c47a8c4990f8406c270 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:40.496343 12608 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260925 09:57:40.509644 11084 ts_tablet_manager.cc:1428] T 086fc96ca247461db2ecd864139a228c P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.047s user 0.006s sys 0.000s
I20260925 09:57:40.510499 11084 tablet_bootstrap.cc:492] T 572c83b4a7c84fdab1fdbb0fbdb0a53d P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:40.513341 11085 tablet_bootstrap.cc:654] T aca5987b14b64152887ac5ee584b257a P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:40.535490 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d54ab15dd054489ab50b7f0b26b79a50" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:40.536018 10976 raft_consensus.cc:3037] T d54ab15dd054489ab50b7f0b26b79a50 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:40.542047 11084 tablet_bootstrap.cc:654] T 572c83b4a7c84fdab1fdbb0fbdb0a53d P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:40.543994 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d54ab15dd054489ab50b7f0b26b79a50" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:40.198060 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25ac35fb43d34c9b85b8e61ad6e386a6" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:40.196339 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbcfc4dca399496fa41105eff0c71c3b" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:40.569176 10976 raft_consensus.cc:2454] T d54ab15dd054489ab50b7f0b26b79a50 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:40.576462 12596 raft_consensus.cc:482] T b39f9e7102ff4a9c96a2cc5d2676f03d P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:40.577311 12596 raft_consensus.cc:504] T b39f9e7102ff4a9c96a2cc5d2676f03d P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.580856 12596 leader_election.cc:288] T b39f9e7102ff4a9c96a2cc5d2676f03d P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:40.598594 11082 tablet_bootstrap.cc:492] T f9c259867cf44c4c90e50864475a4970 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:40.599186 11082 ts_tablet_manager.cc:1397] T f9c259867cf44c4c90e50864475a4970 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.430s user 0.008s sys 0.002s
I20260925 09:57:40.599771 11084 tablet_bootstrap.cc:492] T 572c83b4a7c84fdab1fdbb0fbdb0a53d P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:40.600198 11084 ts_tablet_manager.cc:1397] T 572c83b4a7c84fdab1fdbb0fbdb0a53d P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.090s user 0.006s sys 0.003s
I20260925 09:57:40.602005 11082 raft_consensus.cc:348] T f9c259867cf44c4c90e50864475a4970 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.602671 11082 raft_consensus.cc:374] T f9c259867cf44c4c90e50864475a4970 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:40.603000 11082 raft_consensus.cc:729] T f9c259867cf44c4c90e50864475a4970 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:40.603745 11082 consensus_queue.cc:260] T f9c259867cf44c4c90e50864475a4970 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.606411 11082 ts_tablet_manager.cc:1428] T f9c259867cf44c4c90e50864475a4970 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.007s user 0.004s sys 0.001s
I20260925 09:57:40.618772 11084 raft_consensus.cc:348] T 572c83b4a7c84fdab1fdbb0fbdb0a53d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.619484 11084 raft_consensus.cc:374] T 572c83b4a7c84fdab1fdbb0fbdb0a53d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:40.654460 11085 tablet_bootstrap.cc:492] T aca5987b14b64152887ac5ee584b257a P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:40.654870 11085 ts_tablet_manager.cc:1397] T aca5987b14b64152887ac5ee584b257a P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.205s user 0.009s sys 0.005s
I20260925 09:57:40.657024 11085 raft_consensus.cc:348] T aca5987b14b64152887ac5ee584b257a P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.657640 11085 raft_consensus.cc:374] T aca5987b14b64152887ac5ee584b257a P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:40.657872 11085 raft_consensus.cc:729] T aca5987b14b64152887ac5ee584b257a P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:40.658403 11085 consensus_queue.cc:260] T aca5987b14b64152887ac5ee584b257a P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.668497 12027 ts_tablet_manager.cc:1918] T 4e3120347c8e4a3eab57d6562ae40e8d P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:40.619724 11084 raft_consensus.cc:729] T 572c83b4a7c84fdab1fdbb0fbdb0a53d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:40.682108 11084 consensus_queue.cc:260] T 572c83b4a7c84fdab1fdbb0fbdb0a53d P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:40.683467 10919 leader_election.cc:341] T 25ac35fb43d34c9b85b8e61ad6e386a6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:40.683140 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet cf775c9d9b114ddcb97dc6c6eca8e0fc (table default.Table_a749aff6c09e4425afc67db6a98196bd [id=9236e950b8d24547ba2ae47fabf2de6e]) successfully deleted
I20260925 09:57:40.683895 10919 leader_election.cc:302] T 25ac35fb43d34c9b85b8e61ad6e386a6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:40.684348 11084 ts_tablet_manager.cc:1428] T 572c83b4a7c84fdab1fdbb0fbdb0a53d P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.084s user 0.004s sys 0.003s
I20260925 09:57:40.684473 12537 raft_consensus.cc:2726] T 25ac35fb43d34c9b85b8e61ad6e386a6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:40.685133 11727 ts_tablet_manager.cc:1918] T abfc8f390d434c47a8c4990f8406c270 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260925 09:57:40.685241 10919 leader_election.cc:341] T bbcfc4dca399496fa41105eff0c71c3b P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:40.685475 11727 log.cc:1196] T abfc8f390d434c47a8c4990f8406c270 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/abfc8f390d434c47a8c4990f8406c270
I20260925 09:57:40.685613 10919 leader_election.cc:302] T bbcfc4dca399496fa41105eff0c71c3b P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:40.686108 11727 ts_tablet_manager.cc:1939] T abfc8f390d434c47a8c4990f8406c270 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:40.686185 12537 raft_consensus.cc:2726] T bbcfc4dca399496fa41105eff0c71c3b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:40.690088 10668 tablet_service.cc:1467] Processing CreateTablet for tablet 3a34a167bb944f03b35e2e2d86248aca (DEFAULT_TABLE table=default.Table_f323f1cef50d4ecfbedb14f38b396eac [id=4096f2f208b246a8ad535b868fcbb48a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:40.690789 10658 tablet_service.cc:1467] Processing CreateTablet for tablet 4e0df1d8cf81469692b946b3c2a50caf (DEFAULT_TABLE table=default.Table_bcbc82519cf043738f61029181f29418 [id=b3f7f21ab2214e4cb637bea27ce62abe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:40.691653 10654 tablet_service.cc:1467] Processing CreateTablet for tablet a7ab16337cbd4a46a5e472d3741fb0db (DEFAULT_TABLE table=default.Table_41a628305794438b90de11e177341305 [id=cb6a70931c3a4d2588e9f19978a43188]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:40.693128 10653 tablet_service.cc:1467] Processing CreateTablet for tablet 5f531882bbaa4433954cd45ec98baaab (DEFAULT_TABLE table=default.Table_b5303e844fd045a0b85dc5b75c932806 [id=d9bdad50730e406aa7429d151031b313]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:40.693383 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbc7c7b9a2ee437eabe10150bdbc2e4a" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:40.695952 10679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88e53debe6de43e29b12e85e5171c8b5" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:40.704037 11084 tablet_bootstrap.cc:492] T 2015c10f503e4751ad9ff77404fca50b P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:40.708807 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 410) took 1863 ms. Trace:
I20260925 09:57:40.709112 11727 rpcz_store.cc:276] 0925 09:57:38.845600 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.846044 (+ 444us) service_pool.cc:227] Handling call
0925 09:57:40.480782 (+1634738us) ts_tablet_manager.cc:1145] Deleting tablet abfc8f390d434c47a8c4990f8406c270
0925 09:57:40.480817 (+ 35us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:40.545402 (+ 64585us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:40.684923 (+139521us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:40.708751 (+ 23828us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1634233}
I20260925 09:57:40.710340 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet abfc8f390d434c47a8c4990f8406c270 (table default.Table_2702858b51514e7f9a51a2bf19d36b1f [id=65b6e9ec3f114cea85e70f1509e3d1b6]) successfully deleted
I20260925 09:57:40.711179 11727 tablet_replica.cc:330] T b3d9a3cf5bda4cf6bc1fa3fae1cb9288 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:40.712139 10668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a34a167bb944f03b35e2e2d86248aca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:40.712421 11082 tablet_bootstrap.cc:492] T 3d84c717b6ac462eb331e25d80c918e4 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:40.713299 11727 raft_consensus.cc:2229] T b3d9a3cf5bda4cf6bc1fa3fae1cb9288 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:40.713708 11727 raft_consensus.cc:2258] T b3d9a3cf5bda4cf6bc1fa3fae1cb9288 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:40.715884 11727 ts_tablet_manager.cc:1905] T b3d9a3cf5bda4cf6bc1fa3fae1cb9288 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:40.715945 10658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e0df1d8cf81469692b946b3c2a50caf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:40.719805 10654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7ab16337cbd4a46a5e472d3741fb0db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:40.723856 10653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f531882bbaa4433954cd45ec98baaab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:40.729791 12537 raft_consensus.cc:482] T e9faf70be8ac43c79305980d685a28eb P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:40.730237 12537 raft_consensus.cc:504] T e9faf70be8ac43c79305980d685a28eb P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.732053 12537 leader_election.cc:288] T e9faf70be8ac43c79305980d685a28eb P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:40.697503 11085 ts_tablet_manager.cc:1428] T aca5987b14b64152887ac5ee584b257a P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.042s user 0.006s sys 0.001s
I20260925 09:57:40.734778 12537 raft_consensus.cc:482] T f3bba45d6b7241f9a07907c79b892f7d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:40.733047 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9faf70be8ac43c79305980d685a28eb" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:40.735509 11085 tablet_bootstrap.cc:492] T 350904aa5957496d90a2aa2e637b591c P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
W20260925 09:57:40.736325 10918 leader_election.cc:341] T e9faf70be8ac43c79305980d685a28eb P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:40.735212 12537 raft_consensus.cc:504] T f3bba45d6b7241f9a07907c79b892f7d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.739044 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3bba45d6b7241f9a07907c79b892f7d" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:40.740367 10918 leader_election.cc:341] T f3bba45d6b7241f9a07907c79b892f7d P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:40.740973 11085 tablet_bootstrap.cc:654] T 350904aa5957496d90a2aa2e637b591c P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:40.741262 12537 leader_election.cc:288] T f3bba45d6b7241f9a07907c79b892f7d P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:40.742719 10679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9faf70be8ac43c79305980d685a28eb" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:40.742977 10621 leader_election.cc:341] T d54ab15dd054489ab50b7f0b26b79a50 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:40.743796 12027 log.cc:1196] T 4e3120347c8e4a3eab57d6562ae40e8d P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/4e3120347c8e4a3eab57d6562ae40e8d
I20260925 09:57:40.744979 10687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "defaef4d6b754ca5a2dfc21b7a596a5b" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:40.747243 12027 ts_tablet_manager.cc:1939] T 4e3120347c8e4a3eab57d6562ae40e8d P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:40.747227 10679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3bba45d6b7241f9a07907c79b892f7d" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:40.750475 11082 tablet_bootstrap.cc:654] T 3d84c717b6ac462eb331e25d80c918e4 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:40.750438 10919 leader_election.cc:341] T dbc7c7b9a2ee437eabe10150bdbc2e4a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:40.750790 10919 leader_election.cc:302] T dbc7c7b9a2ee437eabe10150bdbc2e4a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:40.752702 12523 raft_consensus.cc:482] T f62c62715c9b42de826c1a66fd431eab P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:40.753129 12523 raft_consensus.cc:504] T f62c62715c9b42de826c1a66fd431eab P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.754945 12523 leader_election.cc:288] T f62c62715c9b42de826c1a66fd431eab P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:40.752193 12537 raft_consensus.cc:2726] T dbc7c7b9a2ee437eabe10150bdbc2e4a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:40.755841 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f62c62715c9b42de826c1a66fd431eab" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:40.759070 10619 leader_election.cc:302] T 05e2c7f9820f41679c4d58343ff3f829 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:40.760193 12615 raft_consensus.cc:2781] T 05e2c7f9820f41679c4d58343ff3f829 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:40.762635 12615 raft_consensus.cc:686] T 05e2c7f9820f41679c4d58343ff3f829 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:40.763871 10619 leader_election.cc:302] T d54ab15dd054489ab50b7f0b26b79a50 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters: c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:40.766959 11084 tablet_bootstrap.cc:654] T 2015c10f503e4751ad9ff77404fca50b P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:40.767179 12615 consensus_queue.cc:237] T 05e2c7f9820f41679c4d58343ff3f829 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.772619 12612 raft_consensus.cc:2781] T d54ab15dd054489ab50b7f0b26b79a50 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:40.773104 12612 raft_consensus.cc:686] T d54ab15dd054489ab50b7f0b26b79a50 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:40.780041 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 390) took 2082 ms. Trace:
I20260925 09:57:40.780356 12027 rpcz_store.cc:276] 0925 09:57:38.697749 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.699529 (+ 1780us) service_pool.cc:227] Handling call
0925 09:57:40.356137 (+1656608us) ts_tablet_manager.cc:1145] Deleting tablet 4e3120347c8e4a3eab57d6562ae40e8d
0925 09:57:40.356178 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:40.523921 (+167743us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:40.668277 (+144356us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:40.779981 (+111704us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1656078}
I20260925 09:57:40.774152 12612 consensus_queue.cc:237] T d54ab15dd054489ab50b7f0b26b79a50 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.781623 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 4e3120347c8e4a3eab57d6562ae40e8d (table default.Table_8e186a99ccba425b9b588041d2364a40 [id=2d6bc0e2aaee4b1483b01fc678505bb0]) successfully deleted
I20260925 09:57:40.781999 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b39f9e7102ff4a9c96a2cc5d2676f03d" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:40.781512 11082 tablet_bootstrap.cc:492] T 3d84c717b6ac462eb331e25d80c918e4 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:40.782549 10976 raft_consensus.cc:2454] T b39f9e7102ff4a9c96a2cc5d2676f03d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:40.782696 11082 ts_tablet_manager.cc:1397] T 3d84c717b6ac462eb331e25d80c918e4 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.071s user 0.009s sys 0.002s
I20260925 09:57:40.786629 11082 raft_consensus.cc:348] T 3d84c717b6ac462eb331e25d80c918e4 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:40.791991 10919 leader_election.cc:341] T 88e53debe6de43e29b12e85e5171c8b5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:40.792435 10919 leader_election.cc:302] T 88e53debe6de43e29b12e85e5171c8b5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:40.793942 12537 raft_consensus.cc:2726] T 88e53debe6de43e29b12e85e5171c8b5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:40.787222 11082 raft_consensus.cc:374] T 3d84c717b6ac462eb331e25d80c918e4 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:40.801321 11082 raft_consensus.cc:729] T 3d84c717b6ac462eb331e25d80c918e4 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:40.802433 10385 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet f0bf545f70ae4cd6a1ad2540c7158ecc (table default.Table_25edfe533c64421da7700ac5d60265c3 [id=1469f44b239d454caa97056bd5803255]). Expected version 1 got 0
I20260925 09:57:40.802611 11082 consensus_queue.cc:260] T 3d84c717b6ac462eb331e25d80c918e4 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:40.805233 10919 leader_election.cc:341] T e9faf70be8ac43c79305980d685a28eb P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:40.805737 10919 leader_election.cc:302] T e9faf70be8ac43c79305980d685a28eb P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:40.807103 12621 raft_consensus.cc:2726] T e9faf70be8ac43c79305980d685a28eb P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:40.808990 10919 leader_election.cc:341] T f3bba45d6b7241f9a07907c79b892f7d P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:40.809392 10919 leader_election.cc:302] T f3bba45d6b7241f9a07907c79b892f7d P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:40.813268 12537 raft_consensus.cc:2726] T f3bba45d6b7241f9a07907c79b892f7d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:40.815672 11082 ts_tablet_manager.cc:1428] T 3d84c717b6ac462eb331e25d80c918e4 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.031s user 0.006s sys 0.000s
I20260925 09:57:40.816691 11082 tablet_bootstrap.cc:492] T abfc8f390d434c47a8c4990f8406c270 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:40.821782 11085 tablet_bootstrap.cc:492] T 350904aa5957496d90a2aa2e637b591c P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:40.822206 11085 ts_tablet_manager.cc:1397] T 350904aa5957496d90a2aa2e637b591c P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.087s user 0.006s sys 0.002s
I20260925 09:57:40.824327 11085 raft_consensus.cc:348] T 350904aa5957496d90a2aa2e637b591c P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.824985 11085 raft_consensus.cc:374] T 350904aa5957496d90a2aa2e637b591c P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:40.825217 11085 raft_consensus.cc:729] T 350904aa5957496d90a2aa2e637b591c P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:40.825656 11084 tablet_bootstrap.cc:492] T 2015c10f503e4751ad9ff77404fca50b P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:40.826025 11084 ts_tablet_manager.cc:1397] T 2015c10f503e4751ad9ff77404fca50b P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.122s user 0.006s sys 0.003s
I20260925 09:57:40.825770 11085 consensus_queue.cc:260] T 350904aa5957496d90a2aa2e637b591c P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.827673 11085 ts_tablet_manager.cc:1428] T 350904aa5957496d90a2aa2e637b591c P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260925 09:57:40.827978 11084 raft_consensus.cc:348] T 2015c10f503e4751ad9ff77404fca50b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.828542 11084 raft_consensus.cc:374] T 2015c10f503e4751ad9ff77404fca50b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:40.828785 11084 raft_consensus.cc:729] T 2015c10f503e4751ad9ff77404fca50b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:40.830476 10385 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet c52745a8cdee461aba7485b0531245d6 (table default.Table_8e186a99ccba425b9b588041d2364a40 [id=2d6bc0e2aaee4b1483b01fc678505bb0]). Expected version 2 got 0
I20260925 09:57:40.831777 10385 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet cc295f7a9169475f9a9235e657be629f (table default.Table_168840855e0746f7ab027c1900842751 [id=99ba81c4d0dd497398e048a05759419a]). Expected version 1 got 0
I20260925 09:57:40.833659 10393 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet a8be01fe42894b96bbdd5ce2403ff1f4
I20260925 09:57:40.834858 10393 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:35140 (request call id 380) took 4063 ms. Trace:
I20260925 09:57:40.838905 10952 tablet_service.cc:1514] Processing DeleteTablet for tablet a8be01fe42894b96bbdd5ce2403ff1f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:39 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:40.839780 10812 tablet_service.cc:1514] Processing DeleteTablet for tablet a8be01fe42894b96bbdd5ce2403ff1f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:39 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:40.835137 10393 rpcz_store.cc:276] 0925 09:57:36.771817 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.892874 (+2121057us) service_pool.cc:227] Handling call
0925 09:57:39.166141 (+273267us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:57:39.166740 (+ 599us) catalog_manager.cc:2752] Modifying in-memory table state
0925 09:57:39.166856 (+ 116us) catalog_manager.cc:2761] Locking tablets
0925 09:57:39.167450 (+ 594us) catalog_manager.cc:2775] Removing table and tablets from system table
0925 09:57:39.296302 (+128852us) catalog_manager.cc:2793] Removing table from by-name map
0925 09:57:39.525959 (+229657us) catalog_manager.cc:2809] Committing in-memory state
0925 09:57:40.833575 (+1307616us) catalog_manager.cc:2813] Aborting table tasks
0925 09:57:40.834803 (+ 1228us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:39.232875 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:39.233320 (+ 445us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:39.233346 (+ 26us) write_op.cc:435] Acquired schema lock
0925 09:57:39.233360 (+ 14us) tablet.cc:660] Decoding operations
0925 09:57:39.233727 (+ 367us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:39.233766 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:39.233800 (+ 34us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:39.234112 (+ 312us) tablet.cc:699] Row locks acquired
0925 09:57:39.234124 (+ 12us) write_op.cc:260] PREPARE: finished
0925 09:57:39.234258 (+ 134us) write_op.cc:270] Start()
0925 09:57:39.234353 (+ 95us) write_op.cc:276] Timestamp: P: 1790330259234201 usec, L: 0
0925 09:57:39.234372 (+ 19us) op_driver.cc:331] REPLICATION: starting
0925 09:57:39.234899 (+ 527us) log.cc:842] Serialized 756 byte log entry
0925 09:57:39.261057 (+ 26158us) op_driver.cc:444] REPLICATION: finished
0925 09:57:39.281046 (+ 19989us) write_op.cc:301] APPLY: starting
0925 09:57:39.281104 (+ 58us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:39.281209 (+ 105us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:39.281308 (+ 99us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:39.292112 (+ 10804us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:39.292243 (+ 131us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 09:57:39.292298 (+ 55us) write_op.cc:312] APPLY: finished
0925 09:57:39.292473 (+ 175us) log.cc:842] Serialized 29 byte log entry
0925 09:57:39.292601 (+ 128us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:39.292668 (+ 67us) write_op.cc:454] Released schema lock
0925 09:57:39.292700 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:39.299878 (+ 7178us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:39.300091 (+ 213us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":19816,"apply.run_cpu_time_us":1828,"apply.run_wall_time_us":20555,"num_ops":2,"prepare.queue_time_us":124,"prepare.run_cpu_time_us":2239,"prepare.run_wall_time_us":3326,"replication_time_us":22167,"thread_start_us":27278,"threads_started":1}]]}
I20260925 09:57:40.844122 10390 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:35140 (request call id 388) took 1651 ms. Trace:
I20260925 09:57:40.844420 10390 rpcz_store.cc:276] 0925 09:57:39.193036 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:39.473305 (+280269us) service_pool.cc:227] Handling call
0925 09:57:39.520480 (+ 47175us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:57:39.521122 (+ 642us) catalog_manager.cc:3616] Apply alter schema
0925 09:57:39.521641 (+ 519us) catalog_manager.cc:3639] Acquired catalog manager lock
0925 09:57:39.521855 (+ 214us) catalog_manager.cc:3786] Updating metadata on disk
0925 09:57:39.587822 (+ 65967us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:39.587858 (+ 36us) write_op.cc:454] Released schema lock
0925 09:57:39.588082 (+ 224us) catalog_manager.cc:3817] Committing alterations to in-memory state
0925 09:57:40.844068 (+1255986us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:39.561625 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:39.562087 (+ 462us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:39.562134 (+ 47us) write_op.cc:435] Acquired schema lock
0925 09:57:39.562145 (+ 11us) tablet.cc:660] Decoding operations
0925 09:57:39.562443 (+ 298us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:39.562506 (+ 63us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:39.562531 (+ 25us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:39.562683 (+ 152us) tablet.cc:699] Row locks acquired
0925 09:57:39.562695 (+ 12us) write_op.cc:260] PREPARE: finished
0925 09:57:39.562826 (+ 131us) write_op.cc:270] Start()
0925 09:57:39.562911 (+ 85us) write_op.cc:276] Timestamp: P: 1790330259562747 usec, L: 0
0925 09:57:39.562925 (+ 14us) op_driver.cc:331] REPLICATION: starting
0925 09:57:39.564173 (+ 1248us) log.cc:842] Serialized 405 byte log entry
0925 09:57:39.575471 (+ 11298us) op_driver.cc:444] REPLICATION: finished
0925 09:57:39.575933 (+ 462us) write_op.cc:301] APPLY: starting
0925 09:57:39.575980 (+ 47us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:39.576069 (+ 89us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:39.576152 (+ 83us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:39.576292 (+ 140us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:39.576410 (+ 118us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:39.576450 (+ 40us) write_op.cc:312] APPLY: finished
0925 09:57:39.576635 (+ 185us) log.cc:842] Serialized 23 byte log entry
0925 09:57:39.576787 (+ 152us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:39.576850 (+ 63us) write_op.cc:454] Released schema lock
0925 09:57:39.576879 (+ 29us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":329,"apply.run_cpu_time_us":1115,"apply.run_wall_time_us":1114,"num_ops":1,"prepare.queue_time_us":38678,"prepare.run_cpu_time_us":2162,"prepare.run_wall_time_us":2835,"raft.queue_time_us":2283,"raft.run_cpu_time_us":684,"raft.run_wall_time_us":684,"replication_time_us":12473}]]}
I20260925 09:57:40.848201 10276 master-stress-test.cc:580] Killed test thread 6/10
I20260925 09:57:40.848456 10276 master-stress-test.cc:578] Killing test thread 7/10
I20260925 09:57:40.854719 11085 tablet_bootstrap.cc:492] T 9744b9814b69470396ce039348fe5e45 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:40.859674 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 391) took 2159 ms. Trace:
I20260925 09:57:40.860186 12027 rpcz_store.cc:276] 0925 09:57:38.700462 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.701560 (+ 1098us) service_pool.cc:227] Handling call
0925 09:57:40.831626 (+2130066us) ts_tablet_manager.cc:1145] Deleting tablet 6161774f61ae48ab9764a05f2188a80b
0925 09:57:40.859437 (+ 27811us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:40.859613 (+ 176us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2129550}
I20260925 09:57:40.861158 12027 tablet_replica.cc:330] T 545ac025f2cb469da847979011a0b3ce P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
W20260925 09:57:40.861820 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet 6161774f61ae48ab9764a05f2188a80b because the tablet was not found. No further retry: Not found: Tablet not found: 6161774f61ae48ab9764a05f2188a80b
I20260925 09:57:40.861987 12027 raft_consensus.cc:2229] T 545ac025f2cb469da847979011a0b3ce P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:40.862555 12027 raft_consensus.cc:2258] T 545ac025f2cb469da847979011a0b3ce P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:40.864431 12027 ts_tablet_manager.cc:1905] T 545ac025f2cb469da847979011a0b3ce P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:40.862406 11084 consensus_queue.cc:260] T 2015c10f503e4751ad9ff77404fca50b P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.871347 10681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96a80382bb7d4395a14ad9ae972f4a25" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:40.872534 10619 leader_election.cc:302] T b39f9e7102ff4a9c96a2cc5d2676f03d P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:40.887528 12614 raft_consensus.cc:482] T 61aa6380d30a4b21b70ee2d321247bd8 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:40.887984 12614 raft_consensus.cc:504] T 61aa6380d30a4b21b70ee2d321247bd8 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.874161 10654 tablet_service.cc:1514] Processing DeleteTablet for tablet a8be01fe42894b96bbdd5ce2403ff1f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:39 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:40.898782 12379 ts_tablet_manager.cc:1918] T 92814437d53143a79f99199f8ee3a742 P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:40.899204 12379 log.cc:1196] T 92814437d53143a79f99199f8ee3a742 P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/92814437d53143a79f99199f8ee3a742
I20260925 09:57:40.900138 12379 ts_tablet_manager.cc:1939] T 92814437d53143a79f99199f8ee3a742 P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
I20260925 09:57:40.901064 12614 leader_election.cc:288] T 61aa6380d30a4b21b70ee2d321247bd8 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:40.902890 12379 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 419) took 2232 ms. Trace:
I20260925 09:57:40.903263 12379 rpcz_store.cc:276] 0925 09:57:38.670803 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.671235 (+ 432us) service_pool.cc:227] Handling call
0925 09:57:40.225061 (+1553826us) ts_tablet_manager.cc:1145] Deleting tablet 92814437d53143a79f99199f8ee3a742
0925 09:57:40.225120 (+ 59us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:40.569292 (+344172us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:40.820901 (+251609us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:40.902805 (+ 81904us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":53293,"tablet-delete.queue_time_us":1553292}
I20260925 09:57:40.904414 11085 tablet_bootstrap.cc:654] T 9744b9814b69470396ce039348fe5e45 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:40.905652 11082 tablet_bootstrap.cc:654] T abfc8f390d434c47a8c4990f8406c270 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:40.905746 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet 92814437d53143a79f99199f8ee3a742 (table default.Table_755c3887714845d2bd8bb30bd444adef [id=d8fb4efa035549159e9b06af8723d715]) successfully deleted
I20260925 09:57:40.909507 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f62c62715c9b42de826c1a66fd431eab" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:40.907238 12615 raft_consensus.cc:2781] T b39f9e7102ff4a9c96a2cc5d2676f03d P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:40.920328 12615 raft_consensus.cc:482] T b39f9e7102ff4a9c96a2cc5d2676f03d P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:40.921458 12615 raft_consensus.cc:3037] T b39f9e7102ff4a9c96a2cc5d2676f03d P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:40.931707 10836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b39f9e7102ff4a9c96a2cc5d2676f03d" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:40.932453 12379 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 416) took 2264 ms. Trace:
I20260925 09:57:40.932816 12379 rpcz_store.cc:276] 0925 09:57:38.668098 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.669533 (+ 1435us) service_pool.cc:227] Handling call
0925 09:57:40.919245 (+2249712us) ts_tablet_manager.cc:1145] Deleting tablet a21909e2e5794f57971f4e42795235c7
0925 09:57:40.920076 (+ 831us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:40.932398 (+ 12322us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2246152}
W20260925 09:57:40.933766 10775 leader_election.cc:341] T 545ac025f2cb469da847979011a0b3ce P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Not found: Tablet not found: 545ac025f2cb469da847979011a0b3ce
W20260925 09:57:40.934006 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet a21909e2e5794f57971f4e42795235c7 because the tablet was not found. No further retry: Not found: Tablet not found: a21909e2e5794f57971f4e42795235c7
I20260925 09:57:40.934122 10775 leader_election.cc:302] T 545ac025f2cb469da847979011a0b3ce P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
W20260925 09:57:40.934589 10775 raft_consensus.cc:2670] T 545ac025f2cb469da847979011a0b3ce P c43461c4ee854ae1ac52b873ccada318: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260925 09:57:40.934984 12379 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 422) took 2172 ms. Trace:
I20260925 09:57:40.935274 12379 rpcz_store.cc:276] 0925 09:57:38.762662 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.763094 (+ 432us) service_pool.cc:227] Handling call
0925 09:57:40.934710 (+2171616us) ts_tablet_manager.cc:1145] Deleting tablet cfa619b699004882a82976f0ef8204ad
0925 09:57:40.934756 (+ 46us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:40.934938 (+ 182us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2168798}
I20260925 09:57:40.935602 10829 raft_consensus.cc:3037] T aca5987b14b64152887ac5ee584b257a P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:57:40.936348 10380 catalog_manager.cc:4808] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): delete failed for tablet cfa619b699004882a82976f0ef8204ad because the tablet was not found. No further retry: Not found: Tablet not found: cfa619b699004882a82976f0ef8204ad
I20260925 09:57:40.937151 12379 tablet_replica.cc:330] T f62c62715c9b42de826c1a66fd431eab P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
I20260925 09:57:40.938848 10775 leader_election.cc:302] T defaef4d6b754ca5a2dfc21b7a596a5b P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:40.939884 12523 raft_consensus.cc:2726] T defaef4d6b754ca5a2dfc21b7a596a5b P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:40.941702 10775 consensus_peers.cc:487] T 10657a81058241fc988d37e5ea7ff8cf P c43461c4ee854ae1ac52b873ccada318 -> Peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Couldn't send request to peer b8850818e2ee42a082802d0095c08d21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 10657a81058241fc988d37e5ea7ff8cf. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:40.948653 12621 raft_consensus.cc:482] T 572c83b4a7c84fdab1fdbb0fbdb0a53d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:40.949302 11084 ts_tablet_manager.cc:1428] T 2015c10f503e4751ad9ff77404fca50b P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.123s user 0.005s sys 0.000s
I20260925 09:57:40.949100 12621 raft_consensus.cc:504] T 572c83b4a7c84fdab1fdbb0fbdb0a53d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.950240 11084 tablet_bootstrap.cc:492] T 5d6ebe0c2ccc48a09b616816953d83fe P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:40.951025 12621 leader_election.cc:288] T 572c83b4a7c84fdab1fdbb0fbdb0a53d P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:40.953398 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "572c83b4a7c84fdab1fdbb0fbdb0a53d" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:40.959556 12621 raft_consensus.cc:482] T cc295f7a9169475f9a9235e657be629f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260925 09:57:40.960701 10918 leader_election.cc:341] T 572c83b4a7c84fdab1fdbb0fbdb0a53d P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:40.963179 12537 raft_consensus.cc:482] T c52745a8cdee461aba7485b0531245d6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:40.964200 12537 raft_consensus.cc:504] T c52745a8cdee461aba7485b0531245d6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.966140 12537 leader_election.cc:288] T c52745a8cdee461aba7485b0531245d6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:40.966248 10832 raft_consensus.cc:3037] T f62c62715c9b42de826c1a66fd431eab P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:40.964344 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "572c83b4a7c84fdab1fdbb0fbdb0a53d" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:40.963338 12621 raft_consensus.cc:504] T cc295f7a9169475f9a9235e657be629f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:40.968595 10824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c52745a8cdee461aba7485b0531245d6" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:40.969583 12621 leader_election.cc:288] T cc295f7a9169475f9a9235e657be629f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
W20260925 09:57:40.969976 10918 leader_election.cc:341] T c52745a8cdee461aba7485b0531245d6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:40.971223 10824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc295f7a9169475f9a9235e657be629f" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:40.972308 10918 leader_election.cc:341] T cc295f7a9169475f9a9235e657be629f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:40.974917 10919 leader_election.cc:341] T 572c83b4a7c84fdab1fdbb0fbdb0a53d P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:40.975131 10681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c52745a8cdee461aba7485b0531245d6" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:40.975287 10919 leader_election.cc:302] T 572c83b4a7c84fdab1fdbb0fbdb0a53d P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
W20260925 09:57:40.976235 10919 leader_election.cc:341] T c52745a8cdee461aba7485b0531245d6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:40.976552 10919 leader_election.cc:302] T c52745a8cdee461aba7485b0531245d6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:40.977527 12537 raft_consensus.cc:2726] T 572c83b4a7c84fdab1fdbb0fbdb0a53d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:40.978039 12537 raft_consensus.cc:2726] T c52745a8cdee461aba7485b0531245d6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:40.978909 11727 ts_tablet_manager.cc:1918] T b3d9a3cf5bda4cf6bc1fa3fae1cb9288 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:40.979236 11727 log.cc:1196] T b3d9a3cf5bda4cf6bc1fa3fae1cb9288 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/b3d9a3cf5bda4cf6bc1fa3fae1cb9288
I20260925 09:57:40.979875 11727 ts_tablet_manager.cc:1939] T b3d9a3cf5bda4cf6bc1fa3fae1cb9288 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:40.981601 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 411) took 2135 ms. Trace:
I20260925 09:57:41.001813 11085 tablet_bootstrap.cc:492] T 9744b9814b69470396ce039348fe5e45 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:41.002259 11085 ts_tablet_manager.cc:1397] T 9744b9814b69470396ce039348fe5e45 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.148s user 0.009s sys 0.000s
I20260925 09:57:41.001920 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc295f7a9169475f9a9235e657be629f" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:41.005642 10775 leader_election.cc:302] T 96a80382bb7d4395a14ad9ae972f4a25 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:41.006846 12523 raft_consensus.cc:2726] T 96a80382bb7d4395a14ad9ae972f4a25 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 09:57:41.003891 10919 leader_election.cc:341] T cc295f7a9169475f9a9235e657be629f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:41.008001 10919 leader_election.cc:302] T cc295f7a9169475f9a9235e657be629f P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:41.008644 12621 raft_consensus.cc:2726] T cc295f7a9169475f9a9235e657be629f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:41.008908 10775 consensus_peers.cc:487] T 92814437d53143a79f99199f8ee3a742 P c43461c4ee854ae1ac52b873ccada318 -> Peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Couldn't send request to peer b8850818e2ee42a082802d0095c08d21. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:40.981884 11727 rpcz_store.cc:276] 0925 09:57:38.845878 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.846626 (+ 748us) service_pool.cc:227] Handling call
0925 09:57:40.711043 (+1864417us) ts_tablet_manager.cc:1145] Deleting tablet b3d9a3cf5bda4cf6bc1fa3fae1cb9288
0925 09:57:40.711083 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:40.849595 (+138512us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:40.978704 (+129109us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:40.981554 (+ 2850us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1863885}
I20260925 09:57:41.011226 10824 raft_consensus.cc:3037] T 350904aa5957496d90a2aa2e637b591c P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:41.012918 10775 leader_election.cc:302] T f62c62715c9b42de826c1a66fd431eab P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c43461c4ee854ae1ac52b873ccada318; no voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:41.013815 12523 raft_consensus.cc:2726] T f62c62715c9b42de826c1a66fd431eab P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:41.014969 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61aa6380d30a4b21b70ee2d321247bd8" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:41.015544 10838 raft_consensus.cc:2454] T 61aa6380d30a4b21b70ee2d321247bd8 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:41.019037 12621 raft_consensus.cc:482] T d289c2fb23df499a8118de92d82f0d19 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:41.023720 12621 raft_consensus.cc:504] T d289c2fb23df499a8118de92d82f0d19 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.024698 12615 raft_consensus.cc:504] T b39f9e7102ff4a9c96a2cc5d2676f03d P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.026566 12615 leader_election.cc:288] T b39f9e7102ff4a9c96a2cc5d2676f03d P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:41.026599 12621 leader_election.cc:288] T d289c2fb23df499a8118de92d82f0d19 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:41.026880 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d289c2fb23df499a8118de92d82f0d19" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:41.028358 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet b3d9a3cf5bda4cf6bc1fa3fae1cb9288 (table default.Table_cc9f467e078c448f87392bf324b040a8 [id=4cefdaddfd6143468df4502cd35afac4]) successfully deleted
W20260925 09:57:41.029178 10919 leader_election.cc:341] T d289c2fb23df499a8118de92d82f0d19 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:41.030079 10829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d289c2fb23df499a8118de92d82f0d19" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:41.033366 10918 leader_election.cc:341] T d289c2fb23df499a8118de92d82f0d19 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:41.033696 10918 leader_election.cc:302] T d289c2fb23df499a8118de92d82f0d19 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:41.034622 12621 raft_consensus.cc:2726] T d289c2fb23df499a8118de92d82f0d19 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:41.013777 11085 raft_consensus.cc:348] T 9744b9814b69470396ce039348fe5e45 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.036062 11085 raft_consensus.cc:374] T 9744b9814b69470396ce039348fe5e45 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:41.036298 11085 raft_consensus.cc:729] T 9744b9814b69470396ce039348fe5e45 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:41.036826 11085 consensus_queue.cc:260] T 9744b9814b69470396ce039348fe5e45 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.038527 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b39f9e7102ff4a9c96a2cc5d2676f03d" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:41.038854 11085 ts_tablet_manager.cc:1428] T 9744b9814b69470396ce039348fe5e45 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.036s user 0.004s sys 0.000s
I20260925 09:57:41.048832 11082 tablet_bootstrap.cc:492] T abfc8f390d434c47a8c4990f8406c270 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:41.049213 11082 ts_tablet_manager.cc:1397] T abfc8f390d434c47a8c4990f8406c270 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.233s user 0.003s sys 0.013s
W20260925 09:57:41.063508 10621 leader_election.cc:341] T b39f9e7102ff4a9c96a2cc5d2676f03d P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:40.971518 10619 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 99059.7us
I20260925 09:57:41.073019 12027 ts_tablet_manager.cc:1918] T 545ac025f2cb469da847979011a0b3ce P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:41.073444 12027 log.cc:1196] T 545ac025f2cb469da847979011a0b3ce P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/545ac025f2cb469da847979011a0b3ce
I20260925 09:57:41.074198 12027 ts_tablet_manager.cc:1939] T 545ac025f2cb469da847979011a0b3ce P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:41.082355 11084 tablet_bootstrap.cc:654] T 5d6ebe0c2ccc48a09b616816953d83fe P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:41.082706 11085 tablet_bootstrap.cc:492] T 69028aa7908c45b896da4ba8deef07b1 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:41.097913 11727 tablet_replica.cc:330] T f62c62715c9b42de826c1a66fd431eab P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:41.051296 11082 raft_consensus.cc:348] T abfc8f390d434c47a8c4990f8406c270 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.103698 11082 raft_consensus.cc:374] T abfc8f390d434c47a8c4990f8406c270 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:41.103988 11082 raft_consensus.cc:729] T abfc8f390d434c47a8c4990f8406c270 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
W20260925 09:57:41.113914 10621 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 45151.5us
W20260925 09:57:41.123302 10379 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet cf7b2d3f3b3949a0b81283bc68e7b131 (table default.Table_f261a668854045ea93bc3cd825d6a685 [id=e499d19b87184375b75fe2fb263d92b5], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet cf7b2d3f3b3949a0b81283bc68e7b131 consensus state has no leader
I20260925 09:57:41.137014 11085 tablet_bootstrap.cc:654] T 69028aa7908c45b896da4ba8deef07b1 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:41.129199 10392 catalog_manager.cc:5495] T 9744b9814b69470396ce039348fe5e45 P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: HEALTHY } } }
I20260925 09:57:41.143296 11727 raft_consensus.cc:2229] T f62c62715c9b42de826c1a66fd431eab P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:41.144142 11727 raft_consensus.cc:2258] T f62c62715c9b42de826c1a66fd431eab P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:41.138252 12621 raft_consensus.cc:482] T d7459730813f486e9cb9aeffe29509a2 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:41.155604 12621 raft_consensus.cc:504] T d7459730813f486e9cb9aeffe29509a2 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.156760 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 392) took 2454 ms. Trace:
I20260925 09:57:41.157694 12621 leader_election.cc:288] T d7459730813f486e9cb9aeffe29509a2 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:41.157079 12027 rpcz_store.cc:276] 0925 09:57:38.702399 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.723448 (+ 21049us) service_pool.cc:227] Handling call
0925 09:57:40.861022 (+2137574us) ts_tablet_manager.cc:1145] Deleting tablet 545ac025f2cb469da847979011a0b3ce
0925 09:57:40.861058 (+ 36us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:40.940439 (+ 79381us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:41.072775 (+132336us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:41.156686 (+ 83911us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2137004}
I20260925 09:57:41.158422 10824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7459730813f486e9cb9aeffe29509a2" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:41.158690 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 394) took 2425 ms. Trace:
I20260925 09:57:41.159155 11727 ts_tablet_manager.cc:1905] T f62c62715c9b42de826c1a66fd431eab P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:41.158963 12027 rpcz_store.cc:276] 0925 09:57:38.733449 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.733548 (+ 99us) service_pool.cc:227] Handling call
0925 09:57:41.158464 (+2424916us) ts_tablet_manager.cc:1145] Deleting tablet fb9c51f7e7fd4c029be2a710cb9462eb
0925 09:57:41.158501 (+ 37us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:41.158647 (+ 146us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2424454}
W20260925 09:57:41.160676 10918 leader_election.cc:341] T d7459730813f486e9cb9aeffe29509a2 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:41.165046 10681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7459730813f486e9cb9aeffe29509a2" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:41.166263 10919 leader_election.cc:341] T d7459730813f486e9cb9aeffe29509a2 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:41.166641 10919 leader_election.cc:302] T d7459730813f486e9cb9aeffe29509a2 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:41.168100 12537 raft_consensus.cc:2726] T d7459730813f486e9cb9aeffe29509a2 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:41.181243 10983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b39f9e7102ff4a9c96a2cc5d2676f03d" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:41.181744 10983 raft_consensus.cc:3037] T b39f9e7102ff4a9c96a2cc5d2676f03d P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:41.185189 12645 tablet.cc:1722] T 83b0cf21f2a144719a2e13990e1921c0 P be97ab5d75934a648d4ab76341193ec0: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:41.190251 12379 raft_consensus.cc:2229] T f62c62715c9b42de826c1a66fd431eab P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:41.191080 12379 raft_consensus.cc:2258] T f62c62715c9b42de826c1a66fd431eab P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:41.193137 12379 ts_tablet_manager.cc:1905] T f62c62715c9b42de826c1a66fd431eab P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:41.194921 11084 tablet_bootstrap.cc:492] T 5d6ebe0c2ccc48a09b616816953d83fe P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:41.195318 11084 ts_tablet_manager.cc:1397] T 5d6ebe0c2ccc48a09b616816953d83fe P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.245s user 0.005s sys 0.006s
I20260925 09:57:41.197670 11084 raft_consensus.cc:348] T 5d6ebe0c2ccc48a09b616816953d83fe P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.199473 11084 raft_consensus.cc:374] T 5d6ebe0c2ccc48a09b616816953d83fe P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:41.199716 11084 raft_consensus.cc:729] T 5d6ebe0c2ccc48a09b616816953d83fe P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:41.200311 11084 consensus_queue.cc:260] T 5d6ebe0c2ccc48a09b616816953d83fe P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.202221 11084 ts_tablet_manager.cc:1428] T 5d6ebe0c2ccc48a09b616816953d83fe P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.007s user 0.000s sys 0.003s
I20260925 09:57:41.206650 11084 tablet_bootstrap.cc:492] T 30e04d90c962442cadf7bdec6f827dac P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:41.211206 11084 tablet_bootstrap.cc:654] T 30e04d90c962442cadf7bdec6f827dac P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:41.212877 10983 raft_consensus.cc:2454] T b39f9e7102ff4a9c96a2cc5d2676f03d P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:41.213753 12644 tablet.cc:1722] T 83b0cf21f2a144719a2e13990e1921c0 P c43461c4ee854ae1ac52b873ccada318: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260925 09:57:41.214097 10619 leader_election.cc:341] T 61aa6380d30a4b21b70ee2d321247bd8 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 61aa6380d30a4b21b70ee2d321247bd8
I20260925 09:57:41.218466 10619 leader_election.cc:302] T b39f9e7102ff4a9c96a2cc5d2676f03d P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:41.219671 12626 raft_consensus.cc:2781] T b39f9e7102ff4a9c96a2cc5d2676f03d P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:41.220074 12626 raft_consensus.cc:686] T b39f9e7102ff4a9c96a2cc5d2676f03d P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:41.220824 12626 consensus_queue.cc:237] T b39f9e7102ff4a9c96a2cc5d2676f03d P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.104619 11082 consensus_queue.cc:260] T abfc8f390d434c47a8c4990f8406c270 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:41.225783 10379 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.62:34455, user_credentials={real_user=slave}} blocked reactor thread for 86998.1us
I20260925 09:57:41.228273 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 545ac025f2cb469da847979011a0b3ce (table default.Table_14d12f6871c847daa860c3a0cfee4231 [id=f61297f0d10c4656bd06a752c9c7596b]) successfully deleted
I20260925 09:57:41.228995 10621 leader_election.cc:302] T 61aa6380d30a4b21b70ee2d321247bd8 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters: be97ab5d75934a648d4ab76341193ec0
W20260925 09:57:41.229131 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet fb9c51f7e7fd4c029be2a710cb9462eb because the tablet was not found. No further retry: Not found: Tablet not found: fb9c51f7e7fd4c029be2a710cb9462eb
I20260925 09:57:41.229631 12612 raft_consensus.cc:2781] T 61aa6380d30a4b21b70ee2d321247bd8 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:41.229945 12612 raft_consensus.cc:482] T 61aa6380d30a4b21b70ee2d321247bd8 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:41.230240 12612 raft_consensus.cc:3037] T 61aa6380d30a4b21b70ee2d321247bd8 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:41.240913 12621 raft_consensus.cc:482] T 56f8c08025b94c6eb8b429bd7b0408aa P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:41.241463 12621 raft_consensus.cc:504] T 56f8c08025b94c6eb8b429bd7b0408aa P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.243669 12621 leader_election.cc:288] T 56f8c08025b94c6eb8b429bd7b0408aa P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:41.245940 10824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56f8c08025b94c6eb8b429bd7b0408aa" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:41.247507 10918 leader_election.cc:341] T 56f8c08025b94c6eb8b429bd7b0408aa P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:41.251798 11085 tablet_bootstrap.cc:492] T 69028aa7908c45b896da4ba8deef07b1 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:41.252290 11085 ts_tablet_manager.cc:1397] T 69028aa7908c45b896da4ba8deef07b1 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.170s user 0.011s sys 0.000s
I20260925 09:57:41.246024 10681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56f8c08025b94c6eb8b429bd7b0408aa" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:41.254904 10919 leader_election.cc:341] T 56f8c08025b94c6eb8b429bd7b0408aa P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:41.255265 10919 leader_election.cc:302] T 56f8c08025b94c6eb8b429bd7b0408aa P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:41.256325 12621 raft_consensus.cc:2726] T 56f8c08025b94c6eb8b429bd7b0408aa P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:41.257107 11085 raft_consensus.cc:348] T 69028aa7908c45b896da4ba8deef07b1 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.257813 11085 raft_consensus.cc:374] T 69028aa7908c45b896da4ba8deef07b1 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:41.258098 11085 raft_consensus.cc:729] T 69028aa7908c45b896da4ba8deef07b1 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:41.258798 11085 consensus_queue.cc:260] T 69028aa7908c45b896da4ba8deef07b1 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.259508 11082 ts_tablet_manager.cc:1428] T abfc8f390d434c47a8c4990f8406c270 P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.210s user 0.007s sys 0.003s
I20260925 09:57:41.260378 11085 ts_tablet_manager.cc:1428] T 69028aa7908c45b896da4ba8deef07b1 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.008s user 0.004s sys 0.000s
I20260925 09:57:41.260360 11082 tablet_bootstrap.cc:492] T 54157e776efa4c6c8e2bd4c58321fcda P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:41.261193 11085 tablet_bootstrap.cc:492] T a81530200cf14717bc7cbf67bb73f4e5 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
W20260925 09:57:41.265561 10621 leader_election.cc:341] T b39f9e7102ff4a9c96a2cc5d2676f03d P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:41.272746 11085 tablet_bootstrap.cc:654] T a81530200cf14717bc7cbf67bb73f4e5 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:41.293712 11085 tablet_bootstrap.cc:492] T a81530200cf14717bc7cbf67bb73f4e5 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:41.294175 11085 ts_tablet_manager.cc:1397] T a81530200cf14717bc7cbf67bb73f4e5 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.033s user 0.007s sys 0.004s
I20260925 09:57:41.296391 11085 raft_consensus.cc:348] T a81530200cf14717bc7cbf67bb73f4e5 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.297257 11085 raft_consensus.cc:374] T a81530200cf14717bc7cbf67bb73f4e5 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:41.297573 11085 raft_consensus.cc:729] T a81530200cf14717bc7cbf67bb73f4e5 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:41.298197 11085 consensus_queue.cc:260] T a81530200cf14717bc7cbf67bb73f4e5 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.301726 11085 ts_tablet_manager.cc:1428] T a81530200cf14717bc7cbf67bb73f4e5 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20260925 09:57:41.306723 11085 tablet_bootstrap.cc:492] T 3039ec3febf848dba7c4e6fd38bb8053 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:41.312959 11085 tablet_bootstrap.cc:654] T 3039ec3febf848dba7c4e6fd38bb8053 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:41.322537 10621 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 55403.6us
I20260925 09:57:41.327777 12027 tablet_replica.cc:330] T 7a31b999f0fd447eacff13de71aa3fff P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:41.330754 10824 raft_consensus.cc:3037] T defaef4d6b754ca5a2dfc21b7a596a5b P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:41.334316 11084 tablet_bootstrap.cc:492] T 30e04d90c962442cadf7bdec6f827dac P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:41.334412 11085 tablet_bootstrap.cc:492] T 3039ec3febf848dba7c4e6fd38bb8053 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:41.334700 11084 ts_tablet_manager.cc:1397] T 30e04d90c962442cadf7bdec6f827dac P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.128s user 0.026s sys 0.027s
I20260925 09:57:41.334779 11085 ts_tablet_manager.cc:1397] T 3039ec3febf848dba7c4e6fd38bb8053 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.028s user 0.011s sys 0.000s
I20260925 09:57:41.334970 12379 ts_tablet_manager.cc:1918] T f62c62715c9b42de826c1a66fd431eab P b8850818e2ee42a082802d0095c08d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:41.336853 11084 raft_consensus.cc:348] T 30e04d90c962442cadf7bdec6f827dac P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.336948 11085 raft_consensus.cc:348] T 3039ec3febf848dba7c4e6fd38bb8053 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.337473 11084 raft_consensus.cc:374] T 30e04d90c962442cadf7bdec6f827dac P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:41.337587 11085 raft_consensus.cc:374] T 3039ec3febf848dba7c4e6fd38bb8053 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:41.337692 11084 raft_consensus.cc:729] T 30e04d90c962442cadf7bdec6f827dac P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:41.337821 11085 raft_consensus.cc:729] T 3039ec3febf848dba7c4e6fd38bb8053 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:41.338289 11084 consensus_queue.cc:260] T 30e04d90c962442cadf7bdec6f827dac P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.340602 11085 consensus_queue.cc:260] T 3039ec3febf848dba7c4e6fd38bb8053 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.342929 12603 raft_consensus.cc:482] T 3039ec3febf848dba7c4e6fd38bb8053 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:41.343309 12603 raft_consensus.cc:504] T 3039ec3febf848dba7c4e6fd38bb8053 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.345078 12603 leader_election.cc:288] T 3039ec3febf848dba7c4e6fd38bb8053 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
W20260925 09:57:41.348034 10772 leader_election.cc:341] T 3039ec3febf848dba7c4e6fd38bb8053 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 3039ec3febf848dba7c4e6fd38bb8053
I20260925 09:57:41.348119 11084 ts_tablet_manager.cc:1428] T 30e04d90c962442cadf7bdec6f827dac P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.013s user 0.006s sys 0.000s
I20260925 09:57:41.348961 11085 ts_tablet_manager.cc:1428] T 3039ec3febf848dba7c4e6fd38bb8053 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.014s user 0.006s sys 0.000s
I20260925 09:57:41.349752 11085 tablet_bootstrap.cc:492] T 05e2c7f9820f41679c4d58343ff3f829 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:41.355981 11082 tablet_bootstrap.cc:654] T 54157e776efa4c6c8e2bd4c58321fcda P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:41.360051 11084 tablet_bootstrap.cc:492] T 4a6b1284cddc4b2581931d8895212ca6 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:41.361745 11085 tablet_bootstrap.cc:654] T 05e2c7f9820f41679c4d58343ff3f829 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:41.364961 11084 tablet_bootstrap.cc:654] T 4a6b1284cddc4b2581931d8895212ca6 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:41.372186 12027 raft_consensus.cc:2229] T 7a31b999f0fd447eacff13de71aa3fff P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:41.373162 12027 pending_rounds.cc:62] T 7a31b999f0fd447eacff13de71aa3fff P c43461c4ee854ae1ac52b873ccada318: Trying to abort 1 pending ops.
I20260925 09:57:41.373435 12027 pending_rounds.cc:68] T 7a31b999f0fd447eacff13de71aa3fff P c43461c4ee854ae1ac52b873ccada318: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192724908818432 op_type: NO_OP noop_request { }
I20260925 09:57:41.373857 12027 raft_consensus.cc:2866] T 7a31b999f0fd447eacff13de71aa3fff P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:57:41.374308 12027 raft_consensus.cc:2258] T 7a31b999f0fd447eacff13de71aa3fff P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:41.379364 11084 tablet_bootstrap.cc:492] T 4a6b1284cddc4b2581931d8895212ca6 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:41.379791 11084 ts_tablet_manager.cc:1397] T 4a6b1284cddc4b2581931d8895212ca6 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.020s user 0.010s sys 0.002s
I20260925 09:57:41.382287 11084 raft_consensus.cc:348] T 4a6b1284cddc4b2581931d8895212ca6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.382894 11084 raft_consensus.cc:374] T 4a6b1284cddc4b2581931d8895212ca6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:41.383113 11084 raft_consensus.cc:729] T 4a6b1284cddc4b2581931d8895212ca6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:41.398943 11084 consensus_queue.cc:260] T 4a6b1284cddc4b2581931d8895212ca6 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.401292 11084 ts_tablet_manager.cc:1428] T 4a6b1284cddc4b2581931d8895212ca6 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.021s user 0.003s sys 0.002s
I20260925 09:57:41.259049 12612 raft_consensus.cc:504] T 61aa6380d30a4b21b70ee2d321247bd8 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.402171 11084 tablet_bootstrap.cc:492] T 815b4b1bd914433798dcb5deec6ec79c P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:41.403468 12612 leader_election.cc:288] T 61aa6380d30a4b21b70ee2d321247bd8 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:41.405084 12027 ts_tablet_manager.cc:1905] T 7a31b999f0fd447eacff13de71aa3fff P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:41.335338 12379 log.cc:1196] T f62c62715c9b42de826c1a66fd431eab P b8850818e2ee42a082802d0095c08d21: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/f62c62715c9b42de826c1a66fd431eab
I20260925 09:57:41.412153 12379 ts_tablet_manager.cc:1939] T f62c62715c9b42de826c1a66fd431eab P b8850818e2ee42a082802d0095c08d21: Deleting consensus metadata
W20260925 09:57:41.413472 10619 leader_election.cc:341] T 61aa6380d30a4b21b70ee2d321247bd8 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 61aa6380d30a4b21b70ee2d321247bd8
I20260925 09:57:41.414616 12379 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37876 (request call id 428) took 2202 ms. Trace:
I20260925 09:57:41.421341 11084 tablet_bootstrap.cc:654] T 815b4b1bd914433798dcb5deec6ec79c P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:41.429425 11084 tablet_bootstrap.cc:492] T 815b4b1bd914433798dcb5deec6ec79c P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:41.429796 11084 ts_tablet_manager.cc:1397] T 815b4b1bd914433798dcb5deec6ec79c P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.028s user 0.011s sys 0.001s
I20260925 09:57:41.430562 12621 raft_consensus.cc:482] T f1d31dca90df414a83be7c9e404b8d30 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:41.432855 12537 raft_consensus.cc:482] T 086fc96ca247461db2ecd864139a228c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:41.431913 11084 raft_consensus.cc:348] T 815b4b1bd914433798dcb5deec6ec79c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.434558 11084 raft_consensus.cc:374] T 815b4b1bd914433798dcb5deec6ec79c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:41.433252 12537 raft_consensus.cc:504] T 086fc96ca247461db2ecd864139a228c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.434792 11084 raft_consensus.cc:729] T 815b4b1bd914433798dcb5deec6ec79c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:41.435344 11084 consensus_queue.cc:260] T 815b4b1bd914433798dcb5deec6ec79c P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.436117 12537 leader_election.cc:288] T 086fc96ca247461db2ecd864139a228c P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:41.430971 12621 raft_consensus.cc:504] T f1d31dca90df414a83be7c9e404b8d30 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.441763 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1d31dca90df414a83be7c9e404b8d30" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:41.442916 12621 leader_election.cc:288] T f1d31dca90df414a83be7c9e404b8d30 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:41.444971 11727 ts_tablet_manager.cc:1918] T f62c62715c9b42de826c1a66fd431eab P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:41.445399 11727 log.cc:1196] T f62c62715c9b42de826c1a66fd431eab P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/f62c62715c9b42de826c1a66fd431eab
I20260925 09:57:41.445592 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61aa6380d30a4b21b70ee2d321247bd8" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:41.446070 10832 raft_consensus.cc:3037] T 61aa6380d30a4b21b70ee2d321247bd8 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:41.446271 11727 ts_tablet_manager.cc:1939] T f62c62715c9b42de826c1a66fd431eab P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
W20260925 09:57:41.444064 10918 leader_election.cc:341] T f1d31dca90df414a83be7c9e404b8d30 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:41.452725 10824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "086fc96ca247461db2ecd864139a228c" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:41.454016 10918 leader_election.cc:341] T 086fc96ca247461db2ecd864139a228c P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:41.454946 10832 raft_consensus.cc:2454] T 61aa6380d30a4b21b70ee2d321247bd8 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:41.471179 11085 tablet_bootstrap.cc:492] T 05e2c7f9820f41679c4d58343ff3f829 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:41.419206 10679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3039ec3febf848dba7c4e6fd38bb8053" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:41.438516 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "086fc96ca247461db2ecd864139a228c" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:41.472025 10679 raft_consensus.cc:2454] T 3039ec3febf848dba7c4e6fd38bb8053 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
I20260925 09:57:41.444314 10673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1d31dca90df414a83be7c9e404b8d30" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:41.473242 10919 leader_election.cc:341] T 086fc96ca247461db2ecd864139a228c P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:41.473606 10919 leader_election.cc:302] T 086fc96ca247461db2ecd864139a228c P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
W20260925 09:57:41.474565 10919 leader_election.cc:341] T f1d31dca90df414a83be7c9e404b8d30 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:41.474874 10919 leader_election.cc:302] T f1d31dca90df414a83be7c9e404b8d30 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:41.474607 12379 rpcz_store.cc:276] 0925 09:57:39.211880 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:39.212017 (+ 137us) service_pool.cc:227] Handling call
0925 09:57:40.936953 (+1724936us) ts_tablet_manager.cc:1145] Deleting tablet f62c62715c9b42de826c1a66fd431eab
0925 09:57:40.937007 (+ 54us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:41.201339 (+264332us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:41.334763 (+133424us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:41.414564 (+ 79801us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":252299,"tablet-delete.queue_time_us":1724391}
I20260925 09:57:41.475558 12621 raft_consensus.cc:2726] T 086fc96ca247461db2ecd864139a228c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:41.476039 12621 raft_consensus.cc:2726] T f1d31dca90df414a83be7c9e404b8d30 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:41.471661 11085 ts_tablet_manager.cc:1397] T 05e2c7f9820f41679c4d58343ff3f829 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.122s user 0.010s sys 0.001s
I20260925 09:57:41.479426 11085 raft_consensus.cc:348] T 05e2c7f9820f41679c4d58343ff3f829 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.482980 11085 raft_consensus.cc:374] T 05e2c7f9820f41679c4d58343ff3f829 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:41.487834 11085 raft_consensus.cc:729] T 05e2c7f9820f41679c4d58343ff3f829 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:41.488229 12537 raft_consensus.cc:482] T 815b4b1bd914433798dcb5deec6ec79c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:41.488636 12537 raft_consensus.cc:504] T 815b4b1bd914433798dcb5deec6ec79c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.490460 12537 leader_election.cc:288] T 815b4b1bd914433798dcb5deec6ec79c P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:41.491940 10673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "815b4b1bd914433798dcb5deec6ec79c" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:41.493239 10919 leader_election.cc:341] T 815b4b1bd914433798dcb5deec6ec79c P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:41.494175 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "815b4b1bd914433798dcb5deec6ec79c" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:41.469333 11082 tablet_bootstrap.cc:492] T 54157e776efa4c6c8e2bd4c58321fcda P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
I20260925 09:57:41.494913 11082 ts_tablet_manager.cc:1397] T 54157e776efa4c6c8e2bd4c58321fcda P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.235s user 0.008s sys 0.002s
W20260925 09:57:41.495453 10918 leader_election.cc:341] T 815b4b1bd914433798dcb5deec6ec79c P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:41.495877 10918 leader_election.cc:302] T 815b4b1bd914433798dcb5deec6ec79c P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:41.497026 10380 catalog_manager.cc:4841] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): tablet f62c62715c9b42de826c1a66fd431eab (table default.Table_bcbc82519cf043738f61029181f29418 [id=b3f7f21ab2214e4cb637bea27ce62abe]) successfully deleted
I20260925 09:57:41.497253 12537 raft_consensus.cc:2726] T 815b4b1bd914433798dcb5deec6ec79c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:41.488759 11085 consensus_queue.cc:260] T 05e2c7f9820f41679c4d58343ff3f829 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.500217 12603 raft_consensus.cc:482] T 69028aa7908c45b896da4ba8deef07b1 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:41.500679 12603 raft_consensus.cc:504] T 69028aa7908c45b896da4ba8deef07b1 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.502592 12603 leader_election.cc:288] T 69028aa7908c45b896da4ba8deef07b1 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:41.496577 12379 tablet_replica.cc:330] T 05e2c7f9820f41679c4d58343ff3f829 P b8850818e2ee42a082802d0095c08d21: stopping tablet replica
W20260925 09:57:41.504858 10772 leader_election.cc:341] T 69028aa7908c45b896da4ba8deef07b1 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 69028aa7908c45b896da4ba8deef07b1
I20260925 09:57:41.504858 11085 ts_tablet_manager.cc:1428] T 05e2c7f9820f41679c4d58343ff3f829 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.027s user 0.004s sys 0.000s
I20260925 09:57:41.506166 11085 tablet_bootstrap.cc:492] T 1e76f9e0ba554e1cad0cf18863393608 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:41.510810 11085 tablet_bootstrap.cc:654] T 1e76f9e0ba554e1cad0cf18863393608 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:41.513495 12379 raft_consensus.cc:2229] T 05e2c7f9820f41679c4d58343ff3f829 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:41.538486 12537 raft_consensus.cc:482] T 2015c10f503e4751ad9ff77404fca50b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:41.538933 12537 raft_consensus.cc:504] T 2015c10f503e4751ad9ff77404fca50b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.502372 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 413) took 2287 ms. Trace:
I20260925 09:57:41.539726 11727 rpcz_store.cc:276] 0925 09:57:39.215024 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:39.215727 (+ 703us) service_pool.cc:227] Handling call
0925 09:57:41.028303 (+1812576us) ts_tablet_manager.cc:1145] Deleting tablet f62c62715c9b42de826c1a66fd431eab
0925 09:57:41.097770 (+ 69467us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:41.210444 (+112674us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:41.444684 (+234240us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:41.498070 (+ 53386us) spinlock_profiling.cc:243] Waited 48.8 ms on lock 0x7b5800001898. stack: 00007f0a8f2ab033 00007f0a8f2aaf30 00007f0a8ee2c534 00000000004db2b4 00000000004db249 00007f0a8f121f11 00007f0a8f2a5851 00007f0a8f2a5e82 00007f0a981658b9 00007f0a971995d8 00007f0a9725de42 00007f0a9728de10 00007f0a97274e0d 00007f0a97274db9 00007f0a97274d49 00007f0a97274d11
0925 09:57:41.502302 (+ 4232us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":44314,"spinlock_wait_cycles":112241024,"tablet-delete.queue_time_us":1812038}
I20260925 09:57:41.540747 12537 leader_election.cc:288] T 2015c10f503e4751ad9ff77404fca50b P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:41.541664 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet f62c62715c9b42de826c1a66fd431eab (table default.Table_bcbc82519cf043738f61029181f29418 [id=b3f7f21ab2214e4cb637bea27ce62abe]) successfully deleted
I20260925 09:57:41.550235 11727 tablet_replica.cc:330] T 05e2c7f9820f41679c4d58343ff3f829 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:41.551088 11727 raft_consensus.cc:2229] T 05e2c7f9820f41679c4d58343ff3f829 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:41.551591 11727 raft_consensus.cc:2258] T 05e2c7f9820f41679c4d58343ff3f829 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:41.561285 11727 ts_tablet_manager.cc:1905] T 05e2c7f9820f41679c4d58343ff3f829 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:41.570478 12661 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8af3f111e8da4d2dad4e4a98ef0fd5d3)
I20260925 09:57:41.526584 12624 raft_consensus.cc:482] T 3d84c717b6ac462eb331e25d80c918e4 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:41.575675 12661 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } }
I20260925 09:57:41.575635 12624 raft_consensus.cc:504] T 3d84c717b6ac462eb331e25d80c918e4 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.577673 12624 leader_election.cc:288] T 3d84c717b6ac462eb331e25d80c918e4 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:41.578326 10326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" candidate_term: 5 candidate_status { last_received { term: 4 index: 456 } } ignore_live_leader: false dest_uuid: "54c8485a7dcc47a1a75d08ef57647ece" is_pre_election: true
I20260925 09:57:41.579317 12379 pending_rounds.cc:62] T 05e2c7f9820f41679c4d58343ff3f829 P b8850818e2ee42a082802d0095c08d21: Trying to abort 1 pending ops.
I20260925 09:57:41.579556 12379 pending_rounds.cc:68] T 05e2c7f9820f41679c4d58343ff3f829 P b8850818e2ee42a082802d0095c08d21: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192748250939392 op_type: NO_OP noop_request { }
I20260925 09:57:41.579535 10983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d84c717b6ac462eb331e25d80c918e4" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:41.579926 12379 raft_consensus.cc:2866] T 05e2c7f9820f41679c4d58343ff3f829 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:57:41.580084 10983 raft_consensus.cc:2454] T 3d84c717b6ac462eb331e25d80c918e4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:41.580274 12379 raft_consensus.cc:2258] T 05e2c7f9820f41679c4d58343ff3f829 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:41.581024 10619 leader_election.cc:302] T 3d84c717b6ac462eb331e25d80c918e4 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:41.581751 12624 raft_consensus.cc:2781] T 3d84c717b6ac462eb331e25d80c918e4 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:41.582093 12624 raft_consensus.cc:482] T 3d84c717b6ac462eb331e25d80c918e4 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:41.582399 12624 raft_consensus.cc:3037] T 3d84c717b6ac462eb331e25d80c918e4 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:41.582855 12661 leader_election.cc:288] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 54c8485a7dcc47a1a75d08ef57647ece (127.10.9.62:34455), 8af3f111e8da4d2dad4e4a98ef0fd5d3 (127.10.9.61:33047)
I20260925 09:57:41.583636 10403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" candidate_term: 5 candidate_status { last_received { term: 4 index: 456 } } ignore_live_leader: false dest_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" is_pre_election: true
I20260925 09:57:41.585214 12198 leader_election.cc:302] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8be057e0b6d40a3a4b90f2b1f4a0fa2; no voters: 54c8485a7dcc47a1a75d08ef57647ece, 8af3f111e8da4d2dad4e4a98ef0fd5d3
I20260925 09:57:41.586313 12661 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260925 09:57:41.592887 11082 raft_consensus.cc:348] T 54157e776efa4c6c8e2bd4c58321fcda P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.593510 11082 raft_consensus.cc:374] T 54157e776efa4c6c8e2bd4c58321fcda P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:41.593753 11082 raft_consensus.cc:729] T 54157e776efa4c6c8e2bd4c58321fcda P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:41.594389 11082 consensus_queue.cc:260] T 54157e776efa4c6c8e2bd4c58321fcda P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.535784 10621 leader_election.cc:302] T 61aa6380d30a4b21b70ee2d321247bd8 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters: be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:41.543318 10673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2015c10f503e4751ad9ff77404fca50b" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
W20260925 09:57:41.596513 10919 leader_election.cc:341] T 2015c10f503e4751ad9ff77404fca50b P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:41.597474 10621 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 61807.6us
I20260925 09:57:41.600708 12612 raft_consensus.cc:2781] T 61aa6380d30a4b21b70ee2d321247bd8 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:41.602519 12612 raft_consensus.cc:686] T 61aa6380d30a4b21b70ee2d321247bd8 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:41.606302 12612 consensus_queue.cc:237] T 61aa6380d30a4b21b70ee2d321247bd8 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.612076 12624 raft_consensus.cc:504] T 3d84c717b6ac462eb331e25d80c918e4 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.614708 12624 leader_election.cc:288] T 3d84c717b6ac462eb331e25d80c918e4 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:41.614900 10983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d84c717b6ac462eb331e25d80c918e4" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:41.615435 10983 raft_consensus.cc:3037] T 3d84c717b6ac462eb331e25d80c918e4 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:41.631857 10983 raft_consensus.cc:2454] T 3d84c717b6ac462eb331e25d80c918e4 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:41.637104 11084 ts_tablet_manager.cc:1428] T 815b4b1bd914433798dcb5deec6ec79c P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.207s user 0.056s sys 0.036s
I20260925 09:57:41.638041 11084 tablet_bootstrap.cc:492] T 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:41.642613 11084 tablet_bootstrap.cc:654] T 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:41.665604 10952 tablet_service.cc:1467] Processing CreateTablet for tablet 53ccf068342d44678def17a72be39d97 (DEFAULT_TABLE table=default.Table_c8324101e1254bf2ad787feab7261dae [id=79c1c015c2534e57911c9d9ac26bb03e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:41.666548 10952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53ccf068342d44678def17a72be39d97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:41.678802 11084 tablet_bootstrap.cc:492] T 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:41.679193 11084 ts_tablet_manager.cc:1397] T 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.041s user 0.004s sys 0.004s
I20260925 09:57:41.682132 11084 raft_consensus.cc:348] T 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.682809 11084 raft_consensus.cc:374] T 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:41.683037 11084 raft_consensus.cc:729] T 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:41.683673 11084 consensus_queue.cc:260] T 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.685141 10956 tablet_service.cc:1467] Processing CreateTablet for tablet 9eef15f91ad842dda58ff133366387fe (DEFAULT_TABLE table=default.Table_3a4caa0a9b2044689a5696cf420f9bcd [id=956e11b71c9e44ffa1243608fe0bfeac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:41.686053 10956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eef15f91ad842dda58ff133366387fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:41.693887 11084 ts_tablet_manager.cc:1428] T 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.014s user 0.005s sys 0.000s
I20260925 09:57:41.694715 11084 tablet_bootstrap.cc:492] T c47783e4ea694561bf008def26655884 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:41.704332 10946 tablet_service.cc:1467] Processing CreateTablet for tablet 4637237acb184a6aaae94a04218f896b (DEFAULT_TABLE table=default.Table_4066ae2ab4534ff58ac51f74bef05db9 [id=653cef93edf64ea5aad4122799ed9082]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:41.706918 10946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4637237acb184a6aaae94a04218f896b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:41.717264 11727 ts_tablet_manager.cc:1918] T 05e2c7f9820f41679c4d58343ff3f829 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:41.717643 11727 log.cc:1196] T 05e2c7f9820f41679c4d58343ff3f829 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/05e2c7f9820f41679c4d58343ff3f829
I20260925 09:57:41.718286 11727 ts_tablet_manager.cc:1939] T 05e2c7f9820f41679c4d58343ff3f829 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:41.727187 11084 tablet_bootstrap.cc:654] T c47783e4ea694561bf008def26655884 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:41.543798 12603 raft_consensus.cc:482] T 9744b9814b69470396ce039348fe5e45 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:41.544068 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2015c10f503e4751ad9ff77404fca50b" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:41.546087 12646 raft_consensus.cc:482] T a81530200cf14717bc7cbf67bb73f4e5 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:41.728072 12603 raft_consensus.cc:504] T 9744b9814b69470396ce039348fe5e45 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.728607 12646 raft_consensus.cc:504] T a81530200cf14717bc7cbf67bb73f4e5 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.729964 12603 leader_election.cc:288] T 9744b9814b69470396ce039348fe5e45 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:41.617669 10775 leader_election.cc:302] T 3039ec3febf848dba7c4e6fd38bb8053 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters: be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:41.730406 12646 leader_election.cc:288] T a81530200cf14717bc7cbf67bb73f4e5 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
W20260925 09:57:41.730782 10775 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 113209us
I20260925 09:57:41.635336 10812 tablet_service.cc:1467] Processing CreateTablet for tablet 53ccf068342d44678def17a72be39d97 (DEFAULT_TABLE table=default.Table_c8324101e1254bf2ad787feab7261dae [id=79c1c015c2534e57911c9d9ac26bb03e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:41.654006 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d84c717b6ac462eb331e25d80c918e4" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318"
I20260925 09:57:41.680774 10809 tablet_service.cc:1467] Processing CreateTablet for tablet 9eef15f91ad842dda58ff133366387fe (DEFAULT_TABLE table=default.Table_3a4caa0a9b2044689a5696cf420f9bcd [id=956e11b71c9e44ffa1243608fe0bfeac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:41.683257 10813 tablet_service.cc:1467] Processing CreateTablet for tablet 4637237acb184a6aaae94a04218f896b (DEFAULT_TABLE table=default.Table_4066ae2ab4534ff58ac51f74bef05db9 [id=653cef93edf64ea5aad4122799ed9082]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:41.738101 12523 raft_consensus.cc:482] T 37bab60afe7a4c758a08ef7c9c24aba0 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b8850818e2ee42a082802d0095c08d21)
I20260925 09:57:41.738571 12523 raft_consensus.cc:504] T 37bab60afe7a4c758a08ef7c9c24aba0 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:41.738924 10918 leader_election.cc:341] T 2015c10f503e4751ad9ff77404fca50b P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:41.739276 10918 leader_election.cc:302] T 2015c10f503e4751ad9ff77404fca50b P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:41.740417 12523 leader_election.cc:288] T 37bab60afe7a4c758a08ef7c9c24aba0 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:41.742340 10983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9744b9814b69470396ce039348fe5e45" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:41.647524 10824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d84c717b6ac462eb331e25d80c918e4" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:41.730847 12646 raft_consensus.cc:2781] T 3039ec3febf848dba7c4e6fd38bb8053 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:41.744732 12646 raft_consensus.cc:482] T 3039ec3febf848dba7c4e6fd38bb8053 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:41.745069 12646 raft_consensus.cc:3037] T 3039ec3febf848dba7c4e6fd38bb8053 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:57:41.748876 10772 leader_election.cc:341] T a81530200cf14717bc7cbf67bb73f4e5 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: a81530200cf14717bc7cbf67bb73f4e5
W20260925 09:57:41.755604 10772 leader_election.cc:341] T 37bab60afe7a4c758a08ef7c9c24aba0 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 37bab60afe7a4c758a08ef7c9c24aba0
I20260925 09:57:41.759195 10388 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 25ac35fb43d34c9b85b8e61ad6e386a6 (table default.Table_2672598ccbce42eb8f11019f47a6fdda [id=887ce3cafef64210bb69d424ed5dae8a]). Expected version 2 got 0
I20260925 09:57:41.759912 10388 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet f0bf545f70ae4cd6a1ad2540c7158ecc (table default.Table_25edfe533c64421da7700ac5d60265c3 [id=1469f44b239d454caa97056bd5803255]). Expected version 1 got 0
I20260925 09:57:41.766537 12659 raft_consensus.cc:2726] T 2015c10f503e4751ad9ff77404fca50b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:41.773376 12646 raft_consensus.cc:504] T 3039ec3febf848dba7c4e6fd38bb8053 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.773005 10388 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet c52745a8cdee461aba7485b0531245d6 (table default.Table_8e186a99ccba425b9b588041d2364a40 [id=2d6bc0e2aaee4b1483b01fc678505bb0]). Expected version 2 got 0
I20260925 09:57:41.784235 11084 tablet_bootstrap.cc:492] T c47783e4ea694561bf008def26655884 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:41.784665 11084 ts_tablet_manager.cc:1397] T c47783e4ea694561bf008def26655884 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.090s user 0.008s sys 0.008s
I20260925 09:57:41.787818 10388 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 572c83b4a7c84fdab1fdbb0fbdb0a53d (table default.Table_8398f7e9c7a64f688bd18521e99cb669 [id=b8b7198fdaa24d72ae13b7275685c555]). Expected version 1 got 0
I20260925 09:57:41.788534 10388 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet cc295f7a9169475f9a9235e657be629f (table default.Table_168840855e0746f7ab027c1900842751 [id=99ba81c4d0dd497398e048a05759419a]). Expected version 1 got 0
W20260925 09:57:41.789785 10772 leader_election.cc:341] T 3039ec3febf848dba7c4e6fd38bb8053 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 3039ec3febf848dba7c4e6fd38bb8053
I20260925 09:57:41.790851 10388 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 83b0cf21f2a144719a2e13990e1921c0 (table default.Table_36cd59e3f299423fa9f1a2714df4454c [id=e176feb88d2643cf85a1f1d821133b6c]). Expected version 1 got 0
I20260925 09:57:41.791277 12646 leader_election.cc:288] T 3039ec3febf848dba7c4e6fd38bb8053 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:41.786898 11084 raft_consensus.cc:348] T c47783e4ea694561bf008def26655884 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.801457 11084 raft_consensus.cc:374] T c47783e4ea694561bf008def26655884 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:41.801712 11084 raft_consensus.cc:729] T c47783e4ea694561bf008def26655884 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:41.804364 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 415) took 2283 ms. Trace:
I20260925 09:57:41.804674 11727 rpcz_store.cc:276] 0925 09:57:39.520848 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:39.522526 (+ 1678us) service_pool.cc:227] Handling call
0925 09:57:41.543955 (+2021429us) ts_tablet_manager.cc:1145] Deleting tablet 05e2c7f9820f41679c4d58343ff3f829
0925 09:57:41.550118 (+ 6163us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:41.674665 (+124547us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:41.717036 (+ 42371us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:41.804304 (+ 87268us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2020804}
I20260925 09:57:41.805779 11727 tablet_replica.cc:330] T a8be01fe42894b96bbdd5ce2403ff1f4 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:41.806609 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 05e2c7f9820f41679c4d58343ff3f829 (table default.Table_4066ae2ab4534ff58ac51f74bef05db9 [id=653cef93edf64ea5aad4122799ed9082]) successfully deleted
I20260925 09:57:41.802304 11084 consensus_queue.cc:260] T c47783e4ea694561bf008def26655884 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.812686 11084 ts_tablet_manager.cc:1428] T c47783e4ea694561bf008def26655884 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.028s user 0.005s sys 0.001s
I20260925 09:57:41.813531 11084 tablet_bootstrap.cc:492] T d5f28d1c19eb4e849dfa6b0bf532b67e P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:41.633011 10619 leader_election.cc:302] T 3d84c717b6ac462eb331e25d80c918e4 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
W20260925 09:57:41.825994 10619 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 193050us
I20260925 09:57:41.826097 12612 raft_consensus.cc:2781] T 3d84c717b6ac462eb331e25d80c918e4 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:41.638178 12379 ts_tablet_manager.cc:1905] T 05e2c7f9820f41679c4d58343ff3f829 P b8850818e2ee42a082802d0095c08d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:41.649281 12626 raft_consensus.cc:482] T f9c259867cf44c4c90e50864475a4970 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:41.651104 10679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69028aa7908c45b896da4ba8deef07b1" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:41.664825 10654 tablet_service.cc:1467] Processing CreateTablet for tablet 53ccf068342d44678def17a72be39d97 (DEFAULT_TABLE table=default.Table_c8324101e1254bf2ad787feab7261dae [id=79c1c015c2534e57911c9d9ac26bb03e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:41.829715 10679 raft_consensus.cc:2454] T 69028aa7908c45b896da4ba8deef07b1 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
I20260925 09:57:41.830158 10654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53ccf068342d44678def17a72be39d97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:41.830542 12626 raft_consensus.cc:504] T f9c259867cf44c4c90e50864475a4970 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.682518 10658 tablet_service.cc:1467] Processing CreateTablet for tablet 9eef15f91ad842dda58ff133366387fe (DEFAULT_TABLE table=default.Table_3a4caa0a9b2044689a5696cf420f9bcd [id=956e11b71c9e44ffa1243608fe0bfeac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:41.832007 10658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eef15f91ad842dda58ff133366387fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:41.832139 12612 raft_consensus.cc:686] T 3d84c717b6ac462eb331e25d80c918e4 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:41.832818 12626 leader_election.cc:288] T f9c259867cf44c4c90e50864475a4970 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:41.832903 12612 consensus_queue.cc:237] T 3d84c717b6ac462eb331e25d80c918e4 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.702242 10653 tablet_service.cc:1467] Processing CreateTablet for tablet 4637237acb184a6aaae94a04218f896b (DEFAULT_TABLE table=default.Table_4066ae2ab4534ff58ac51f74bef05db9 [id=653cef93edf64ea5aad4122799ed9082]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:57:41.836256 10653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4637237acb184a6aaae94a04218f896b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:41.703305 12664 tablet.cc:1722] T 83b0cf21f2a144719a2e13990e1921c0 P b8850818e2ee42a082802d0095c08d21: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:57:41.845813 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c259867cf44c4c90e50864475a4970" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:41.846343 12626 consensus_queue.cc:1035] T d54ab15dd054489ab50b7f0b26b79a50 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:57:41.846558 10976 raft_consensus.cc:2454] T f9c259867cf44c4c90e50864475a4970 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 0.
I20260925 09:57:41.848848 10619 leader_election.cc:302] T f9c259867cf44c4c90e50864475a4970 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:41.854586 12612 raft_consensus.cc:2781] T f9c259867cf44c4c90e50864475a4970 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:41.855374 12612 raft_consensus.cc:482] T f9c259867cf44c4c90e50864475a4970 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:41.856122 12612 raft_consensus.cc:3037] T f9c259867cf44c4c90e50864475a4970 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:41.856834 10813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4637237acb184a6aaae94a04218f896b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:41.860224 10812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53ccf068342d44678def17a72be39d97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:41.872085 10809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eef15f91ad842dda58ff133366387fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:57:41.796343 11082 ts_tablet_manager.cc:1428] T 54157e776efa4c6c8e2bd4c58321fcda P b8850818e2ee42a082802d0095c08d21: Time spent starting tablet: real 0.301s user 0.003s sys 0.003s
I20260925 09:57:41.874635 11082 tablet_bootstrap.cc:492] T b3d9a3cf5bda4cf6bc1fa3fae1cb9288 P b8850818e2ee42a082802d0095c08d21: Bootstrap starting.
I20260925 09:57:41.884707 11085 tablet_bootstrap.cc:492] T 1e76f9e0ba554e1cad0cf18863393608 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:41.885075 11085 ts_tablet_manager.cc:1397] T 1e76f9e0ba554e1cad0cf18863393608 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.379s user 0.006s sys 0.001s
W20260925 09:57:41.887256 10772 consensus_peers.cc:487] T a8be01fe42894b96bbdd5ce2403ff1f4 P c43461c4ee854ae1ac52b873ccada318 -> Peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Couldn't send request to peer be97ab5d75934a648d4ab76341193ec0. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:41.894163 11082 tablet_bootstrap.cc:654] T b3d9a3cf5bda4cf6bc1fa3fae1cb9288 P b8850818e2ee42a082802d0095c08d21: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:41.898926 12027 ts_tablet_manager.cc:1918] T 7a31b999f0fd447eacff13de71aa3fff P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260925 09:57:41.899995 10621 leader_election.cc:341] T 3d84c717b6ac462eb331e25d80c918e4 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:41.902366 11084 tablet_bootstrap.cc:654] T d5f28d1c19eb4e849dfa6b0bf532b67e P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:41.917062 10621 leader_election.cc:341] T 3d84c717b6ac462eb331e25d80c918e4 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:41.900337 12612 raft_consensus.cc:504] T f9c259867cf44c4c90e50864475a4970 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
W20260925 09:57:41.907501 10775 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 47774.9us
I20260925 09:57:41.925544 11085 raft_consensus.cc:348] T 1e76f9e0ba554e1cad0cf18863393608 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.926208 11085 raft_consensus.cc:374] T 1e76f9e0ba554e1cad0cf18863393608 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:41.926474 11085 raft_consensus.cc:729] T 1e76f9e0ba554e1cad0cf18863393608 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:41.926297 10775 leader_election.cc:302] T 69028aa7908c45b896da4ba8deef07b1 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318; no voters: be97ab5d75934a648d4ab76341193ec0
I20260925 09:57:41.936683 12523 raft_consensus.cc:2781] T 69028aa7908c45b896da4ba8deef07b1 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:41.937076 12523 raft_consensus.cc:482] T 69028aa7908c45b896da4ba8deef07b1 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:41.937456 12523 raft_consensus.cc:3037] T 69028aa7908c45b896da4ba8deef07b1 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:41.941089 11727 raft_consensus.cc:2229] T a8be01fe42894b96bbdd5ce2403ff1f4 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:41.941711 11727 raft_consensus.cc:2258] T a8be01fe42894b96bbdd5ce2403ff1f4 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:41.927035 11085 consensus_queue.cc:260] T 1e76f9e0ba554e1cad0cf18863393608 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.943707 11727 ts_tablet_manager.cc:1905] T a8be01fe42894b96bbdd5ce2403ff1f4 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:41.944128 11084 tablet_bootstrap.cc:492] T d5f28d1c19eb4e849dfa6b0bf532b67e P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:41.944495 11084 ts_tablet_manager.cc:1397] T d5f28d1c19eb4e849dfa6b0bf532b67e P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.131s user 0.005s sys 0.003s
I20260925 09:57:41.945205 11085 ts_tablet_manager.cc:1428] T 1e76f9e0ba554e1cad0cf18863393608 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.060s user 0.003s sys 0.003s
I20260925 09:57:41.945982 11085 tablet_bootstrap.cc:492] T 9a73a1f242e7430cbf52229c07b4dc82 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:41.946555 11084 raft_consensus.cc:348] T d5f28d1c19eb4e849dfa6b0bf532b67e P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.947144 11084 raft_consensus.cc:374] T d5f28d1c19eb4e849dfa6b0bf532b67e P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:41.947897 12027 log.cc:1196] T 7a31b999f0fd447eacff13de71aa3fff P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/7a31b999f0fd447eacff13de71aa3fff
I20260925 09:57:41.928372 10824 raft_consensus.cc:3037] T 9744b9814b69470396ce039348fe5e45 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:41.948704 12027 ts_tablet_manager.cc:1939] T 7a31b999f0fd447eacff13de71aa3fff P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:41.950716 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 393) took 3226 ms. Trace:
I20260925 09:57:41.930385 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c259867cf44c4c90e50864475a4970" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:41.955109 12612 leader_election.cc:288] T f9c259867cf44c4c90e50864475a4970 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Requested vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:41.947366 11084 raft_consensus.cc:729] T d5f28d1c19eb4e849dfa6b0bf532b67e P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:41.973215 11084 consensus_queue.cc:260] T d5f28d1c19eb4e849dfa6b0bf532b67e P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:41.950985 12027 rpcz_store.cc:276] 0925 09:57:38.732721 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.732818 (+ 97us) service_pool.cc:227] Handling call
0925 09:57:41.326154 (+2593336us) ts_tablet_manager.cc:1145] Deleting tablet 7a31b999f0fd447eacff13de71aa3fff
0925 09:57:41.327661 (+ 1507us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:41.638909 (+311248us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:41.898711 (+259802us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:41.950663 (+ 51952us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":43566,"spinlock_wait_cycles":5376,"tablet-delete.queue_time_us":2582470}
I20260925 09:57:41.979878 12027 tablet_replica.cc:330] T f62c62715c9b42de826c1a66fd431eab P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:41.980963 12027 raft_consensus.cc:2229] T f62c62715c9b42de826c1a66fd431eab P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:41.981040 10983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9c259867cf44c4c90e50864475a4970" candidate_uuid: "b8850818e2ee42a082802d0095c08d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:41.981467 12027 raft_consensus.cc:2258] T f62c62715c9b42de826c1a66fd431eab P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:41.981662 10983 raft_consensus.cc:3037] T f9c259867cf44c4c90e50864475a4970 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:41.983767 12027 ts_tablet_manager.cc:1905] T f62c62715c9b42de826c1a66fd431eab P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:41.984454 11085 tablet_bootstrap.cc:654] T 9a73a1f242e7430cbf52229c07b4dc82 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:41.996430 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 7a31b999f0fd447eacff13de71aa3fff (table default.Table_489e2c74ed6249ce9cf54541e0a491cd [id=8d2c4997954141129ebbec9e72658d90]) successfully deleted
I20260925 09:57:42.005599 12671 raft_consensus.cc:482] T 1e76f9e0ba554e1cad0cf18863393608 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:42.006062 12671 raft_consensus.cc:504] T 1e76f9e0ba554e1cad0cf18863393608 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.008464 12671 leader_election.cc:288] T 1e76f9e0ba554e1cad0cf18863393608 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:41.966115 10390 catalog_manager.cc:5495] T d54ab15dd054489ab50b7f0b26b79a50 P b8850818e2ee42a082802d0095c08d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8850818e2ee42a082802d0095c08d21 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b8850818e2ee42a082802d0095c08d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } health_report { overall_health: UNKNOWN } } }
W20260925 09:57:42.010080 10772 leader_election.cc:341] T 1e76f9e0ba554e1cad0cf18863393608 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 1e76f9e0ba554e1cad0cf18863393608
I20260925 09:57:41.988364 12523 raft_consensus.cc:504] T 69028aa7908c45b896da4ba8deef07b1 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.014825 12523 leader_election.cc:288] T 69028aa7908c45b896da4ba8deef07b1 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
W20260925 09:57:42.016816 10772 leader_election.cc:341] T 69028aa7908c45b896da4ba8deef07b1 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 69028aa7908c45b896da4ba8deef07b1
I20260925 09:57:42.020784 10673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9744b9814b69470396ce039348fe5e45" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:42.025437 12659 raft_consensus.cc:482] T c47783e4ea694561bf008def26655884 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:42.025930 12659 raft_consensus.cc:504] T c47783e4ea694561bf008def26655884 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.028921 12659 leader_election.cc:288] T c47783e4ea694561bf008def26655884 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:42.030262 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c47783e4ea694561bf008def26655884" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:42.031545 10918 leader_election.cc:341] T c47783e4ea694561bf008def26655884 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:42.032827 10679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c47783e4ea694561bf008def26655884" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:42.034381 11082 tablet_bootstrap.cc:492] T b3d9a3cf5bda4cf6bc1fa3fae1cb9288 P b8850818e2ee42a082802d0095c08d21: No bootstrap required, opened a new log
W20260925 09:57:42.034442 10919 leader_election.cc:341] T c47783e4ea694561bf008def26655884 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:42.034803 11082 ts_tablet_manager.cc:1397] T b3d9a3cf5bda4cf6bc1fa3fae1cb9288 P b8850818e2ee42a082802d0095c08d21: Time spent bootstrapping tablet: real 0.160s user 0.006s sys 0.005s
I20260925 09:57:42.034766 10919 leader_election.cc:302] T c47783e4ea694561bf008def26655884 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:42.022514 10673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a81530200cf14717bc7cbf67bb73f4e5" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:42.035871 10673 raft_consensus.cc:2454] T a81530200cf14717bc7cbf67bb73f4e5 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
I20260925 09:57:42.006488 10983 raft_consensus.cc:2454] T f9c259867cf44c4c90e50864475a4970 P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8850818e2ee42a082802d0095c08d21 in term 1.
I20260925 09:57:42.036705 12659 raft_consensus.cc:2726] T c47783e4ea694561bf008def26655884 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:42.042999 10673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3039ec3febf848dba7c4e6fd38bb8053" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:42.043654 10673 raft_consensus.cc:3037] T 3039ec3febf848dba7c4e6fd38bb8053 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:42.043678 10385 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: default.Table_8180842f664042b0a9073b7008438ff7 [id=416a4b989570457e919fb7e7b16a017d] alter complete (version 1)
I20260925 09:57:42.046895 10391 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: Sending DeleteTablet for 3 replicas of tablet 26fae3fbfe374f348a1729c21ec751b9
I20260925 09:57:42.048297 10391 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:35140 (request call id 386) took 3292 ms. Trace:
I20260925 09:57:42.048470 10619 leader_election.cc:302] T f9c259867cf44c4c90e50864475a4970 P b8850818e2ee42a082802d0095c08d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8850818e2ee42a082802d0095c08d21, be97ab5d75934a648d4ab76341193ec0; no voters:
I20260925 09:57:42.049108 12626 raft_consensus.cc:2781] T f9c259867cf44c4c90e50864475a4970 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:42.049486 12626 raft_consensus.cc:686] T f9c259867cf44c4c90e50864475a4970 P b8850818e2ee42a082802d0095c08d21 [term 1 LEADER]: Becoming Leader. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Running, Role: LEADER
I20260925 09:57:42.050233 12626 consensus_queue.cc:237] T f9c259867cf44c4c90e50864475a4970 P b8850818e2ee42a082802d0095c08d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.054909 10946 tablet_service.cc:1514] Processing DeleteTablet for tablet 26fae3fbfe374f348a1729c21ec751b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:39 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:42.055028 10806 tablet_service.cc:1514] Processing DeleteTablet for tablet 26fae3fbfe374f348a1729c21ec751b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:39 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:42.054455 10673 raft_consensus.cc:2454] T 3039ec3febf848dba7c4e6fd38bb8053 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:42.057017 11084 ts_tablet_manager.cc:1428] T d5f28d1c19eb4e849dfa6b0bf532b67e P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.112s user 0.003s sys 0.004s
I20260925 09:57:42.058715 10653 tablet_service.cc:1514] Processing DeleteTablet for tablet 26fae3fbfe374f348a1729c21ec751b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:39 UTC) from {username='slave'} at 127.0.0.1:37876
I20260925 09:57:42.059332 11082 raft_consensus.cc:348] T b3d9a3cf5bda4cf6bc1fa3fae1cb9288 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.059999 11082 raft_consensus.cc:374] T b3d9a3cf5bda4cf6bc1fa3fae1cb9288 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:42.060225 11082 raft_consensus.cc:729] T b3d9a3cf5bda4cf6bc1fa3fae1cb9288 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8850818e2ee42a082802d0095c08d21, State: Initialized, Role: FOLLOWER
I20260925 09:57:42.060756 11082 consensus_queue.cc:260] T b3d9a3cf5bda4cf6bc1fa3fae1cb9288 P b8850818e2ee42a082802d0095c08d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.044245 10384 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: default.Table_0db927e92a3b4970a7efea3fb765e65e [id=a988264fe43c4eb88a84f8b6359e77a6] alter complete (version 1)
I20260925 09:57:42.069407 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e76f9e0ba554e1cad0cf18863393608" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:42.070020 10684 raft_consensus.cc:2454] T 1e76f9e0ba554e1cad0cf18863393608 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
I20260925 09:57:42.071274 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69028aa7908c45b896da4ba8deef07b1" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21"
I20260925 09:57:42.071851 10684 raft_consensus.cc:3037] T 69028aa7908c45b896da4ba8deef07b1 P b8850818e2ee42a082802d0095c08d21 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:42.078886 10684 raft_consensus.cc:2454] T 69028aa7908c45b896da4ba8deef07b1 P b8850818e2ee42a082802d0095c08d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:42.084735 12659 raft_consensus.cc:482] T 4a6b1284cddc4b2581931d8895212ca6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:42.085184 12659 raft_consensus.cc:504] T 4a6b1284cddc4b2581931d8895212ca6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.087025 10393 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: default.Table_82a8d1df379d442e9333fd4ab294738c [id=0040224e69114e918cec485791935895] alter complete (version 1)
I20260925 09:57:42.089740 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a6b1284cddc4b2581931d8895212ca6" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:42.087297 12659 leader_election.cc:288] T 4a6b1284cddc4b2581931d8895212ca6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
I20260925 09:57:42.090780 11084 tablet_bootstrap.cc:492] T bc1d7b25d6ba42bc91130927c32d4ae6 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
W20260925 09:57:42.091048 10918 leader_election.cc:341] T 4a6b1284cddc4b2581931d8895212ca6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:42.097164 12659 raft_consensus.cc:482] T 5d6ebe0c2ccc48a09b616816953d83fe P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:42.097596 12659 raft_consensus.cc:504] T 5d6ebe0c2ccc48a09b616816953d83fe P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.100510 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d6ebe0c2ccc48a09b616816953d83fe" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:42.101704 10918 leader_election.cc:341] T 5d6ebe0c2ccc48a09b616816953d83fe P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:42.102639 10684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a6b1284cddc4b2581931d8895212ca6" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8850818e2ee42a082802d0095c08d21" is_pre_election: true
I20260925 09:57:42.103044 12659 leader_election.cc:288] T 5d6ebe0c2ccc48a09b616816953d83fe P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:42.048578 10391 rpcz_store.cc:276] 0925 09:57:38.756119 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:39.262316 (+506197us) service_pool.cc:227] Handling call
0925 09:57:39.290025 (+ 27709us) spinlock_profiling.cc:243] Waited 26.6 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2adb466 00007fabf061cb34 00007fabf061cac2 00007fabf061c9f1 00007fabf061c96c 00007fabf061bc22 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:39.548035 (+258010us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:57:39.548583 (+ 548us) catalog_manager.cc:2752] Modifying in-memory table state
0925 09:57:39.548707 (+ 124us) catalog_manager.cc:2761] Locking tablets
0925 09:57:39.819450 (+270743us) catalog_manager.cc:2775] Removing table and tablets from system table
0925 09:57:39.835546 (+ 16096us) catalog_manager.cc:2793] Removing table from by-name map
0925 09:57:39.835774 (+ 228us) catalog_manager.cc:2809] Committing in-memory state
0925 09:57:42.046811 (+2211037us) catalog_manager.cc:2813] Aborting table tasks
0925 09:57:42.048230 (+ 1419us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:39.821166 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:39.821604 (+ 438us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:39.821631 (+ 27us) write_op.cc:435] Acquired schema lock
0925 09:57:39.821647 (+ 16us) tablet.cc:660] Decoding operations
0925 09:57:39.822003 (+ 356us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:39.822049 (+ 46us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:39.822073 (+ 24us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:57:39.822368 (+ 295us) tablet.cc:699] Row locks acquired
0925 09:57:39.822384 (+ 16us) write_op.cc:260] PREPARE: finished
0925 09:57:39.822495 (+ 111us) write_op.cc:270] Start()
0925 09:57:39.822582 (+ 87us) write_op.cc:276] Timestamp: P: 1790330259822446 usec, L: 0
0925 09:57:39.822600 (+ 18us) op_driver.cc:331] REPLICATION: starting
0925 09:57:39.823093 (+ 493us) log.cc:842] Serialized 762 byte log entry
0925 09:57:39.833029 (+ 9936us) op_driver.cc:444] REPLICATION: finished
0925 09:57:39.833957 (+ 928us) write_op.cc:301] APPLY: starting
0925 09:57:39.834006 (+ 49us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:39.834110 (+ 104us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:39.834202 (+ 92us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:39.834478 (+ 276us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:39.834611 (+ 133us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0925 09:57:39.834653 (+ 42us) write_op.cc:312] APPLY: finished
0925 09:57:39.834822 (+ 169us) log.cc:842] Serialized 29 byte log entry
0925 09:57:39.834961 (+ 139us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:39.835028 (+ 67us) write_op.cc:454] Released schema lock
0925 09:57:39.835063 (+ 35us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:39.835223 (+ 160us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:39.835247 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":61089152,"child_traces":[["op",{"apply.queue_time_us":776,"apply.run_cpu_time_us":1509,"apply.run_wall_time_us":4162,"num_ops":2,"prepare.queue_time_us":101,"prepare.run_cpu_time_us":2231,"prepare.run_wall_time_us":4032,"raft.queue_time_us":80,"raft.run_cpu_time_us":601,"raft.run_wall_time_us":601,"replication_time_us":10359}]]}
I20260925 09:57:42.110558 10276 master-stress-test.cc:580] Killed test thread 7/10
I20260925 09:57:42.110911 10276 master-stress-test.cc:578] Killing test thread 8/10
I20260925 09:57:42.111205 10276 master-stress-test.cc:580] Killed test thread 8/10
I20260925 09:57:42.111428 10276 master-stress-test.cc:578] Killing test thread 9/10
I20260925 09:57:42.111694 10276 master-stress-test.cc:580] Killed test thread 9/10
I20260925 09:57:42.111879 10276 master-stress-test.cc:578] Killing test thread 10/10
I20260925 09:57:42.112186 10276 master-stress-test.cc:580] Killed test thread 10/10
I20260925 09:57:42.112530 10276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu with pid 10561
I20260925 09:57:42.128453 11084 tablet_bootstrap.cc:654] T bc1d7b25d6ba42bc91130927c32d4ae6 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:42.129606 12659 raft_consensus.cc:482] T 30e04d90c962442cadf7bdec6f827dac P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:42.130043 12659 raft_consensus.cc:504] T 30e04d90c962442cadf7bdec6f827dac P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.132036 12659 leader_election.cc:288] T 30e04d90c962442cadf7bdec6f827dac P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:42.133325 10824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30e04d90c962442cadf7bdec6f827dac" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:42.134394 10918 leader_election.cc:341] T 30e04d90c962442cadf7bdec6f827dac P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:42.161170 11084 tablet_bootstrap.cc:492] T bc1d7b25d6ba42bc91130927c32d4ae6 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:42.161655 11084 ts_tablet_manager.cc:1397] T bc1d7b25d6ba42bc91130927c32d4ae6 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.071s user 0.005s sys 0.004s
I20260925 09:57:42.167099 10813 tablet_service.cc:1514] Processing DeleteTablet for tablet 545ac025f2cb469da847979011a0b3ce with delete_type TABLET_DATA_DELETED (replaced by tablet ab6bdba5ca5d41ae921a90c1f77006c5) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:42.173688 11084 raft_consensus.cc:348] T bc1d7b25d6ba42bc91130927c32d4ae6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.176860 11084 raft_consensus.cc:374] T bc1d7b25d6ba42bc91130927c32d4ae6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:42.177222 11084 raft_consensus.cc:729] T bc1d7b25d6ba42bc91130927c32d4ae6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:42.178987 11084 consensus_queue.cc:260] T bc1d7b25d6ba42bc91130927c32d4ae6 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.191681 11084 ts_tablet_manager.cc:1428] T bc1d7b25d6ba42bc91130927c32d4ae6 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.030s user 0.007s sys 0.001s
I20260925 09:57:42.193307 11084 tablet_bootstrap.cc:492] T ac8230d622214346bc7412bfc6cdb203 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:42.204692 11085 tablet_bootstrap.cc:492] T 9a73a1f242e7430cbf52229c07b4dc82 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:42.205137 11085 ts_tablet_manager.cc:1397] T 9a73a1f242e7430cbf52229c07b4dc82 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.259s user 0.013s sys 0.002s
I20260925 09:57:42.208189 10813 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a31b999f0fd447eacff13de71aa3fff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:23 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:42.209582 10813 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e3120347c8e4a3eab57d6562ae40e8d with delete_type TABLET_DATA_DELETED (replaced by tablet c52745a8cdee461aba7485b0531245d6) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:42.209798 11727 ts_tablet_manager.cc:1918] T a8be01fe42894b96bbdd5ce2403ff1f4 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:42.210148 11727 log.cc:1196] T a8be01fe42894b96bbdd5ce2403ff1f4 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/a8be01fe42894b96bbdd5ce2403ff1f4
I20260925 09:57:42.210898 11727 ts_tablet_manager.cc:1939] T a8be01fe42894b96bbdd5ce2403ff1f4 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:42.211097 10813 tablet_service.cc:1514] Processing DeleteTablet for tablet 5608c5f9b65b4da7b2fbb56417c92d88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:36 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:42.214099 10813 tablet_service.cc:1514] Processing DeleteTablet for tablet 93e1b5837aff48d7907890c40dfcd0d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:32 UTC) from {username='slave'} at 127.0.0.1:33344
W20260925 09:57:42.214970 10384 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.2:40761 (request call id 46) took 2136 ms (client timeout 1000 ms). Trace:
W20260925 09:57:42.215335 10384 rpcz_store.cc:269] 0925 09:57:40.078475 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:40.078600 (+ 125us) service_pool.cc:227] Handling call
0925 09:57:42.214892 (+2136292us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:40.483649 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:40.484088 (+ 439us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:40.484116 (+ 28us) write_op.cc:435] Acquired schema lock
0925 09:57:40.484132 (+ 16us) tablet.cc:660] Decoding operations
0925 09:57:40.484427 (+ 295us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:40.484472 (+ 45us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:40.484496 (+ 24us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:40.484652 (+ 156us) tablet.cc:699] Row locks acquired
0925 09:57:40.484665 (+ 13us) write_op.cc:260] PREPARE: finished
0925 09:57:40.484916 (+ 251us) write_op.cc:270] Start()
0925 09:57:40.485012 (+ 96us) write_op.cc:276] Timestamp: P: 1790330260484832 usec, L: 0
0925 09:57:40.485030 (+ 18us) op_driver.cc:331] REPLICATION: starting
0925 09:57:40.485533 (+ 503us) log.cc:842] Serialized 356 byte log entry
0925 09:57:40.492177 (+ 6644us) op_driver.cc:444] REPLICATION: finished
0925 09:57:40.492420 (+ 243us) write_op.cc:301] APPLY: starting
0925 09:57:40.492473 (+ 53us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:40.492556 (+ 83us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:40.492641 (+ 85us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:40.492899 (+ 258us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:40.493016 (+ 117us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:40.493056 (+ 40us) write_op.cc:312] APPLY: finished
0925 09:57:40.493200 (+ 144us) log.cc:842] Serialized 23 byte log entry
0925 09:57:40.493321 (+ 121us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:40.493388 (+ 67us) write_op.cc:454] Released schema lock
0925 09:57:40.493415 (+ 27us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:40.493562 (+ 147us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:40.493584 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":58367,"reported_tablets":29,"child_traces":[["op",{"apply.queue_time_us":85,"apply.run_cpu_time_us":1301,"apply.run_wall_time_us":1301,"num_ops":1,"prepare.queue_time_us":4576,"prepare.run_cpu_time_us":2203,"prepare.run_wall_time_us":2203,"raft.queue_time_us":166,"raft.run_cpu_time_us":580,"raft.run_wall_time_us":579,"replication_time_us":7079}]]}
I20260925 09:57:42.213908 11084 tablet_bootstrap.cc:654] T ac8230d622214346bc7412bfc6cdb203 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:42.237903 10813 tablet_service.cc:1514] Processing DeleteTablet for tablet 350904aa5957496d90a2aa2e637b591c with delete_type TABLET_DATA_DELETED (replaced by tablet 154083d32e0f474f9e3a02e6cca940ef) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:42.238818 10813 tablet_service.cc:1514] Processing DeleteTablet for tablet 61aa6380d30a4b21b70ee2d321247bd8 with delete_type TABLET_DATA_DELETED (replaced by tablet 77ebb59faf3e4fb187bbce29612c25ea) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:42.248154 10812 tablet_service.cc:1514] Processing DeleteTablet for tablet cf775c9d9b114ddcb97dc6c6eca8e0fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:26 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:42.252851 11727 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53560 (request call id 420) took 1414 ms. Trace:
I20260925 09:57:42.253257 11727 rpcz_store.cc:276] 0925 09:57:40.838639 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:40.838744 (+ 105us) service_pool.cc:227] Handling call
0925 09:57:41.805642 (+966898us) ts_tablet_manager.cc:1145] Deleting tablet a8be01fe42894b96bbdd5ce2403ff1f4
0925 09:57:41.805681 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:41.972607 (+166926us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:42.209592 (+236985us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:42.252789 (+ 43197us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":134534,"tablet-delete.queue_time_us":966385}
I20260925 09:57:42.254757 11727 tablet_replica.cc:330] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:42.275086 11084 tablet_bootstrap.cc:492] T ac8230d622214346bc7412bfc6cdb203 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:42.275545 11084 ts_tablet_manager.cc:1397] T ac8230d622214346bc7412bfc6cdb203 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.082s user 0.008s sys 0.002s
I20260925 09:57:42.280066 12027 ts_tablet_manager.cc:1918] T f62c62715c9b42de826c1a66fd431eab P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:42.280603 12027 log.cc:1196] T f62c62715c9b42de826c1a66fd431eab P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/f62c62715c9b42de826c1a66fd431eab
I20260925 09:57:42.283277 12027 ts_tablet_manager.cc:1939] T f62c62715c9b42de826c1a66fd431eab P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:42.289083 11085 raft_consensus.cc:348] T 9a73a1f242e7430cbf52229c07b4dc82 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.289830 11085 raft_consensus.cc:374] T 9a73a1f242e7430cbf52229c07b4dc82 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:42.277571 11084 raft_consensus.cc:348] T ac8230d622214346bc7412bfc6cdb203 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.290086 11085 raft_consensus.cc:729] T 9a73a1f242e7430cbf52229c07b4dc82 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:42.290126 11084 raft_consensus.cc:374] T ac8230d622214346bc7412bfc6cdb203 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:42.290374 11084 raft_consensus.cc:729] T ac8230d622214346bc7412bfc6cdb203 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:42.290455 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 399) took 3082 ms. Trace:
W20260925 09:57:42.292186 10392 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.1:56563 (request call id 46) took 2223 ms (client timeout 1000 ms). Trace:
W20260925 09:57:42.292555 10392 rpcz_store.cc:269] 0925 09:57:40.068956 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:40.069092 (+ 136us) service_pool.cc:227] Handling call
0925 09:57:42.292123 (+2223031us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:41.293397 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:41.293801 (+ 404us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:41.293830 (+ 29us) write_op.cc:435] Acquired schema lock
0925 09:57:41.293862 (+ 32us) tablet.cc:660] Decoding operations
0925 09:57:41.294172 (+ 310us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:41.294213 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:41.294241 (+ 28us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:41.294406 (+ 165us) tablet.cc:699] Row locks acquired
0925 09:57:41.294422 (+ 16us) write_op.cc:260] PREPARE: finished
0925 09:57:41.294539 (+ 117us) write_op.cc:270] Start()
0925 09:57:41.294633 (+ 94us) write_op.cc:276] Timestamp: P: 1790330261294504 usec, L: 0
0925 09:57:41.294654 (+ 21us) op_driver.cc:331] REPLICATION: starting
0925 09:57:41.295217 (+ 563us) log.cc:842] Serialized 524 byte log entry
0925 09:57:41.302947 (+ 7730us) op_driver.cc:444] REPLICATION: finished
0925 09:57:41.303356 (+ 409us) write_op.cc:301] APPLY: starting
0925 09:57:41.303422 (+ 66us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:41.303511 (+ 89us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:41.303599 (+ 88us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:41.303786 (+ 187us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:41.303906 (+ 120us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:41.303946 (+ 40us) write_op.cc:312] APPLY: finished
0925 09:57:41.304106 (+ 160us) log.cc:842] Serialized 23 byte log entry
0925 09:57:41.304650 (+ 544us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:41.304715 (+ 65us) write_op.cc:454] Released schema lock
0925 09:57:41.304755 (+ 40us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:41.304907 (+ 152us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:41.304929 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":41125,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":265,"apply.run_cpu_time_us":1331,"apply.run_wall_time_us":1733,"num_ops":1,"prepare.queue_time_us":147379,"prepare.run_cpu_time_us":2187,"prepare.run_wall_time_us":3231,"raft.queue_time_us":327,"raft.run_cpu_time_us":633,"raft.run_wall_time_us":633,"replication_time_us":8216}]]}
I20260925 09:57:42.295032 11085 consensus_queue.cc:260] T 9a73a1f242e7430cbf52229c07b4dc82 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.295485 12027 rpcz_store.cc:276] 0925 09:57:39.207919 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:39.208026 (+ 107us) service_pool.cc:227] Handling call
0925 09:57:41.979735 (+2771709us) ts_tablet_manager.cc:1145] Deleting tablet f62c62715c9b42de826c1a66fd431eab
0925 09:57:41.979774 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:41.998830 (+ 19056us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:42.279833 (+281003us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:42.290301 (+ 10468us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2771199}
I20260925 09:57:42.297216 11085 ts_tablet_manager.cc:1428] T 9a73a1f242e7430cbf52229c07b4dc82 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.092s user 0.004s sys 0.002s
I20260925 09:57:42.290970 11084 consensus_queue.cc:260] T ac8230d622214346bc7412bfc6cdb203 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.301054 11084 ts_tablet_manager.cc:1428] T ac8230d622214346bc7412bfc6cdb203 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.025s user 0.004s sys 0.000s
I20260925 09:57:42.308072 11084 tablet_bootstrap.cc:492] T 476567ad4efa4043b6536f3f28d2ee28 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:42.314280 11727 raft_consensus.cc:2229] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:42.314798 11727 raft_consensus.cc:2258] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:42.315322 11084 tablet_bootstrap.cc:654] T 476567ad4efa4043b6536f3f28d2ee28 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:42.316694 11727 ts_tablet_manager.cc:1905] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:42.330240 12027 tablet_replica.cc:330] T 37bab60afe7a4c758a08ef7c9c24aba0 P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:42.330669 11085 tablet_bootstrap.cc:492] T d54ab15dd054489ab50b7f0b26b79a50 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:42.331240 12671 raft_consensus.cc:482] T 05e2c7f9820f41679c4d58343ff3f829 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:42.331691 12671 raft_consensus.cc:504] T 05e2c7f9820f41679c4d58343ff3f829 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.333526 12671 leader_election.cc:288] T 05e2c7f9820f41679c4d58343ff3f829 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:42.335031 12027 raft_consensus.cc:2229] T 37bab60afe7a4c758a08ef7c9c24aba0 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:42.335584 12027 raft_consensus.cc:2258] T 37bab60afe7a4c758a08ef7c9c24aba0 P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 09:57:42.340485 10772 leader_election.cc:341] T 05e2c7f9820f41679c4d58343ff3f829 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 05e2c7f9820f41679c4d58343ff3f829
I20260925 09:57:42.254479 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet a8be01fe42894b96bbdd5ce2403ff1f4 (table default.Table_b13e058d7a264358a7b057cb23f04d18 [id=5e17848977874de3af045b3202674dbb]) successfully deleted
W20260925 09:57:42.343642 10377 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.10.9.3:44553, user_credentials={real_user=slave}} blocked reactor thread for 89200.1us
I20260925 09:57:42.297173 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet f62c62715c9b42de826c1a66fd431eab (table default.Table_bcbc82519cf043738f61029181f29418 [id=b3f7f21ab2214e4cb637bea27ce62abe]) successfully deleted
I20260925 09:57:42.345767 10945 tablet_service.cc:1514] Processing DeleteTablet for tablet a58cf1dc81454710a9ad1a8baf155c9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:30 UTC) from {username='slave'} at 127.0.0.1:53560
W20260925 09:57:42.345844 10379 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.10.9.2:43257, user_credentials={real_user=slave}} blocked reactor thread for 48690.4us
I20260925 09:57:42.346293 10960 tablet_service.cc:1514] Processing DeleteTablet for tablet a21909e2e5794f57971f4e42795235c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:32 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:42.346436 10948 tablet_service.cc:1514] Processing DeleteTablet for tablet f62c62715c9b42de826c1a66fd431eab with delete_type TABLET_DATA_DELETED (replaced by tablet 4e0df1d8cf81469692b946b3c2a50caf) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:42.346819 10946 tablet_service.cc:1514] Processing DeleteTablet for tablet cfa619b699004882a82976f0ef8204ad with delete_type TABLET_DATA_DELETED (replaced by tablet e83d6dc8657d4a1f89ab0893f435abf6) from {username='slave'} at 127.0.0.1:53560
W20260925 09:57:42.246400 10393 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.2:40761 (request call id 47) took 1016 ms (client timeout 1000 ms). Trace:
I20260925 09:57:42.346925 10947 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b1b2d66397b44f38056077d79941ac4 with delete_type TABLET_DATA_DELETED (replaced by tablet f535784bf0a74769a6d97115fb3b0c8a) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:42.347070 10945 tablet_service.cc:1514] Processing DeleteTablet for tablet 5608c5f9b65b4da7b2fbb56417c92d88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:36 UTC) from {username='slave'} at 127.0.0.1:53560
W20260925 09:57:42.316308 10389 catalog_manager.cc:5687] Failed to send AlterTable RPC for tablet c52745a8cdee461aba7485b0531245d6 (table default.Table_8e186a99ccba425b9b588041d2364a40 [id=2d6bc0e2aaee4b1483b01fc678505bb0], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet c52745a8cdee461aba7485b0531245d6 consensus state has no leader
I20260925 09:57:42.347361 10952 tablet_service.cc:1514] Processing DeleteTablet for tablet 93e1b5837aff48d7907890c40dfcd0d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:32 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:42.347914 10956 tablet_service.cc:1514] Processing DeleteTablet for tablet abfc8f390d434c47a8c4990f8406c270 with delete_type TABLET_DATA_DELETED (replaced by tablet b025f8ba34694ba09cb01ca959d3629c) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:42.349494 12671 raft_consensus.cc:482] T 9a73a1f242e7430cbf52229c07b4dc82 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:42.349882 12671 raft_consensus.cc:504] T 9a73a1f242e7430cbf52229c07b4dc82 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.350437 11085 tablet_bootstrap.cc:654] T d54ab15dd054489ab50b7f0b26b79a50 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:42.351089 10389 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.3:41469 (request call id 34) took 3899 ms (client timeout 1000 ms). Trace:
W20260925 09:57:42.351359 10389 rpcz_store.cc:269] 0925 09:57:38.451602 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.634954 (+183352us) service_pool.cc:227] Handling call
0925 09:57:42.351024 (+3716070us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":186755,"reported_tablets":72}
I20260925 09:57:42.352521 12671 leader_election.cc:288] T 9a73a1f242e7430cbf52229c07b4dc82 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
W20260925 09:57:42.347208 10393 rpcz_store.cc:269] 0925 09:57:41.230375 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:41.230539 (+ 164us) service_pool.cc:227] Handling call
0925 09:57:42.246329 (+1015790us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:42.048031 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:42.048393 (+ 362us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:42.048416 (+ 23us) write_op.cc:435] Acquired schema lock
0925 09:57:42.048431 (+ 15us) tablet.cc:660] Decoding operations
0925 09:57:42.048671 (+ 240us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:42.048704 (+ 33us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:42.048729 (+ 25us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:42.048870 (+ 141us) tablet.cc:699] Row locks acquired
0925 09:57:42.048885 (+ 15us) write_op.cc:260] PREPARE: finished
0925 09:57:42.048981 (+ 96us) write_op.cc:270] Start()
0925 09:57:42.049055 (+ 74us) write_op.cc:276] Timestamp: P: 1790330262048931 usec, L: 0
0925 09:57:42.049072 (+ 17us) op_driver.cc:331] REPLICATION: starting
0925 09:57:42.049503 (+ 431us) log.cc:842] Serialized 356 byte log entry
0925 09:57:42.082035 (+ 32532us) op_driver.cc:444] REPLICATION: finished
0925 09:57:42.084296 (+ 2261us) write_op.cc:301] APPLY: starting
0925 09:57:42.084345 (+ 49us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:42.084422 (+ 77us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:42.084517 (+ 95us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:42.084654 (+ 137us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:42.084766 (+ 112us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:42.084804 (+ 38us) write_op.cc:312] APPLY: finished
0925 09:57:42.084940 (+ 136us) log.cc:842] Serialized 23 byte log entry
0925 09:57:42.085044 (+ 104us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:42.085093 (+ 49us) write_op.cc:454] Released schema lock
0925 09:57:42.085120 (+ 27us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:42.085259 (+ 139us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:42.085290 (+ 31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":63778,"reported_tablets":33,"child_traces":[["op",{"apply.queue_time_us":2128,"apply.run_cpu_time_us":1135,"apply.run_wall_time_us":1134,"num_ops":1,"prepare.queue_time_us":1321,"prepare.run_cpu_time_us":1736,"prepare.run_wall_time_us":2884,"raft.queue_time_us":1063,"raft.run_cpu_time_us":9,"raft.run_wall_time_us":10,"replication_time_us":32899,"thread_start_us":129903,"threads_started":1}]]}
I20260925 09:57:42.353820 10956 tablet_service.cc:1514] Processing DeleteTablet for tablet b3d9a3cf5bda4cf6bc1fa3fae1cb9288 with delete_type TABLET_DATA_DELETED (replaced by tablet f46095351ea64812a7fdcfc79cd92c1f) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:42.354938 10956 tablet_service.cc:1514] Processing DeleteTablet for tablet 350904aa5957496d90a2aa2e637b591c with delete_type TABLET_DATA_DELETED (replaced by tablet 154083d32e0f474f9e3a02e6cca940ef) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:42.355558 10952 tablet_service.cc:1514] Processing DeleteTablet for tablet a8be01fe42894b96bbdd5ce2403ff1f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:39 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:42.356467 10952 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e76f9e0ba554e1cad0cf18863393608 with delete_type TABLET_DATA_DELETED (replaced by tablet 963e5bf2ac2848b4967074d28acdae52) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:42.357280 10952 tablet_service.cc:1514] Processing DeleteTablet for tablet 05e2c7f9820f41679c4d58343ff3f829 with delete_type TABLET_DATA_DELETED (replaced by tablet 4637237acb184a6aaae94a04218f896b) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:42.358105 10952 tablet_service.cc:1514] Processing DeleteTablet for tablet 17bd30416fe14c0e9ab2e0c0bf5f2fb5 with delete_type TABLET_DATA_DELETED (replaced by tablet 4a6b1284cddc4b2581931d8895212ca6) from {username='slave'} at 127.0.0.1:53560
W20260925 09:57:42.358875 10772 leader_election.cc:341] T 9a73a1f242e7430cbf52229c07b4dc82 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 9a73a1f242e7430cbf52229c07b4dc82
W20260925 09:57:42.320756 10386 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.1:56563 (request call id 45) took 3467 ms (client timeout 1000 ms). Trace:
W20260925 09:57:42.327059 10380 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 963e5bf2ac2848b4967074d28acdae52 (table default.Table_257e35b439d7481fbcfcd5d50b6c6c08 [id=5dc3fd1666fb4431b1c3bbf27b02ba54], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 963e5bf2ac2848b4967074d28acdae52 consensus state has no leader
I20260925 09:57:42.359786 12688 raft_consensus.cc:482] T 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:42.360215 12688 raft_consensus.cc:504] T 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.360737 12027 ts_tablet_manager.cc:1905] T 37bab60afe7a4c758a08ef7c9c24aba0 P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:42.361647 11084 tablet_bootstrap.cc:492] T 476567ad4efa4043b6536f3f28d2ee28 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:42.362059 11084 ts_tablet_manager.cc:1397] T 476567ad4efa4043b6536f3f28d2ee28 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.054s user 0.008s sys 0.000s
I20260925 09:57:42.362260 12688 leader_election.cc:288] T 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:42.364197 10824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05a6ea9a6a7449fe8a1a2f7d9ff0c5ef" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:42.365597 10918 leader_election.cc:341] T 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:42.369537 11084 raft_consensus.cc:348] T 476567ad4efa4043b6536f3f28d2ee28 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.370182 11084 raft_consensus.cc:374] T 476567ad4efa4043b6536f3f28d2ee28 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:42.370419 11084 raft_consensus.cc:729] T 476567ad4efa4043b6536f3f28d2ee28 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
W20260925 09:57:42.371464 10386 rpcz_store.cc:269] 0925 09:57:38.853054 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:38.853706 (+ 652us) service_pool.cc:227] Handling call
0925 09:57:42.320686 (+3466980us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:39.726538 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:39.727093 (+ 555us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:39.727134 (+ 41us) write_op.cc:435] Acquired schema lock
0925 09:57:39.727157 (+ 23us) tablet.cc:660] Decoding operations
0925 09:57:39.727828 (+ 671us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:39.727892 (+ 64us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:39.727929 (+ 37us) tablet.cc:683] Acquiring locks for 3 operations
0925 09:57:39.728302 (+ 373us) tablet.cc:699] Row locks acquired
0925 09:57:39.728331 (+ 29us) write_op.cc:260] PREPARE: finished
0925 09:57:39.728484 (+ 153us) write_op.cc:270] Start()
0925 09:57:39.728606 (+ 122us) write_op.cc:276] Timestamp: P: 1790330259728435 usec, L: 0
0925 09:57:39.728627 (+ 21us) op_driver.cc:331] REPLICATION: starting
0925 09:57:39.729345 (+ 718us) log.cc:842] Serialized 1266 byte log entry
0925 09:57:39.737887 (+ 8542us) op_driver.cc:444] REPLICATION: finished
0925 09:57:39.738219 (+ 332us) write_op.cc:301] APPLY: starting
0925 09:57:39.738276 (+ 57us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:39.738458 (+ 182us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:39.738572 (+ 114us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:39.738930 (+ 358us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:39.739072 (+ 142us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0925 09:57:39.739119 (+ 47us) write_op.cc:312] APPLY: finished
0925 09:57:39.739313 (+ 194us) log.cc:842] Serialized 35 byte log entry
0925 09:57:39.744646 (+ 5333us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:39.744719 (+ 73us) write_op.cc:454] Released schema lock
0925 09:57:39.744750 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:39.744949 (+ 199us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:39.744990 (+ 41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":61990,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":184,"apply.run_cpu_time_us":1856,"apply.run_wall_time_us":6979,"num_ops":3,"prepare.queue_time_us":17679,"prepare.run_cpu_time_us":3136,"prepare.run_wall_time_us":5046,"raft.queue_time_us":82,"raft.run_cpu_time_us":599,"raft.run_wall_time_us":600,"replication_time_us":9169}]]}
I20260925 09:57:42.388654 11084 consensus_queue.cc:260] T 476567ad4efa4043b6536f3f28d2ee28 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.390677 11084 ts_tablet_manager.cc:1428] T 476567ad4efa4043b6536f3f28d2ee28 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.028s user 0.005s sys 0.000s
W20260925 09:57:42.415180 10390 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.1:56563 (request call id 47) took 1223 ms (client timeout 1000 ms). Trace:
I20260925 09:57:42.417021 11085 tablet_bootstrap.cc:492] T d54ab15dd054489ab50b7f0b26b79a50 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:42.417444 11085 ts_tablet_manager.cc:1397] T d54ab15dd054489ab50b7f0b26b79a50 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.087s user 0.008s sys 0.000s
W20260925 09:57:42.415562 10390 rpcz_store.cc:269] 0925 09:57:41.191756 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:41.191904 (+ 148us) service_pool.cc:227] Handling call
0925 09:57:42.412826 (+1220922us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:42.045874 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:42.046418 (+ 544us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:42.046444 (+ 26us) write_op.cc:435] Acquired schema lock
0925 09:57:42.046459 (+ 15us) tablet.cc:660] Decoding operations
0925 09:57:42.046753 (+ 294us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:42.046798 (+ 45us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:42.046822 (+ 24us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:42.046964 (+ 142us) tablet.cc:699] Row locks acquired
0925 09:57:42.046979 (+ 15us) write_op.cc:260] PREPARE: finished
0925 09:57:42.047084 (+ 105us) write_op.cc:270] Start()
0925 09:57:42.047169 (+ 85us) write_op.cc:276] Timestamp: P: 1790330262047029 usec, L: 0
0925 09:57:42.047188 (+ 19us) op_driver.cc:331] REPLICATION: starting
0925 09:57:42.047665 (+ 477us) log.cc:842] Serialized 524 byte log entry
0925 09:57:42.081795 (+ 34130us) op_driver.cc:444] REPLICATION: finished
0925 09:57:42.082473 (+ 678us) write_op.cc:301] APPLY: starting
0925 09:57:42.082534 (+ 61us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:42.082630 (+ 96us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:42.082722 (+ 92us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:42.082890 (+ 168us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:42.083012 (+ 122us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:42.083052 (+ 40us) write_op.cc:312] APPLY: finished
0925 09:57:42.083220 (+ 168us) log.cc:842] Serialized 23 byte log entry
0925 09:57:42.083367 (+ 147us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:42.083445 (+ 78us) write_op.cc:454] Released schema lock
0925 09:57:42.083476 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:42.083645 (+ 169us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:42.083671 (+ 26us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":527,"apply.run_cpu_time_us":1373,"apply.run_wall_time_us":1699,"num_ops":1,"prepare.queue_time_us":103,"prepare.run_cpu_time_us":2061,"prepare.run_wall_time_us":2061,"raft.queue_time_us":2118,"raft.run_cpu_time_us":646,"raft.run_wall_time_us":645,"replication_time_us":34530}]]}
I20260925 09:57:42.419525 11085 raft_consensus.cc:348] T d54ab15dd054489ab50b7f0b26b79a50 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.420104 11085 raft_consensus.cc:374] T d54ab15dd054489ab50b7f0b26b79a50 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:42.420325 11085 raft_consensus.cc:729] T d54ab15dd054489ab50b7f0b26b79a50 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:42.420864 11084 tablet_bootstrap.cc:492] T e329410c044c40a4933550730f46001e P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:42.421646 11727 ts_tablet_manager.cc:1918] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260925 09:57:42.422015 11727 log.cc:1196] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/26fae3fbfe374f348a1729c21ec751b9
I20260925 09:57:42.422688 11727 ts_tablet_manager.cc:1939] T 26fae3fbfe374f348a1729c21ec751b9 P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:42.424337 11085 consensus_queue.cc:260] T d54ab15dd054489ab50b7f0b26b79a50 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.426589 11085 ts_tablet_manager.cc:1428] T d54ab15dd054489ab50b7f0b26b79a50 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.009s user 0.005s sys 0.001s
I20260925 09:57:42.428864 11084 tablet_bootstrap.cc:654] T e329410c044c40a4933550730f46001e P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:42.431823 11085 tablet_bootstrap.cc:492] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:42.454769 11085 tablet_bootstrap.cc:654] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:42.460100 12697 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8af3f111e8da4d2dad4e4a98ef0fd5d3)
I20260925 09:57:42.462769 12697 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } }
I20260925 09:57:42.466079 10326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" candidate_term: 5 candidate_status { last_received { term: 4 index: 456 } } ignore_live_leader: false dest_uuid: "54c8485a7dcc47a1a75d08ef57647ece" is_pre_election: true
I20260925 09:57:42.466799 11084 tablet_bootstrap.cc:492] T e329410c044c40a4933550730f46001e P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:42.467198 11084 ts_tablet_manager.cc:1397] T e329410c044c40a4933550730f46001e P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.076s user 0.007s sys 0.003s
I20260925 09:57:42.467481 12697 leader_election.cc:288] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 54c8485a7dcc47a1a75d08ef57647ece (127.10.9.62:34455), 8af3f111e8da4d2dad4e4a98ef0fd5d3 (127.10.9.61:33047)
I20260925 09:57:42.468801 10952 tablet_service.cc:1514] Processing DeleteTablet for tablet e9faf70be8ac43c79305980d685a28eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:28 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:42.477582 11727 tablet_replica.cc:330] T a58cf1dc81454710a9ad1a8baf155c9f P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:42.477473 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 26fae3fbfe374f348a1729c21ec751b9 (table default.Table_0011387b45014de4b24b466f4cb907f7 [id=e3f7aa5e988c4b6c98be5cd3d465245d]) successfully deleted
I20260925 09:57:42.478003 11084 raft_consensus.cc:348] T e329410c044c40a4933550730f46001e P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.478636 11084 raft_consensus.cc:374] T e329410c044c40a4933550730f46001e P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:42.478888 11084 raft_consensus.cc:729] T e329410c044c40a4933550730f46001e P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:42.479101 11727 raft_consensus.cc:2229] T a58cf1dc81454710a9ad1a8baf155c9f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:42.479449 11084 consensus_queue.cc:260] T e329410c044c40a4933550730f46001e P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.479991 11727 raft_consensus.cc:2258] T a58cf1dc81454710a9ad1a8baf155c9f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260925 09:57:42.426339 10385 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.3:41469 (request call id 35) took 2641 ms (client timeout 1000 ms). Trace:
I20260925 09:57:42.481421 11084 ts_tablet_manager.cc:1428] T e329410c044c40a4933550730f46001e P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.014s user 0.004s sys 0.000s
I20260925 09:57:42.465797 10403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" candidate_term: 5 candidate_status { last_received { term: 4 index: 456 } } ignore_live_leader: false dest_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" is_pre_election: true
I20260925 09:57:42.482436 11727 ts_tablet_manager.cc:1905] T a58cf1dc81454710a9ad1a8baf155c9f P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:42.481451 12027 ts_tablet_manager.cc:1918] T 37bab60afe7a4c758a08ef7c9c24aba0 P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:42.482884 12198 leader_election.cc:302] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8be057e0b6d40a3a4b90f2b1f4a0fa2; no voters: 54c8485a7dcc47a1a75d08ef57647ece, 8af3f111e8da4d2dad4e4a98ef0fd5d3
I20260925 09:57:42.483880 12697 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260925 09:57:42.481038 10385 rpcz_store.cc:269] 0925 09:57:39.784935 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:39.785065 (+ 130us) service_pool.cc:227] Handling call
0925 09:57:42.426270 (+2641205us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:57:41.075340 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:57:41.075908 (+ 568us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:57:41.075946 (+ 38us) write_op.cc:435] Acquired schema lock
0925 09:57:41.075969 (+ 23us) tablet.cc:660] Decoding operations
0925 09:57:41.076325 (+ 356us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:57:41.076400 (+ 75us) write_op.cc:641] Partition lock acquired for write op
0925 09:57:41.076434 (+ 34us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:57:41.076625 (+ 191us) tablet.cc:699] Row locks acquired
0925 09:57:41.076647 (+ 22us) write_op.cc:260] PREPARE: finished
0925 09:57:41.076805 (+ 158us) write_op.cc:270] Start()
0925 09:57:41.076913 (+ 108us) write_op.cc:276] Timestamp: P: 1790330261076737 usec, L: 0
0925 09:57:41.076934 (+ 21us) op_driver.cc:331] REPLICATION: starting
0925 09:57:41.077547 (+ 613us) log.cc:842] Serialized 356 byte log entry
0925 09:57:41.135951 (+ 58404us) op_driver.cc:444] REPLICATION: finished
0925 09:57:41.147274 (+ 11323us) write_op.cc:301] APPLY: starting
0925 09:57:41.147330 (+ 56us) tablet.cc:1365] starting BulkCheckPresence
0925 09:57:41.147437 (+ 107us) tablet.cc:1368] finished BulkCheckPresence
0925 09:57:41.147546 (+ 109us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:57:41.147752 (+ 206us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:57:41.147938 (+ 186us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:57:41.148001 (+ 63us) write_op.cc:312] APPLY: finished
0925 09:57:41.148213 (+ 212us) log.cc:842] Serialized 23 byte log entry
0925 09:57:41.252065 (+103852us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:41.252157 (+ 92us) write_op.cc:454] Released schema lock
0925 09:57:41.252201 (+ 44us) write_op.cc:341] FINISH: Updating metrics
0925 09:57:41.263842 (+ 11641us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:57:41.263882 (+ 40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":37743,"reported_tablets":70,"child_traces":[["op",{"apply.queue_time_us":11135,"apply.run_cpu_time_us":1938,"apply.run_wall_time_us":116822,"num_ops":1,"prepare.queue_time_us":66779,"prepare.run_cpu_time_us":2581,"prepare.run_wall_time_us":2773,"raft.queue_time_us":2488,"raft.run_cpu_time_us":622,"raft.run_wall_time_us":622,"replication_time_us":58938,"thread_start_us":81839,"threads_started":2}]]}
I20260925 09:57:42.497314 12027 log.cc:1196] T 37bab60afe7a4c758a08ef7c9c24aba0 P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/37bab60afe7a4c758a08ef7c9c24aba0
I20260925 09:57:42.496942 11085 tablet_bootstrap.cc:492] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:42.497925 11085 ts_tablet_manager.cc:1397] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.066s user 0.007s sys 0.002s
W20260925 09:57:42.470860 10388 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.3:41469 (request call id 36) took 1632 ms (client timeout 1000 ms). Trace:
W20260925 09:57:42.498353 10388 rpcz_store.cc:269] 0925 09:57:40.837969 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:40.838099 (+ 130us) service_pool.cc:227] Handling call
0925 09:57:42.470788 (+1632689us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":182768,"reported_tablets":72}
I20260925 09:57:42.500349 11085 raft_consensus.cc:348] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.501035 11085 raft_consensus.cc:374] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:42.501309 11085 raft_consensus.cc:729] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:42.501991 11085 consensus_queue.cc:260] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.504493 11085 ts_tablet_manager.cc:1428] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260925 09:57:42.505321 11085 tablet_bootstrap.cc:492] T abfc8f390d434c47a8c4990f8406c270 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:42.513423 11084 tablet_bootstrap.cc:492] T 451d1984f9b144b68f0681837f8eacf1 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:42.506570 12027 ts_tablet_manager.cc:1939] T 37bab60afe7a4c758a08ef7c9c24aba0 P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:42.523864 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 400) took 3081 ms. Trace:
I20260925 09:57:42.525533 12027 rpcz_store.cc:276] 0925 09:57:39.442314 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:39.442522 (+ 208us) service_pool.cc:227] Handling call
0925 09:57:42.297740 (+2855218us) ts_tablet_manager.cc:1145] Deleting tablet 37bab60afe7a4c758a08ef7c9c24aba0
0925 09:57:42.330113 (+ 32373us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:42.416628 (+ 86515us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:42.481216 (+ 64588us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:42.523804 (+ 42588us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2854634}
I20260925 09:57:42.527742 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 37bab60afe7a4c758a08ef7c9c24aba0 (table default.Table_8b45818055ec40e386d43e2641ec6ed0 [id=38b800f8e1e948909d40cc976272fdd6]) successfully deleted
I20260925 09:57:42.528860 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 401) took 3074 ms. Trace:
I20260925 09:57:42.519197 12688 raft_consensus.cc:482] T 2d79538d6474433fbda26c91645a0619 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:42.531936 12688 raft_consensus.cc:504] T 2d79538d6474433fbda26c91645a0619 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.533845 12688 leader_election.cc:288] T 2d79538d6474433fbda26c91645a0619 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:42.535981 11084 tablet_bootstrap.cc:654] T 451d1984f9b144b68f0681837f8eacf1 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:42.534488 10824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d79538d6474433fbda26c91645a0619" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:42.536286 12027 rpcz_store.cc:276] 0925 09:57:39.454179 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:39.454290 (+ 111us) service_pool.cc:227] Handling call
0925 09:57:42.528619 (+3074329us) ts_tablet_manager.cc:1145] Deleting tablet dba14a44a9de492886d629441606767e
0925 09:57:42.528661 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:42.528811 (+ 150us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3073788}
W20260925 09:57:42.537297 10918 leader_election.cc:341] T 2d79538d6474433fbda26c91645a0619 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:42.537462 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet dba14a44a9de492886d629441606767e because the tablet was not found. No further retry: Not found: Tablet not found: dba14a44a9de492886d629441606767e
I20260925 09:57:42.538484 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 402) took 3028 ms. Trace:
I20260925 09:57:42.543505 12027 rpcz_store.cc:276] 0925 09:57:39.509931 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:39.510052 (+ 121us) service_pool.cc:227] Handling call
0925 09:57:42.538216 (+3028164us) ts_tablet_manager.cc:1145] Deleting tablet cfa619b699004882a82976f0ef8204ad
0925 09:57:42.538257 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:42.538435 (+ 178us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3011655}
W20260925 09:57:42.552205 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet cfa619b699004882a82976f0ef8204ad because the tablet was not found. No further retry: Not found: Tablet not found: cfa619b699004882a82976f0ef8204ad
I20260925 09:57:42.553766 12027 tablet_replica.cc:330] T 05e2c7f9820f41679c4d58343ff3f829 P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:42.554586 12027 raft_consensus.cc:2229] T 05e2c7f9820f41679c4d58343ff3f829 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:42.558314 11085 tablet_bootstrap.cc:654] T abfc8f390d434c47a8c4990f8406c270 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:42.557767 12027 raft_consensus.cc:2258] T 05e2c7f9820f41679c4d58343ff3f829 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:42.560607 12027 ts_tablet_manager.cc:1905] T 05e2c7f9820f41679c4d58343ff3f829 P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:42.570992 12688 raft_consensus.cc:482] T ac8230d622214346bc7412bfc6cdb203 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:42.571575 12688 raft_consensus.cc:504] T ac8230d622214346bc7412bfc6cdb203 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.573848 12688 leader_election.cc:288] T ac8230d622214346bc7412bfc6cdb203 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:42.575484 10824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac8230d622214346bc7412bfc6cdb203" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:42.576833 10918 leader_election.cc:341] T ac8230d622214346bc7412bfc6cdb203 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:42.581480 11084 tablet_bootstrap.cc:492] T 451d1984f9b144b68f0681837f8eacf1 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:42.582731 11084 ts_tablet_manager.cc:1397] T 451d1984f9b144b68f0681837f8eacf1 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.069s user 0.009s sys 0.002s
I20260925 09:57:42.584821 11084 raft_consensus.cc:348] T 451d1984f9b144b68f0681837f8eacf1 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.585438 11084 raft_consensus.cc:374] T 451d1984f9b144b68f0681837f8eacf1 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:42.585664 11084 raft_consensus.cc:729] T 451d1984f9b144b68f0681837f8eacf1 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:42.586215 11084 consensus_queue.cc:260] T 451d1984f9b144b68f0681837f8eacf1 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.597067 11085 tablet_bootstrap.cc:492] T abfc8f390d434c47a8c4990f8406c270 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:42.597543 11085 ts_tablet_manager.cc:1397] T abfc8f390d434c47a8c4990f8406c270 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.092s user 0.005s sys 0.004s
I20260925 09:57:42.599709 11085 raft_consensus.cc:348] T abfc8f390d434c47a8c4990f8406c270 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.600294 11085 raft_consensus.cc:374] T abfc8f390d434c47a8c4990f8406c270 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:42.600525 11085 raft_consensus.cc:729] T abfc8f390d434c47a8c4990f8406c270 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:42.601063 11085 consensus_queue.cc:260] T abfc8f390d434c47a8c4990f8406c270 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.602924 11085 ts_tablet_manager.cc:1428] T abfc8f390d434c47a8c4990f8406c270 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260925 09:57:42.603648 11084 ts_tablet_manager.cc:1428] T 451d1984f9b144b68f0681837f8eacf1 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.021s user 0.005s sys 0.000s
I20260925 09:57:42.604491 11084 tablet_bootstrap.cc:492] T a3edf29118e048c5a59b4ff2643ed206 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:42.614836 12688 raft_consensus.cc:482] T d5f28d1c19eb4e849dfa6b0bf532b67e P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:42.615329 12688 raft_consensus.cc:504] T d5f28d1c19eb4e849dfa6b0bf532b67e P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.616554 11727 ts_tablet_manager.cc:1918] T a58cf1dc81454710a9ad1a8baf155c9f P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:42.616863 11727 log.cc:1196] T a58cf1dc81454710a9ad1a8baf155c9f P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/a58cf1dc81454710a9ad1a8baf155c9f
I20260925 09:57:42.617416 11727 ts_tablet_manager.cc:1939] T a58cf1dc81454710a9ad1a8baf155c9f P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:42.618571 10824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5f28d1c19eb4e849dfa6b0bf532b67e" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:42.619493 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet a58cf1dc81454710a9ad1a8baf155c9f (table default.Table_0478cdcfe9d64b66ba0e3fbc638f9a3b [id=58904eafabaa40eeb24a79dba89ae346]) successfully deleted
W20260925 09:57:42.619817 10918 leader_election.cc:341] T d5f28d1c19eb4e849dfa6b0bf532b67e P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:42.620296 12688 leader_election.cc:288] T d5f28d1c19eb4e849dfa6b0bf532b67e P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:42.632422 11084 tablet_bootstrap.cc:654] T a3edf29118e048c5a59b4ff2643ed206 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:42.633149 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet a21909e2e5794f57971f4e42795235c7 because the tablet was not found. No further retry: Not found: Tablet not found: a21909e2e5794f57971f4e42795235c7
W20260925 09:57:42.633749 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet f62c62715c9b42de826c1a66fd431eab because the tablet was not found. No further retry: Not found: Tablet not found: f62c62715c9b42de826c1a66fd431eab
W20260925 09:57:42.634254 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet cfa619b699004882a82976f0ef8204ad because the tablet was not found. No further retry: Not found: Tablet not found: cfa619b699004882a82976f0ef8204ad
W20260925 09:57:42.634760 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 2b1b2d66397b44f38056077d79941ac4 because the tablet was not found. No further retry: Not found: Tablet not found: 2b1b2d66397b44f38056077d79941ac4
W20260925 09:57:42.635262 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 5608c5f9b65b4da7b2fbb56417c92d88 because the tablet was not found. No further retry: Not found: Tablet not found: 5608c5f9b65b4da7b2fbb56417c92d88
W20260925 09:57:42.647758 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 93e1b5837aff48d7907890c40dfcd0d0 because the tablet was not found. No further retry: Not found: Tablet not found: 93e1b5837aff48d7907890c40dfcd0d0
I20260925 09:57:42.654228 11084 tablet_bootstrap.cc:492] T a3edf29118e048c5a59b4ff2643ed206 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:42.654613 11084 ts_tablet_manager.cc:1397] T a3edf29118e048c5a59b4ff2643ed206 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.050s user 0.004s sys 0.005s
I20260925 09:57:42.657464 11084 raft_consensus.cc:348] T a3edf29118e048c5a59b4ff2643ed206 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.658306 11084 raft_consensus.cc:374] T a3edf29118e048c5a59b4ff2643ed206 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:42.658543 11084 raft_consensus.cc:729] T a3edf29118e048c5a59b4ff2643ed206 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:42.665675 11084 consensus_queue.cc:260] T a3edf29118e048c5a59b4ff2643ed206 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.678293 11085 tablet_bootstrap.cc:492] T 54157e776efa4c6c8e2bd4c58321fcda P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:42.682582 11084 ts_tablet_manager.cc:1428] T a3edf29118e048c5a59b4ff2643ed206 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.028s user 0.006s sys 0.001s
I20260925 09:57:42.683751 11085 tablet_bootstrap.cc:654] T 54157e776efa4c6c8e2bd4c58321fcda P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:42.686823 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet abfc8f390d434c47a8c4990f8406c270 because the tablet was not found. No further retry: Not found: Tablet not found: abfc8f390d434c47a8c4990f8406c270
W20260925 09:57:42.687471 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet b3d9a3cf5bda4cf6bc1fa3fae1cb9288 because the tablet was not found. No further retry: Not found: Tablet not found: b3d9a3cf5bda4cf6bc1fa3fae1cb9288
W20260925 09:57:42.687956 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 350904aa5957496d90a2aa2e637b591c because the tablet was not found. No further retry: Not found: Tablet not found: 350904aa5957496d90a2aa2e637b591c
W20260925 09:57:42.688416 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet a8be01fe42894b96bbdd5ce2403ff1f4 because the tablet was not found. No further retry: Not found: Tablet not found: a8be01fe42894b96bbdd5ce2403ff1f4
W20260925 09:57:42.688859 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 1e76f9e0ba554e1cad0cf18863393608 because the tablet was not found. No further retry: Not found: Tablet not found: 1e76f9e0ba554e1cad0cf18863393608
I20260925 09:57:42.688918 11727 tablet_replica.cc:330] T e9faf70be8ac43c79305980d685a28eb P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
W20260925 09:57:42.689342 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 05e2c7f9820f41679c4d58343ff3f829 because the tablet was not found. No further retry: Not found: Tablet not found: 05e2c7f9820f41679c4d58343ff3f829
I20260925 09:57:42.689779 11727 raft_consensus.cc:2229] T e9faf70be8ac43c79305980d685a28eb P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260925 09:57:42.689817 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 17bd30416fe14c0e9ab2e0c0bf5f2fb5 because the tablet was not found. No further retry: Not found: Tablet not found: 17bd30416fe14c0e9ab2e0c0bf5f2fb5
I20260925 09:57:42.690148 11084 tablet_bootstrap.cc:492] T d822e8ac13d9479aa9ce4f74ace55dd5 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:42.690187 11727 raft_consensus.cc:2258] T e9faf70be8ac43c79305980d685a28eb P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:42.693576 11727 ts_tablet_manager.cc:1905] T e9faf70be8ac43c79305980d685a28eb P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:42.696447 12027 ts_tablet_manager.cc:1918] T 05e2c7f9820f41679c4d58343ff3f829 P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:42.697696 12027 log.cc:1196] T 05e2c7f9820f41679c4d58343ff3f829 P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/05e2c7f9820f41679c4d58343ff3f829
I20260925 09:57:42.698357 12027 ts_tablet_manager.cc:1939] T 05e2c7f9820f41679c4d58343ff3f829 P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:42.705463 10387 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet f0bf545f70ae4cd6a1ad2540c7158ecc (table default.Table_25edfe533c64421da7700ac5d60265c3 [id=1469f44b239d454caa97056bd5803255]). Expected version 1 got 0
I20260925 09:57:42.719925 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 403) took 3193 ms. Trace:
I20260925 09:57:42.720359 12027 rpcz_store.cc:276] 0925 09:57:39.526259 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:39.526387 (+ 128us) service_pool.cc:227] Handling call
0925 09:57:42.553592 (+3027205us) ts_tablet_manager.cc:1145] Deleting tablet 05e2c7f9820f41679c4d58343ff3f829
0925 09:57:42.553649 (+ 57us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:42.606658 (+ 53009us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:42.696208 (+ 89550us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:42.719869 (+ 23661us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":16640,"tablet-delete.queue_time_us":3026574}
I20260925 09:57:42.721738 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 404) took 3177 ms. Trace:
I20260925 09:57:42.722097 12027 rpcz_store.cc:276] 0925 09:57:39.544594 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:39.544737 (+ 143us) service_pool.cc:227] Handling call
0925 09:57:42.721487 (+3176750us) ts_tablet_manager.cc:1145] Deleting tablet a21909e2e5794f57971f4e42795235c7
0925 09:57:42.721531 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:42.721691 (+ 160us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3176233}
I20260925 09:57:42.723862 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 05e2c7f9820f41679c4d58343ff3f829 (table default.Table_4066ae2ab4534ff58ac51f74bef05db9 [id=653cef93edf64ea5aad4122799ed9082]) successfully deleted
I20260925 09:57:42.724884 12027 tablet_replica.cc:330] T 10657a81058241fc988d37e5ea7ff8cf P c43461c4ee854ae1ac52b873ccada318: stopping tablet replica
I20260925 09:57:42.725672 12027 raft_consensus.cc:2229] T 10657a81058241fc988d37e5ea7ff8cf P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:57:42.726357 12027 raft_consensus.cc:2258] T 10657a81058241fc988d37e5ea7ff8cf P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 09:57:42.726877 10379 catalog_manager.cc:4808] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): delete failed for tablet a21909e2e5794f57971f4e42795235c7 because the tablet was not found. No further retry: Not found: Tablet not found: a21909e2e5794f57971f4e42795235c7
I20260925 09:57:42.724372 11085 tablet_bootstrap.cc:492] T 54157e776efa4c6c8e2bd4c58321fcda P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:42.727617 11085 ts_tablet_manager.cc:1397] T 54157e776efa4c6c8e2bd4c58321fcda P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.049s user 0.008s sys 0.001s
I20260925 09:57:42.729571 11085 raft_consensus.cc:348] T 54157e776efa4c6c8e2bd4c58321fcda P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.730180 11085 raft_consensus.cc:374] T 54157e776efa4c6c8e2bd4c58321fcda P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:42.731662 11085 raft_consensus.cc:729] T 54157e776efa4c6c8e2bd4c58321fcda P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:42.732187 11085 consensus_queue.cc:260] T 54157e776efa4c6c8e2bd4c58321fcda P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.744068 11084 tablet_bootstrap.cc:654] T d822e8ac13d9479aa9ce4f74ace55dd5 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:42.744642 11085 ts_tablet_manager.cc:1428] T 54157e776efa4c6c8e2bd4c58321fcda P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.017s user 0.003s sys 0.002s
I20260925 09:57:42.745512 11085 tablet_bootstrap.cc:492] T f9c259867cf44c4c90e50864475a4970 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:42.757676 10387 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet c52745a8cdee461aba7485b0531245d6 (table default.Table_8e186a99ccba425b9b588041d2364a40 [id=2d6bc0e2aaee4b1483b01fc678505bb0]). Expected version 2 got 0
I20260925 09:57:42.761811 12027 ts_tablet_manager.cc:1905] T 10657a81058241fc988d37e5ea7ff8cf P c43461c4ee854ae1ac52b873ccada318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:42.760488 10387 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 572c83b4a7c84fdab1fdbb0fbdb0a53d (table default.Table_8398f7e9c7a64f688bd18521e99cb669 [id=b8b7198fdaa24d72ae13b7275685c555]). Expected version 1 got 0
I20260925 09:57:42.763744 10387 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet cc295f7a9169475f9a9235e657be629f (table default.Table_168840855e0746f7ab027c1900842751 [id=99ba81c4d0dd497398e048a05759419a]). Expected version 1 got 0
I20260925 09:57:42.769971 11084 tablet_bootstrap.cc:492] T d822e8ac13d9479aa9ce4f74ace55dd5 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:42.770581 11084 ts_tablet_manager.cc:1397] T d822e8ac13d9479aa9ce4f74ace55dd5 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.081s user 0.008s sys 0.002s
I20260925 09:57:42.772691 11084 raft_consensus.cc:348] T d822e8ac13d9479aa9ce4f74ace55dd5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.773301 11084 raft_consensus.cc:374] T d822e8ac13d9479aa9ce4f74ace55dd5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260925 09:57:42.773736 12411 debug-util.cc:398] Leaking SignalData structure 0x7b08000712e0 after lost signal to thread 11465
W20260925 09:57:42.774331 12411 debug-util.cc:398] Leaking SignalData structure 0x7b08000d36e0 after lost signal to thread 12424
I20260925 09:57:42.778020 10387 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 25ac35fb43d34c9b85b8e61ad6e386a6 (table default.Table_2672598ccbce42eb8f11019f47a6fdda [id=887ce3cafef64210bb69d424ed5dae8a]). Expected version 2 got 0
I20260925 09:57:42.778820 12523 raft_consensus.cc:482] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:42.779241 12523 raft_consensus.cc:504] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.779847 11727 ts_tablet_manager.cc:1918] T e9faf70be8ac43c79305980d685a28eb P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:42.779820 12674 raft_consensus.cc:482] T 54157e776efa4c6c8e2bd4c58321fcda P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:42.780203 11727 log.cc:1196] T e9faf70be8ac43c79305980d685a28eb P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/e9faf70be8ac43c79305980d685a28eb
I20260925 09:57:42.780248 12674 raft_consensus.cc:504] T 54157e776efa4c6c8e2bd4c58321fcda P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.780788 11727 ts_tablet_manager.cc:1939] T e9faf70be8ac43c79305980d685a28eb P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:42.781234 12523 leader_election.cc:288] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:42.782171 12674 leader_election.cc:288] T 54157e776efa4c6c8e2bd4c58321fcda P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:42.784555 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54157e776efa4c6c8e2bd4c58321fcda" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0" is_pre_election: true
I20260925 09:57:42.785118 10976 raft_consensus.cc:2454] T 54157e776efa4c6c8e2bd4c58321fcda P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 0.
W20260925 09:57:42.786414 10772 leader_election.cc:341] T 17bd30416fe14c0e9ab2e0c0bf5f2fb5 P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): Not found: Tablet not found: 17bd30416fe14c0e9ab2e0c0bf5f2fb5
I20260925 09:57:42.786938 10772 leader_election.cc:302] T 54157e776efa4c6c8e2bd4c58321fcda P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:42.787787 12674 raft_consensus.cc:2781] T 54157e776efa4c6c8e2bd4c58321fcda P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:57:42.788147 12674 raft_consensus.cc:482] T 54157e776efa4c6c8e2bd4c58321fcda P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:57:42.788507 12674 raft_consensus.cc:3037] T 54157e776efa4c6c8e2bd4c58321fcda P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:42.797626 11085 tablet_bootstrap.cc:654] T f9c259867cf44c4c90e50864475a4970 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:42.798050 12674 raft_consensus.cc:504] T 54157e776efa4c6c8e2bd4c58321fcda P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.773533 11084 raft_consensus.cc:729] T d822e8ac13d9479aa9ce4f74ace55dd5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:42.804139 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet e9faf70be8ac43c79305980d685a28eb (table default.Table_4f4ab1f0170a42e2826f1e3143803a6b [id=86ea83e3afc04657ad2aff23057b36d4]) successfully deleted
I20260925 09:57:42.804755 11084 consensus_queue.cc:260] T d822e8ac13d9479aa9ce4f74ace55dd5 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.815884 10391 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3: default.Table_36cd59e3f299423fa9f1a2714df4454c [id=e176feb88d2643cf85a1f1d821133b6c] alter complete (version 1)
I20260925 09:57:42.820806 10976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54157e776efa4c6c8e2bd4c58321fcda" candidate_uuid: "c43461c4ee854ae1ac52b873ccada318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be97ab5d75934a648d4ab76341193ec0"
I20260925 09:57:42.821388 10976 raft_consensus.cc:3037] T 54157e776efa4c6c8e2bd4c58321fcda P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:57:42.828788 10976 raft_consensus.cc:2454] T 54157e776efa4c6c8e2bd4c58321fcda P be97ab5d75934a648d4ab76341193ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43461c4ee854ae1ac52b873ccada318 in term 1.
I20260925 09:57:42.831103 11085 tablet_bootstrap.cc:492] T f9c259867cf44c4c90e50864475a4970 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:42.831560 11085 ts_tablet_manager.cc:1397] T f9c259867cf44c4c90e50864475a4970 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.086s user 0.003s sys 0.007s
I20260925 09:57:42.833175 10772 leader_election.cc:302] T 54157e776efa4c6c8e2bd4c58321fcda P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0, c43461c4ee854ae1ac52b873ccada318; no voters:
I20260925 09:57:42.833895 12523 raft_consensus.cc:2781] T 54157e776efa4c6c8e2bd4c58321fcda P c43461c4ee854ae1ac52b873ccada318 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:57:42.834304 12523 raft_consensus.cc:686] T 54157e776efa4c6c8e2bd4c58321fcda P c43461c4ee854ae1ac52b873ccada318 [term 1 LEADER]: Becoming Leader. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Running, Role: LEADER
I20260925 09:57:42.835119 12523 consensus_queue.cc:237] T 54157e776efa4c6c8e2bd4c58321fcda P c43461c4ee854ae1ac52b873ccada318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.811700 11084 ts_tablet_manager.cc:1428] T d822e8ac13d9479aa9ce4f74ace55dd5 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.041s user 0.006s sys 0.000s
I20260925 09:57:42.833612 11085 raft_consensus.cc:348] T f9c259867cf44c4c90e50864475a4970 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.843197 11085 raft_consensus.cc:374] T f9c259867cf44c4c90e50864475a4970 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:42.843569 11085 raft_consensus.cc:729] T f9c259867cf44c4c90e50864475a4970 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
I20260925 09:57:42.842877 12674 leader_election.cc:288] T 54157e776efa4c6c8e2bd4c58321fcda P c43461c4ee854ae1ac52b873ccada318 [CANDIDATE]: Term 1 election: Requested vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553)
I20260925 09:57:42.844138 11085 consensus_queue.cc:260] T f9c259867cf44c4c90e50864475a4970 P c43461c4ee854ae1ac52b873ccada318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.846120 11085 ts_tablet_manager.cc:1428] T f9c259867cf44c4c90e50864475a4970 P c43461c4ee854ae1ac52b873ccada318: Time spent starting tablet: real 0.014s user 0.003s sys 0.000s
I20260925 09:57:42.843924 11084 tablet_bootstrap.cc:492] T 76b1f79efa784662a0adf032c074ce4a P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:42.807056 12027 ts_tablet_manager.cc:1918] T 10657a81058241fc988d37e5ea7ff8cf P c43461c4ee854ae1ac52b873ccada318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:57:42.860110 12027 log.cc:1196] T 10657a81058241fc988d37e5ea7ff8cf P c43461c4ee854ae1ac52b873ccada318: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/10657a81058241fc988d37e5ea7ff8cf
I20260925 09:57:42.860950 12027 ts_tablet_manager.cc:1939] T 10657a81058241fc988d37e5ea7ff8cf P c43461c4ee854ae1ac52b873ccada318: Deleting consensus metadata
I20260925 09:57:42.879473 11084 tablet_bootstrap.cc:654] T 76b1f79efa784662a0adf032c074ce4a P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:42.905067 11084 tablet_bootstrap.cc:492] T 76b1f79efa784662a0adf032c074ce4a P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:42.905449 11084 ts_tablet_manager.cc:1397] T 76b1f79efa784662a0adf032c074ce4a P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.062s user 0.008s sys 0.000s
I20260925 09:57:42.907996 11084 raft_consensus.cc:348] T 76b1f79efa784662a0adf032c074ce4a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.908633 11084 raft_consensus.cc:374] T 76b1f79efa784662a0adf032c074ce4a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:42.908890 11084 raft_consensus.cc:729] T 76b1f79efa784662a0adf032c074ce4a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:42.909596 11084 consensus_queue.cc:260] T 76b1f79efa784662a0adf032c074ce4a P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:42.943691 11084 ts_tablet_manager.cc:1428] T 76b1f79efa784662a0adf032c074ce4a P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.038s user 0.007s sys 0.000s
I20260925 09:57:42.944664 11084 tablet_bootstrap.cc:492] T 0a018bea4200474a9904f62aee37240b P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:42.946381 10806 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e76f9e0ba554e1cad0cf18863393608 with delete_type TABLET_DATA_DELETED (replaced by tablet 963e5bf2ac2848b4967074d28acdae52) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:42.946897 10810 tablet_service.cc:1514] Processing DeleteTablet for tablet 3039ec3febf848dba7c4e6fd38bb8053 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:26 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:42.947881 10812 tablet_service.cc:1514] Processing DeleteTablet for tablet a81530200cf14717bc7cbf67bb73f4e5 with delete_type TABLET_DATA_DELETED (replaced by tablet e9faf70be8ac43c79305980d685a28eb) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:42.948855 10813 tablet_service.cc:1514] Processing DeleteTablet for tablet f62c62715c9b42de826c1a66fd431eab with delete_type TABLET_DATA_DELETED (replaced by tablet 4e0df1d8cf81469692b946b3c2a50caf) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:42.949851 10815 tablet_service.cc:1514] Processing DeleteTablet for tablet 05e2c7f9820f41679c4d58343ff3f829 with delete_type TABLET_DATA_DELETED (replaced by tablet 4637237acb184a6aaae94a04218f896b) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:42.946380 10809 tablet_service.cc:1514] Processing DeleteTablet for tablet 37bab60afe7a4c758a08ef7c9c24aba0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:26 UTC) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:42.946381 10817 tablet_service.cc:1514] Processing DeleteTablet for tablet 69028aa7908c45b896da4ba8deef07b1 with delete_type TABLET_DATA_DELETED (replaced by tablet 2015c10f503e4751ad9ff77404fca50b) from {username='slave'} at 127.0.0.1:33344
I20260925 09:57:42.864463 11085 tablet_bootstrap.cc:492] T 3d84c717b6ac462eb331e25d80c918e4 P c43461c4ee854ae1ac52b873ccada318: Bootstrap starting.
I20260925 09:57:43.011170 11085 tablet_bootstrap.cc:654] T 3d84c717b6ac462eb331e25d80c918e4 P c43461c4ee854ae1ac52b873ccada318: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:43.042897 12027 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33344 (request call id 405) took 3496 ms. Trace:
I20260925 09:57:43.043443 12027 rpcz_store.cc:276] 0925 09:57:39.546226 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:39.546361 (+ 135us) service_pool.cc:227] Handling call
0925 09:57:42.724761 (+3178400us) ts_tablet_manager.cc:1145] Deleting tablet 10657a81058241fc988d37e5ea7ff8cf
0925 09:57:42.724795 (+ 34us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:57:42.800237 (+ 75442us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:42.806855 (+ 6618us) tablet_metadata.cc:641] Metadata flushed
0925 09:57:43.042661 (+235806us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3177846}
I20260925 09:57:43.045261 10379 catalog_manager.cc:4841] TS c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): tablet 10657a81058241fc988d37e5ea7ff8cf (table default.Table_64ba123898174e72a6884dbb9ad07de3 [id=7ee526720b2442119e0fc487b7c4fcd4]) successfully deleted
I20260925 09:57:43.046631 11084 tablet_bootstrap.cc:654] T 0a018bea4200474a9904f62aee37240b P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:43.078953 11085 tablet_bootstrap.cc:492] T 3d84c717b6ac462eb331e25d80c918e4 P c43461c4ee854ae1ac52b873ccada318: No bootstrap required, opened a new log
I20260925 09:57:43.079567 11085 ts_tablet_manager.cc:1397] T 3d84c717b6ac462eb331e25d80c918e4 P c43461c4ee854ae1ac52b873ccada318: Time spent bootstrapping tablet: real 0.215s user 0.009s sys 0.007s
I20260925 09:57:43.082083 10956 tablet_service.cc:1514] Processing DeleteTablet for tablet a8be01fe42894b96bbdd5ce2403ff1f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:39 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:43.082386 10952 tablet_service.cc:1514] Processing DeleteTablet for tablet 815b4b1bd914433798dcb5deec6ec79c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:30 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:43.082918 10945 tablet_service.cc:1514] Processing DeleteTablet for tablet 26fae3fbfe374f348a1729c21ec751b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:39 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:43.084096 11727 tablet_replica.cc:330] T 815b4b1bd914433798dcb5deec6ec79c P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
I20260925 09:57:43.084800 11727 raft_consensus.cc:2229] T 815b4b1bd914433798dcb5deec6ec79c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:43.085160 11727 raft_consensus.cc:2258] T 815b4b1bd914433798dcb5deec6ec79c P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:57:43.086746 11727 ts_tablet_manager.cc:1905] T 815b4b1bd914433798dcb5deec6ec79c P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:43.086838 11085 raft_consensus.cc:348] T 3d84c717b6ac462eb331e25d80c918e4 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:43.087751 11085 raft_consensus.cc:374] T 3d84c717b6ac462eb331e25d80c918e4 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:43.088001 11085 raft_consensus.cc:729] T 3d84c717b6ac462eb331e25d80c918e4 P c43461c4ee854ae1ac52b873ccada318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43461c4ee854ae1ac52b873ccada318, State: Initialized, Role: FOLLOWER
W20260925 09:57:43.089344 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet a8be01fe42894b96bbdd5ce2403ff1f4 because the tablet was not found. No further retry: Not found: Tablet not found: a8be01fe42894b96bbdd5ce2403ff1f4
W20260925 09:57:43.091681 10774 connection.cc:535] server connection from 127.10.9.1:46089 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 09:57:43.092621 10380 connection.cc:535] client connection to 127.10.9.1:33197 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 09:57:43.092631 10919 connection.cc:535] client connection to 127.10.9.1:33197 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 09:57:43.092973 10380 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107) [suppressed 2 similar messages]
W20260925 09:57:43.093017 10919 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 09:57:43.094014 12688 raft_consensus.cc:482] T 76b1f79efa784662a0adf032c074ce4a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:43.094420 12688 raft_consensus.cc:504] T 76b1f79efa784662a0adf032c074ce4a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:39.701100 12316 rpcz_store.cc:276] 0925 09:57:36.830490 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:36.830889 (+ 399us) service_pool.cc:227] Handling call
0925 09:57:37.126566 (+295677us) raft_consensus.cc:1457] Updating replica for 450 ops
0925 09:57:37.550327 (+423761us) raft_consensus.cc:1520] Early marking committed up to index 0
0925 09:57:37.550370 (+ 43us) raft_consensus.cc:1525] Triggering prepare for 450 ops
0925 09:57:37.550925 (+ 555us) write_op.cc:270] Start()
0925 09:57:37.551050 (+ 125us) write_op.cc:276] Timestamp: P: 1790330225180874 usec, L: 0
0925 09:57:37.620043 (+ 68993us) write_op.cc:270] Start()
0925 09:57:37.620133 (+ 90us) write_op.cc:276] Timestamp: P: 1790330225187962 usec, L: 0
0925 09:57:37.620632 (+ 499us) write_op.cc:270] Start()
0925 09:57:37.620769 (+ 137us) write_op.cc:276] Timestamp: P: 1790330225190875 usec, L: 0
0925 09:57:37.621400 (+ 631us) write_op.cc:270] Start()
0925 09:57:37.621475 (+ 75us) write_op.cc:276] Timestamp: P: 1790330225193202 usec, L: 0
0925 09:57:37.621939 (+ 464us) write_op.cc:270] Start()
0925 09:57:37.622036 (+ 97us) write_op.cc:276] Timestamp: P: 1790330225212428 usec, L: 0
0925 09:57:37.622429 (+ 393us) write_op.cc:270] Start()
0925 09:57:37.622519 (+ 90us) write_op.cc:276] Timestamp: P: 1790330225220193 usec, L: 0
0925 09:57:37.623077 (+ 558us) write_op.cc:270] Start()
0925 09:57:37.623242 (+ 165us) write_op.cc:276] Timestamp: P: 1790330225230883 usec, L: 0
0925 09:57:37.623704 (+ 462us) write_op.cc:270] Start()
0925 09:57:37.623823 (+ 119us) write_op.cc:276] Timestamp: P: 1790330225241784 usec, L: 0
0925 09:57:37.624399 (+ 576us) write_op.cc:270] Start()
0925 09:57:37.624483 (+ 84us) write_op.cc:276] Timestamp: P: 1790330225245310 usec, L: 0
0925 09:57:37.624895 (+ 412us) write_op.cc:270] Start()
0925 09:57:37.624971 (+ 76us) write_op.cc:276] Timestamp: P: 1790330225326678 usec, L: 0
0925 09:57:37.625576 (+ 605us) write_op.cc:270] Start()
0925 09:57:37.625675 (+ 99us) write_op.cc:276] Timestamp: P: 1790330225417001 usec, L: 0
0925 09:57:37.626092 (+ 417us) write_op.cc:270] Start()
0925 09:57:37.626184 (+ 92us) write_op.cc:276] Timestamp: P: 1790330225428253 usec, L: 0
0925 09:57:37.626806 (+ 622us) write_op.cc:270] Start()
0925 09:57:37.626882 (+ 76us) write_op.cc:276] Timestamp: P: 1790330225461634 usec, L: 0
0925 09:57:37.627289 (+ 407us) write_op.cc:270] Start()
0925 09:57:37.627370 (+ 81us) write_op.cc:276] Timestamp: P: 1790330225473708 usec, L: 0
0925 09:57:37.648101 (+ 20731us) write_op.cc:270] Start()
0925 09:57:37.648217 (+ 116us) write_op.cc:276] Timestamp: P: 1790330225484582 usec, L: 0
0925 09:57:37.648701 (+ 484us) write_op.cc:270] Start()
0925 09:57:37.648794 (+ 93us) write_op.cc:276] Timestamp: P: 1790330225488244 usec, L: 0
0925 09:57:37.649272 (+ 478us) write_op.cc:270] Start()
0925 09:57:37.649349 (+ 77us) write_op.cc:276] Timestamp: P: 1790330225491622 usec, L: 0
0925 09:57:37.649764 (+ 415us) write_op.cc:270] Start()
0925 09:57:37.649846 (+ 82us) write_op.cc:276] Timestamp: P: 1790330225546009 usec, L: 0
0925 09:57:37.650294 (+ 448us) write_op.cc:270] Start()
0925 09:57:37.650378 (+ 84us) write_op.cc:276] Timestamp: P: 1790330225573669 usec, L: 0
0925 09:57:37.650773 (+ 395us) write_op.cc:270] Start()
0925 09:57:37.650843 (+ 70us) write_op.cc:276] Timestamp: P: 1790330225589728 usec, L: 0
0925 09:57:37.651220 (+ 377us) write_op.cc:270] Start()
0925 09:57:37.651288 (+ 68us) write_op.cc:276] Timestamp: P: 1790330225688952 usec, L: 0
0925 09:57:37.651703 (+ 415us) write_op.cc:270] Start()
0925 09:57:37.651786 (+ 83us) write_op.cc:276] Timestamp: P: 1790330225696293 usec, L: 0
0925 09:57:37.667472 (+ 15686us) write_op.cc:270] Start()
0925 09:57:37.667577 (+ 105us) write_op.cc:276] Timestamp: P: 1790330225720215 usec, L: 0
0925 09:57:37.669337 (+ 1760us) write_op.cc:270] Start()
0925 09:57:37.669485 (+ 148us) write_op.cc:276] Timestamp: P: 1790330225727802 usec, L: 0
0925 09:57:37.671292 (+ 1807us) write_op.cc:270] Start()
0925 09:57:37.671403 (+ 111us) write_op.cc:276] Timestamp: P: 1790330225734167 usec, L: 0
0925 09:57:37.671864 (+ 461us) write_op.cc:270] Start()
0925 09:57:37.671940 (+ 76us) write_op.cc:276] Timestamp: P: 1790330225751134 usec, L: 0
0925 09:57:37.672346 (+ 406us) write_op.cc:270] Start()
0925 09:57:37.672427 (+ 81us) write_op.cc:276] Timestamp: P: 1790330225805045 usec, L: 0
0925 09:57:37.677285 (+ 4858us) write_op.cc:270] Start()
0925 09:57:37.677386 (+ 101us) write_op.cc:276] Timestamp: P: 1790330225849594 usec, L: 0
0925 09:57:37.677836 (+ 450us) write_op.cc:270] Start()
0925 09:57:37.677906 (+ 70us) write_op.cc:276] Timestamp: P: 1790330225913148 usec, L: 0
0925 09:57:37.678285 (+ 379us) write_op.cc:270] Start()
0925 09:57:37.678376 (+ 91us) write_op.cc:276] Timestamp: P: 1790330225959988 usec, L: 0
0925 09:57:37.678805 (+ 429us) write_op.cc:270] Start()
0925 09:57:37.678885 (+ 80us) write_op.cc:276] Timestamp: P: 1790330226007680 usec, L: 0
0925 09:57:37.679333 (+ 448us) write_op.cc:270] Start()
0925 09:57:37.679475 (+ 142us) write_op.cc:276] Timestamp: P: 1790330226026659 usec, L: 0
0925 09:57:37.679981 (+ 506us) write_op.cc:270] Start()
0925 09:57:37.680062 (+ 81us) write_op.cc:276] Timestamp: P: 1790330226043173 usec, L: 0
0925 09:57:37.680493 (+ 431us) write_op.cc:270] Start()
0925 09:57:37.680591 (+ 98us) write_op.cc:276] Timestamp: P: 1790330226111142 usec, L: 0
0925 09:57:37.681024 (+ 433us) write_op.cc:270] Start()
0925 09:57:37.681101 (+ 77us) write_op.cc:276] Timestamp: P: 1790330226145672 usec, L: 0
0925 09:57:37.681561 (+ 460us) write_op.cc:270] Start()
0925 09:57:37.681640 (+ 79us) write_op.cc:276] Timestamp: P: 1790330226189809 usec, L: 0
0925 09:57:37.682099 (+ 459us) write_op.cc:270] Start()
0925 09:57:37.682181 (+ 82us) write_op.cc:276] Timestamp: P: 1790330226205701 usec, L: 0
0925 09:57:37.682626 (+ 445us) write_op.cc:270] Start()
0925 09:57:37.682720 (+ 94us) write_op.cc:276] Timestamp: P: 1790330226268101 usec, L: 0
0925 09:57:37.683164 (+ 444us) write_op.cc:270] Start()
0925 09:57:37.683238 (+ 74us) write_op.cc:276] Timestamp: P: 1790330226368808 usec, L: 0
0925 09:57:37.696307 (+ 13069us) write_op.cc:270] Start()
0925 09:57:37.696416 (+ 109us) write_op.cc:276] Timestamp: P: 1790330226377280 usec, L: 0
0925 09:57:37.696975 (+ 559us) write_op.cc:270] Start()
0925 09:57:37.697087 (+ 112us) write_op.cc:276] Timestamp: P: 1790330226415633 usec, L: 0
0925 09:57:37.697766 (+ 679us) write_op.cc:270] Start()
0925 09:57:37.697846 (+ 80us) write_op.cc:276] Timestamp: P: 1790330226499748 usec, L: 0
0925 09:57:37.698350 (+ 504us) write_op.cc:270] Start()
0925 09:57:37.698428 (+ 78us) write_op.cc:276] Timestamp: P: 1790330226544192 usec, L: 0
0925 09:57:37.699651 (+ 1223us) write_op.cc:270] Start()
0925 09:57:37.703594 (+ 3943us) write_op.cc:276] Timestamp: P: 1790330226550291 usec, L: 0
0925 09:57:37.704181 (+ 587us) write_op.cc:270] Start()
0925 09:57:37.704292 (+ 111us) write_op.cc:276] Timestamp: P: 1790330226594968 usec, L: 0
0925 09:57:37.704805 (+ 513us) write_op.cc:270] Start()
0925 09:57:37.704895 (+ 90us) write_op.cc:276] Timestamp: P: 1790330226619939 usec, L: 0
0925 09:57:37.705366 (+ 471us) write_op.cc:270] Start()
0925 09:57:37.705455 (+ 89us) write_op.cc:276] Timestamp: P: 1790330226715873 usec, L: 0
0925 09:57:37.705924 (+ 469us) write_op.cc:270] Start()
0925 09:57:37.706108 (+ 184us) write_op.cc:276] Timestamp: P: 1790330226766949 usec, L: 0
0925 09:57:37.706645 (+ 537us) write_op.cc:270] Start()
0925 09:57:37.706728 (+ 83us) write_op.cc:276] Timestamp: P: 1790330226795282 usec, L: 0
0925 09:57:37.707166 (+ 438us) write_op.cc:270] Start()
0925 09:57:37.707258 (+ 92us) write_op.cc:276] Timestamp: P: 1790330226819293 usec, L: 0
0925 09:57:37.711790 (+ 4532us) write_op.cc:270] Start()
0925 09:57:37.711911 (+ 121us) write_op.cc:276] Timestamp: P: 1790330226833748 usec, L: 0
0925 09:57:37.714210 (+ 2299us) write_op.cc:270] Start()
0925 09:57:37.714297 (+ 87us) write_op.cc:276] Timestamp: P: 1790330226861916 usec, L: 0
0925 09:57:37.714764 (+ 467us) write_op.cc:270] Start()
0925 09:57:37.714843 (+ 79us) write_op.cc:276] Timestamp: P: 1790330226869436 usec, L: 0
0925 09:57:37.715309 (+ 466us) write_op.cc:270] Start()
0925 09:57:37.715406 (+ 97us) write_op.cc:276] Timestamp: P: 1790330226905382 usec, L: 0
0925 09:57:37.715885 (+ 479us) write_op.cc:270] Start()
0925 09:57:37.715964 (+ 79us) write_op.cc:276] Timestamp: P: 1790330226925028 usec, L: 0
0925 09:57:37.716485 (+ 521us) write_op.cc:270] Start()
0925 09:57:37.716566 (+ 81us) write_op.cc:276] Timestamp: P: 1790330226996107 usec, L: 0
0925 09:57:37.717073 (+ 507us) write_op.cc:270] Start()
0925 09:57:37.717157 (+ 84us) write_op.cc:276] Timestamp: P: 1790330227068190 usec, L: 0
0925 09:57:37.717675 (+ 518us) write_op.cc:270] Start()
0925 09:57:37.717751 (+ 76us) write_op.cc:276] Timestamp: P: 1790330227124349 usec, L: 0
0925 09:57:37.718217 (+ 466us) write_op.cc:270] Start()
0925 09:57:37.718297 (+ 80us) write_op.cc:276] Timestamp: P: 1790330227198696 usec, L: 0
0925 09:57:37.727943 (+ 9646us) write_op.cc:270] Start()
0925 09:57:37.728128 (+ 185us) write_op.cc:276] Timestamp: P: 1790330227209426 usec, L: 0
0925 09:57:37.728673 (+ 545us) write_op.cc:270] Start()
0925 09:57:37.728762 (+ 89us) write_op.cc:276] Timestamp: P: 1790330227220008 usec, L: 0
0925 09:57:37.729286 (+ 524us) write_op.cc:270] Start()
0925 09:57:37.729390 (+ 104us) write_op.cc:276] Timestamp: P: 1790330227321413 usec, L: 0
0925 09:57:37.729971 (+ 581us) write_op.cc:270] Start()
0925 09:57:37.730061 (+ 90us) write_op.cc:276] Timestamp: P: 1790330227323667 usec, L: 0
0925 09:57:37.730553 (+ 492us) write_op.cc:270] Start()
0925 09:57:37.730639 (+ 86us) write_op.cc:276] Timestamp: P: 1790330227330983 usec, L: 0
0925 09:57:37.731340 (+ 701us) write_op.cc:270] Start()
0925 09:57:37.731506 (+ 166us) write_op.cc:276] Timestamp: P: 1790330227335141 usec, L: 0
0925 09:57:37.732017 (+ 511us) write_op.cc:270] Start()
0925 09:57:37.732101 (+ 84us) write_op.cc:276] Timestamp: P: 1790330227410816 usec, L: 0
0925 09:57:37.732635 (+ 534us) write_op.cc:270] Start()
0925 09:57:37.732725 (+ 90us) write_op.cc:276] Timestamp: P: 1790330227508896 usec, L: 0
0925 09:57:37.733182 (+ 457us) write_op.cc:270] Start()
0925 09:57:37.733281 (+ 99us) write_op.cc:276] Timestamp: P: 1790330227751377 usec, L: 0
0925 09:57:37.733738 (+ 457us) write_op.cc:270] Start()
0925 09:57:37.733839 (+ 101us) write_op.cc:276] Timestamp: P: 1790330227754018 usec, L: 0
0925 09:57:37.734338 (+ 499us) write_op.cc:270] Start()
0925 09:57:37.734436 (+ 98us) write_op.cc:276] Timestamp: P: 1790330227814674 usec, L: 0
0925 09:57:37.734913 (+ 477us) write_op.cc:270] Start()
0925 09:57:37.734998 (+ 85us) write_op.cc:276] Timestamp: P: 1790330227840999 usec, L: 0
0925 09:57:37.735528 (+ 530us) write_op.cc:270] Start()
0925 09:57:37.735615 (+ 87us) write_op.cc:276] Timestamp: P: 1790330227908351 usec, L: 0
0925 09:57:37.736099 (+ 484us) write_op.cc:270] Start()
0925 09:57:37.736191 (+ 92us) write_op.cc:276] Timestamp: P: 1790330227977613 usec, L: 0
0925 09:57:37.756646 (+ 20455us) write_op.cc:270] Start()
0925 09:57:37.756755 (+ 109us) write_op.cc:276] Timestamp: P: 1790330228083917 usec, L: 0
0925 09:57:37.757331 (+ 576us) write_op.cc:270] Start()
0925 09:57:37.757414 (+ 83us) write_op.cc:276] Timestamp: P: 1790330228117021 usec, L: 0
0925 09:57:37.757872 (+ 458us) write_op.cc:270] Start()
0925 09:57:37.757965 (+ 93us) write_op.cc:276] Timestamp: P: 1790330228166212 usec, L: 0
0925 09:57:37.758461 (+ 496us) write_op.cc:270] Start()
0925 09:57:37.758550 (+ 89us) write_op.cc:276] Timestamp: P: 1790330228192395 usec, L: 0
0925 09:57:37.759020 (+ 470us) write_op.cc:270] Start()
0925 09:57:37.759106 (+ 86us) write_op.cc:276] Timestamp: P: 1790330228200083 usec, L: 0
0925 09:57:37.759636 (+ 530us) write_op.cc:270] Start()
0925 09:57:37.759722 (+ 86us) write_op.cc:276] Timestamp: P: 1790330228218228 usec, L: 0
0925 09:57:37.760243 (+ 521us) write_op.cc:270] Start()
0925 09:57:37.760327 (+ 84us) write_op.cc:276] Timestamp: P: 1790330228296892 usec, L: 0
0925 09:57:37.760792 (+ 465us) write_op.cc:270] Start()
0925 09:57:37.760883 (+ 91us) write_op.cc:276] Timestamp: P: 1790330228350176 usec, L: 0
0925 09:57:37.790068 (+ 29185us) write_op.cc:270] Start()
0925 09:57:37.790191 (+ 123us) write_op.cc:276] Timestamp: P: 1790330228383928 usec, L: 0
0925 09:57:37.790821 (+ 630us) write_op.cc:270] Start()
0925 09:57:37.790911 (+ 90us) write_op.cc:276] Timestamp: P: 1790330228481536 usec, L: 0
0925 09:57:37.791452 (+ 541us) write_op.cc:270] Start()
0925 09:57:37.791541 (+ 89us) write_op.cc:276] Timestamp: P: 1790330228557081 usec, L: 0
0925 09:57:37.792066 (+ 525us) write_op.cc:270] Start()
0925 09:57:37.792154 (+ 88us) write_op.cc:276] Timestamp: P: 1790330228643134 usec, L: 0
0925 09:57:37.792663 (+ 509us) write_op.cc:270] Start()
0925 09:57:37.792752 (+ 89us) write_op.cc:276] Timestamp: P: 1790330228654779 usec, L: 0
0925 09:57:37.793246 (+ 494us) write_op.cc:270] Start()
0925 09:57:37.793354 (+ 108us) write_op.cc:276] Timestamp: P: 1790330228696443 usec, L: 0
0925 09:57:37.793857 (+ 503us) write_op.cc:270] Start()
0925 09:57:37.793948 (+ 91us) write_op.cc:276] Timestamp: P: 1790330228700104 usec, L: 0
0925 09:57:37.794481 (+ 533us) write_op.cc:270] Start()
0925 09:57:37.794569 (+ 88us) write_op.cc:276] Timestamp: P: 1790330228715497 usec, L: 0
0925 09:57:37.795094 (+ 525us) write_op.cc:270] Start()
0925 09:57:37.795187 (+ 93us) write_op.cc:276] Timestamp: P: 1790330228718410 usec, L: 0
0925 09:57:37.813641 (+ 18454us) write_op.cc:270] Start()
0925 09:57:37.813756 (+ 115us) write_op.cc:276] Timestamp: P: 1790330229156506 usec, L: 0
0925 09:57:37.814318 (+ 562us) write_op.cc:270] Start()
0925 09:57:37.814406 (+ 88us) write_op.cc:276] Timestamp: P: 1790330229158771 usec, L: 0
0925 09:57:37.814902 (+ 496us) write_op.cc:270] Start()
0925 09:57:37.814987 (+ 85us) write_op.cc:276] Timestamp: P: 1790330229209566 usec, L: 0
0925 09:57:37.815484 (+ 497us) write_op.cc:270] Start()
0925 09:57:37.815575 (+ 91us) write_op.cc:276] Timestamp: P: 1790330229296926 usec, L: 0
0925 09:57:37.816033 (+ 458us) write_op.cc:270] Start()
0925 09:57:37.816129 (+ 96us) write_op.cc:276] Timestamp: P: 1790330229302060 usec, L: 0
0925 09:57:37.816689 (+ 560us) write_op.cc:270] Start()
0925 09:57:37.816766 (+ 77us) write_op.cc:276] Timestamp: P: 1790330229325654 usec, L: 0
0925 09:57:37.817178 (+ 412us) write_op.cc:270] Start()
0925 09:57:37.817263 (+ 85us) write_op.cc:276] Timestamp: P: 1790330229336834 usec, L: 0
0925 09:57:37.817701 (+ 438us) write_op.cc:270] Start()
0925 09:57:37.817922 (+ 221us) write_op.cc:276] Timestamp: P: 1790330229348233 usec, L: 0
0925 09:57:37.818532 (+ 610us) write_op.cc:270] Start()
0925 09:57:37.818618 (+ 86us) write_op.cc:276] Timestamp: P: 1790330229547173 usec, L: 0
0925 09:57:37.819102 (+ 484us) write_op.cc:270] Start()
0925 09:57:37.819172 (+ 70us) write_op.cc:276] Timestamp: P: 1790330229690705 usec, L: 0
0925 09:57:37.850979 (+ 31807us) write_op.cc:270] Start()
0925 09:57:37.851070 (+ 91us) write_op.cc:276] Timestamp: P: 1790330229748631 usec, L: 0
0925 09:57:37.851601 (+ 531us) write_op.cc:270] Start()
0925 09:57:37.851685 (+ 84us) write_op.cc:276] Timestamp: P: 1790330229763626 usec, L: 0
0925 09:57:37.852122 (+ 437us) write_op.cc:270] Start()
0925 09:57:37.852193 (+ 71us) write_op.cc:276] Timestamp: P: 1790330229883845 usec, L: 0
0925 09:57:37.852575 (+ 382us) write_op.cc:270] Start()
0925 09:57:37.852639 (+ 64us) write_op.cc:276] Timestamp: P: 1790330229987338 usec, L: 0
0925 09:57:37.853212 (+ 573us) write_op.cc:270] Start()
0925 09:57:37.853285 (+ 73us) write_op.cc:276] Timestamp: P: 1790330230023306 usec, L: 0
0925 09:57:37.853672 (+ 387us) write_op.cc:270] Start()
0925 09:57:37.853737 (+ 65us) write_op.cc:276] Timestamp: P: 1790330230118393 usec, L: 0
0925 09:57:37.854157 (+ 420us) write_op.cc:270] Start()
0925 09:57:37.854238 (+ 81us) write_op.cc:276] Timestamp: P: 1790330230140752 usec, L: 0
0925 09:57:37.854634 (+ 396us) write_op.cc:270] Start()
0925 09:57:37.854722 (+ 88us) write_op.cc:276] Timestamp: P: 1790330230214692 usec, L: 0
0925 09:57:37.855139 (+ 417us) write_op.cc:270] Start()
0925 09:57:37.855238 (+ 99us) write_op.cc:276] Timestamp: P: 1790330230342852 usec, L: 0
0925 09:57:37.860935 (+ 5697us) write_op.cc:270] Start()
0925 09:57:37.861011 (+ 76us) write_op.cc:276] Timestamp: P: 1790330230361321 usec, L: 0
0925 09:57:37.861435 (+ 424us) write_op.cc:270] Start()
0925 09:57:37.861508 (+ 73us) write_op.cc:276] Timestamp: P: 1790330230528505 usec, L: 0
0925 09:57:37.861919 (+ 411us) write_op.cc:270] Start()
0925 09:57:37.862019 (+ 100us) write_op.cc:276] Timestamp: P: 1790330230532285 usec, L: 0
0925 09:57:37.862415 (+ 396us) write_op.cc:270] Start()
0925 09:57:37.862488 (+ 73us) write_op.cc:276] Timestamp: P: 1790330230607214 usec, L: 0
0925 09:57:37.862876 (+ 388us) write_op.cc:270] Start()
0925 09:57:37.862945 (+ 69us) write_op.cc:276] Timestamp: P: 1790330230625554 usec, L: 0
0925 09:57:37.863340 (+ 395us) write_op.cc:270] Start()
0925 09:57:37.863434 (+ 94us) write_op.cc:276] Timestamp: P: 1790330230748355 usec, L: 0
0925 09:57:37.863808 (+ 374us) write_op.cc:270] Start()
0925 09:57:37.863873 (+ 65us) write_op.cc:276] Timestamp: P: 1790330230802333 usec, L: 0
0925 09:57:37.864251 (+ 378us) write_op.cc:270] Start()
0925 09:57:37.864317 (+ 66us) write_op.cc:276] Timestamp: P: 1790330230887725 usec, L: 0
0925 09:57:37.864671 (+ 354us) write_op.cc:270] Start()
0925 09:57:37.864736 (+ 65us) write_op.cc:276] Timestamp: P: 1790330230977950 usec, L: 0
0925 09:57:37.865125 (+ 389us) write_op.cc:270] Start()
0925 09:57:37.865204 (+ 79us) write_op.cc:276] Timestamp: P: 1790330231203655 usec, L: 0
0925 09:57:37.865557 (+ 353us) write_op.cc:270] Start()
0925 09:57:37.865622 (+ 65us) write_op.cc:276] Timestamp: P: 1790330231248395 usec, L: 0
0925 09:57:37.866019 (+ 397us) write_op.cc:270] Start()
0925 09:57:37.866083 (+ 64us) write_op.cc:276] Timestamp: P: 1790330231251918 usec, L: 0
0925 09:57:37.866467 (+ 384us) write_op.cc:270] Start()
0925 09:57:37.866533 (+ 66us) write_op.cc:276] Timestamp: P: 1790330231276916 usec, L: 0
0925 09:57:37.879522 (+ 12989us) write_op.cc:270] Start()
0925 09:57:37.879608 (+ 86us) write_op.cc:276] Timestamp: P: 1790330231318314 usec, L: 0
0925 09:57:37.880052 (+ 444us) write_op.cc:270] Start()
0925 09:57:37.880414 (+ 362us) write_op.cc:276] Timestamp: P: 1790330231342439 usec, L: 0
0925 09:57:37.880825 (+ 411us) write_op.cc:270] Start()
0925 09:57:37.880891 (+ 66us) write_op.cc:276] Timestamp: P: 1790330231499301 usec, L: 0
0925 09:57:37.881282 (+ 391us) write_op.cc:270] Start()
0925 09:57:37.881363 (+ 81us) write_op.cc:276] Timestamp: P: 1790330231531862 usec, L: 0
0925 09:57:37.881768 (+ 405us) write_op.cc:270] Start()
0925 09:57:37.881839 (+ 71us) write_op.cc:276] Timestamp: P: 1790330231541292 usec, L: 0
0925 09:57:37.882233 (+ 394us) write_op.cc:270] Start()
0925 09:57:37.882297 (+ 64us) write_op.cc:276] Timestamp: P: 1790330231583950 usec, L: 0
0925 09:57:37.882853 (+ 556us) write_op.cc:270] Start()
0925 09:57:37.882917 (+ 64us) write_op.cc:276] Timestamp: P: 1790330231639349 usec, L: 0
0925 09:57:37.883306 (+ 389us) write_op.cc:270] Start()
0925 09:57:37.883369 (+ 63us) write_op.cc:276] Timestamp: P: 1790330231655870 usec, L: 0
0925 09:57:37.883784 (+ 415us) write_op.cc:270] Start()
0925 09:57:37.883929 (+ 145us) write_op.cc:276] Timestamp: P: 1790330231725557 usec, L: 0
0925 09:57:37.884351 (+ 422us) write_op.cc:270] Start()
0925 09:57:37.884421 (+ 70us) write_op.cc:276] Timestamp: P: 1790330231852237 usec, L: 0
0925 09:57:37.884933 (+ 512us) write_op.cc:270] Start()
0925 09:57:37.885004 (+ 71us) write_op.cc:276] Timestamp: P: 1790330231874970 usec, L: 0
0925 09:57:37.885395 (+ 391us) write_op.cc:270] Start()
0925 09:57:37.885457 (+ 62us) write_op.cc:276] Timestamp: P: 1790330231900317 usec, L: 0
0925 09:57:37.885802 (+ 345us) write_op.cc:270] Start()
0925 09:57:37.885868 (+ 66us) write_op.cc:276] Timestamp: P: 1790330231910664 usec, L: 0
0925 09:57:37.886217 (+ 349us) write_op.cc:270] Start()
0925 09:57:37.886290 (+ 73us) write_op.cc:276] Timestamp: P: 1790330231938186 usec, L: 0
0925 09:57:37.886677 (+ 387us) write_op.cc:270] Start()
0925 09:57:37.886740 (+ 63us) write_op.cc:276] Timestamp: P: 1790330231942483 usec, L: 0
0925 09:57:37.887104 (+ 364us) write_op.cc:270] Start()
0925 09:57:37.887171 (+ 67us) write_op.cc:276] Timestamp: P: 1790330231947872 usec, L: 0
0925 09:57:37.905696 (+ 18525us) write_op.cc:270] Start()
0925 09:57:37.905779 (+ 83us) write_op.cc:276] Timestamp: P: 1790330231954349 usec, L: 0
0925 09:57:37.906183 (+ 404us) write_op.cc:270] Start()
0925 09:57:37.906264 (+ 81us) write_op.cc:276] Timestamp: P: 1790330231973408 usec, L: 0
0925 09:57:37.915704 (+ 9440us) write_op.cc:270] Start()
0925 09:57:37.915776 (+ 72us) write_op.cc:276] Timestamp: P: 1790330231999853 usec, L: 0
0925 09:57:37.916175 (+ 399us) write_op.cc:270] Start()
0925 09:57:37.916237 (+ 62us) write_op.cc:276] Timestamp: P: 1790330232102954 usec, L: 0
0925 09:57:37.916591 (+ 354us) write_op.cc:270] Start()
0925 09:57:37.916658 (+ 67us) write_op.cc:276] Timestamp: P: 1790330232117111 usec, L: 0
0925 09:57:37.916991 (+ 333us) write_op.cc:270] Start()
0925 09:57:37.917050 (+ 59us) write_op.cc:276] Timestamp: P: 1790330232131538 usec, L: 0
0925 09:57:37.917395 (+ 345us) write_op.cc:270] Start()
0925 09:57:37.917454 (+ 59us) write_op.cc:276] Timestamp: P: 1790330232152555 usec, L: 0
0925 09:57:37.917765 (+ 311us) write_op.cc:270] Start()
0925 09:57:37.917821 (+ 56us) write_op.cc:276] Timestamp: P: 1790330232181038 usec, L: 0
0925 09:57:37.918164 (+ 343us) write_op.cc:270] Start()
0925 09:57:37.918223 (+ 59us) write_op.cc:276] Timestamp: P: 1790330232229731 usec, L: 0
0925 09:57:37.918563 (+ 340us) write_op.cc:270] Start()
0925 09:57:37.918624 (+ 61us) write_op.cc:276] Timestamp: P: 1790330232241688 usec, L: 0
0925 09:57:37.918945 (+ 321us) write_op.cc:270] Start()
0925 09:57:37.919016 (+ 71us) write_op.cc:276] Timestamp: P: 1790330232245285 usec, L: 0
0925 09:57:37.919404 (+ 388us) write_op.cc:270] Start()
0925 09:57:37.919481 (+ 77us) write_op.cc:276] Timestamp: P: 1790330232575450 usec, L: 0
0925 09:57:37.937159 (+ 17678us) write_op.cc:270] Start()
0925 09:57:37.937265 (+ 106us) write_op.cc:276] Timestamp: P: 1790330232579151 usec, L: 0
0925 09:57:37.943005 (+ 5740us) write_op.cc:270] Start()
0925 09:57:37.943083 (+ 78us) write_op.cc:276] Timestamp: P: 1790330232596066 usec, L: 0
0925 09:57:37.943583 (+ 500us) write_op.cc:270] Start()
0925 09:57:37.943650 (+ 67us) write_op.cc:276] Timestamp: P: 1790330232654422 usec, L: 0
0925 09:57:37.944137 (+ 487us) write_op.cc:270] Start()
0925 09:57:37.944201 (+ 64us) write_op.cc:276] Timestamp: P: 1790330232671890 usec, L: 0
0925 09:57:37.944559 (+ 358us) write_op.cc:270] Start()
0925 09:57:37.944620 (+ 61us) write_op.cc:276] Timestamp: P: 1790330232682850 usec, L: 0
0925 09:57:37.944972 (+ 352us) write_op.cc:270] Start()
0925 09:57:37.945040 (+ 68us) write_op.cc:276] Timestamp: P: 1790330232706047 usec, L: 0
0925 09:57:37.945411 (+ 371us) write_op.cc:270] Start()
0925 09:57:37.945475 (+ 64us) write_op.cc:276] Timestamp: P: 1790330232723258 usec, L: 0
0925 09:57:37.945816 (+ 341us) write_op.cc:270] Start()
0925 09:57:37.945883 (+ 67us) write_op.cc:276] Timestamp: P: 1790330232743016 usec, L: 0
0925 09:57:37.946213 (+ 330us) write_op.cc:270] Start()
0925 09:57:37.946272 (+ 59us) write_op.cc:276] Timestamp: P: 1790330232762038 usec, L: 0
0925 09:57:37.946619 (+ 347us) write_op.cc:270] Start()
0925 09:57:37.946679 (+ 60us) write_op.cc:276] Timestamp: P: 1790330232826940 usec, L: 0
0925 09:57:37.947000 (+ 321us) write_op.cc:270] Start()
0925 09:57:37.947062 (+ 62us) write_op.cc:276] Timestamp: P: 1790330232897253 usec, L: 0
0925 09:57:37.947416 (+ 354us) write_op.cc:270] Start()
0925 09:57:37.947476 (+ 60us) write_op.cc:276] Timestamp: P: 1790330232967915 usec, L: 0
0925 09:57:37.952983 (+ 5507us) write_op.cc:270] Start()
0925 09:57:37.953057 (+ 74us) write_op.cc:276] Timestamp: P: 1790330233362328 usec, L: 0
0925 09:57:37.953498 (+ 441us) write_op.cc:270] Start()
0925 09:57:37.953571 (+ 73us) write_op.cc:276] Timestamp: P: 1790330233365319 usec, L: 0
0925 09:57:37.953978 (+ 407us) write_op.cc:270] Start()
0925 09:57:37.954050 (+ 72us) write_op.cc:276] Timestamp: P: 1790330233367060 usec, L: 0
0925 09:57:37.954450 (+ 400us) write_op.cc:270] Start()
0925 09:57:37.954516 (+ 66us) write_op.cc:276] Timestamp: P: 1790330233545657 usec, L: 0
0925 09:57:37.954992 (+ 476us) write_op.cc:270] Start()
0925 09:57:37.955062 (+ 70us) write_op.cc:276] Timestamp: P: 1790330233692962 usec, L: 0
0925 09:57:37.955505 (+ 443us) write_op.cc:270] Start()
0925 09:57:37.956450 (+ 945us) write_op.cc:276] Timestamp: P: 1790330233774496 usec, L: 0
0925 09:57:37.959895 (+ 3445us) write_op.cc:270] Start()
0925 09:57:37.959961 (+ 66us) write_op.cc:276] Timestamp: P: 1790330233899606 usec, L: 0
0925 09:57:37.960340 (+ 379us) write_op.cc:270] Start()
0925 09:57:37.960419 (+ 79us) write_op.cc:276] Timestamp: P: 1790330233902565 usec, L: 0
0925 09:57:37.960856 (+ 437us) write_op.cc:270] Start()
0925 09:57:37.960918 (+ 62us) write_op.cc:276] Timestamp: P: 1790330234054783 usec, L: 0
0925 09:57:37.961260 (+ 342us) write_op.cc:270] Start()
0925 09:57:37.961321 (+ 61us) write_op.cc:276] Timestamp: P: 1790330234166395 usec, L: 0
0925 09:57:37.961650 (+ 329us) write_op.cc:270] Start()
0925 09:57:37.961708 (+ 58us) write_op.cc:276] Timestamp: P: 1790330234203319 usec, L: 0
0925 09:57:37.962066 (+ 358us) write_op.cc:270] Start()
0925 09:57:37.962132 (+ 66us) write_op.cc:276] Timestamp: P: 1790330234206997 usec, L: 0
0925 09:57:37.962491 (+ 359us) write_op.cc:270] Start()
0925 09:57:37.962554 (+ 63us) write_op.cc:276] Timestamp: P: 1790330234649183 usec, L: 0
0925 09:57:37.962873 (+ 319us) write_op.cc:270] Start()
0925 09:57:37.962932 (+ 59us) write_op.cc:276] Timestamp: P: 1790330234655887 usec, L: 0
0925 09:57:37.963265 (+ 333us) write_op.cc:270] Start()
0925 09:57:37.963326 (+ 61us) write_op.cc:276] Timestamp: P: 1790330234659824 usec, L: 0
0925 09:57:37.963697 (+ 371us) write_op.cc:270] Start()
0925 09:57:37.963754 (+ 57us) write_op.cc:276] Timestamp: P: 1790330234664838 usec, L: 0
0925 09:57:37.964091 (+ 337us) write_op.cc:270] Start()
0925 09:57:37.964150 (+ 59us) write_op.cc:276] Timestamp: P: 1790330234704469 usec, L: 0
0925 09:57:37.964488 (+ 338us) write_op.cc:270] Start()
0925 09:57:37.964550 (+ 62us) write_op.cc:276] Timestamp: P: 1790330234924946 usec, L: 0
0925 09:57:37.964878 (+ 328us) write_op.cc:270] Start()
0925 09:57:37.964937 (+ 59us) write_op.cc:276] Timestamp: P: 1790330234939182 usec, L: 0
0925 09:57:37.965267 (+ 330us) write_op.cc:270] Start()
0925 09:57:37.965334 (+ 67us) write_op.cc:276] Timestamp: P: 1790330235008717 usec, L: 0
0925 09:57:37.965673 (+ 339us) write_op.cc:270] Start()
0925 09:57:37.965731 (+ 58us) write_op.cc:276] Timestamp: P: 1790330235012781 usec, L: 0
0925 09:57:37.966076 (+ 345us) write_op.cc:270] Start()
0925 09:57:37.966137 (+ 61us) write_op.cc:276] Timestamp: P: 1790330235076525 usec, L: 0
0925 09:57:37.966540 (+ 403us) write_op.cc:270] Start()
0925 09:57:37.966617 (+ 77us) write_op.cc:276] Timestamp: P: 1790330235113977 usec, L: 0
0925 09:57:37.967070 (+ 453us) write_op.cc:270] Start()
0925 09:57:37.967152 (+ 82us) write_op.cc:276] Timestamp: P: 1790330235145159 usec, L: 0
0925 09:57:37.983047 (+ 15895us) write_op.cc:270] Start()
0925 09:57:37.983125 (+ 78us) write_op.cc:276] Timestamp: P: 1790330235171231 usec, L: 0
0925 09:57:37.983590 (+ 465us) write_op.cc:270] Start()
0925 09:57:37.983660 (+ 70us) write_op.cc:276] Timestamp: P: 1790330235188896 usec, L: 0
0925 09:57:37.984009 (+ 349us) write_op.cc:270] Start()
0925 09:57:37.984069 (+ 60us) write_op.cc:276] Timestamp: P: 1790330235245104 usec, L: 0
0925 09:57:37.984423 (+ 354us) write_op.cc:270] Start()
0925 09:57:37.984489 (+ 66us) write_op.cc:276] Timestamp: P: 1790330235545457 usec, L: 0
0925 09:57:37.984830 (+ 341us) write_op.cc:270] Start()
0925 09:57:37.984888 (+ 58us) write_op.cc:276] Timestamp: P: 1790330235554888 usec, L: 0
0925 09:57:37.985207 (+ 319us) write_op.cc:270] Start()
0925 09:57:37.985268 (+ 61us) write_op.cc:276] Timestamp: P: 1790330235561379 usec, L: 0
0925 09:57:37.985651 (+ 383us) write_op.cc:270] Start()
0925 09:57:37.985709 (+ 58us) write_op.cc:276] Timestamp: P: 1790330235668921 usec, L: 0
0925 09:57:37.986048 (+ 339us) write_op.cc:270] Start()
0925 09:57:37.986106 (+ 58us) write_op.cc:276] Timestamp: P: 1790330235918947 usec, L: 0
0925 09:57:37.986437 (+ 331us) write_op.cc:270] Start()
0925 09:57:37.986495 (+ 58us) write_op.cc:276] Timestamp: P: 1790330235930731 usec, L: 0
0925 09:57:37.986826 (+ 331us) write_op.cc:270] Start()
0925 09:57:37.986888 (+ 62us) write_op.cc:276] Timestamp: P: 1790330236008772 usec, L: 0
0925 09:57:37.987211 (+ 323us) write_op.cc:270] Start()
0925 09:57:37.987268 (+ 57us) write_op.cc:276] Timestamp: P: 17903302
I20260925 09:57:43.096347 12688 leader_election.cc:288] T 76b1f79efa784662a0adf032c074ce4a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
W20260925 09:57:43.101063 10775 connection.cc:535] client connection to 127.10.9.1:33197 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 09:57:43.101588 10775 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 09:57:43.102258 10377 connection.cc:535] server connection from 127.10.9.1:56563 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 09:57:43.102797 12688 raft_consensus.cc:482] T 476567ad4efa4043b6536f3f28d2ee28 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:43.103176 12688 raft_consensus.cc:504] T 476567ad4efa4043b6536f3f28d2ee28 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:43.103935 10824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76b1f79efa784662a0adf032c074ce4a" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:43.104959 12688 leader_election.cc:288] T 476567ad4efa4043b6536f3f28d2ee28 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
W20260925 09:57:43.105170 10918 leader_election.cc:341] T 76b1f79efa784662a0adf032c074ce4a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:43.104511 11084 tablet_bootstrap.cc:492] T 0a018bea4200474a9904f62aee37240b P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:43.105854 11084 ts_tablet_manager.cc:1397] T 0a018bea4200474a9904f62aee37240b P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.161s user 0.005s sys 0.004s
I20260925 09:57:43.105882 10824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "476567ad4efa4043b6536f3f28d2ee28" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
W20260925 09:57:43.106988 10918 leader_election.cc:341] T 476567ad4efa4043b6536f3f28d2ee28 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:57:43.125241 10387 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.3:41469 (request call id 37) took 998 ms (client timeout 1000 ms). Trace:
W20260925 09:57:43.125597 10387 rpcz_store.cc:269] 0925 09:57:42.127330 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:57:42.127477 (+ 147us) service_pool.cc:227] Handling call
0925 09:57:42.220238 (+ 92761us) spinlock_profiling.cc:243] Waited 75.8 ms on lock 0x7b54000118d4. stack: 00007fabe9b84033 00007fabe9b83f30 00007fabe9705534 00000000004db2b4 00000000004db249 00007fabf0a90af0 00007fabf0b13f2a 00007fabf29dfd6c 00007fabf2ad9c3e 00007fabf0610fb4 00007fabf0610f42 00007fabf0610e71 00007fabf0610dec 00007fabf06100a2 00007fabee8ba94c 00007fabee8b9d86
0925 09:57:43.125180 (+904942us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":94321,"reported_tablets":76,"spinlock_wait_cycles":174429568}
I20260925 09:57:43.123425 12688 raft_consensus.cc:482] T bc1d7b25d6ba42bc91130927c32d4ae6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:43.124629 10945 tablet_service.cc:1514] Processing DeleteTablet for tablet e9faf70be8ac43c79305980d685a28eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:28 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:43.124629 10956 tablet_service.cc:1514] Processing DeleteTablet for tablet 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:30 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:43.128166 12688 raft_consensus.cc:504] T bc1d7b25d6ba42bc91130927c32d4ae6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:43.124854 10952 tablet_service.cc:1514] Processing DeleteTablet for tablet a58cf1dc81454710a9ad1a8baf155c9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:57:30 UTC) from {username='slave'} at 127.0.0.1:53560
I20260925 09:57:43.130390 12688 leader_election.cc:288] T bc1d7b25d6ba42bc91130927c32d4ae6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:43.109519 11084 raft_consensus.cc:348] T 0a018bea4200474a9904f62aee37240b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:43.131115 10824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc1d7b25d6ba42bc91130927c32d4ae6" candidate_uuid: "be97ab5d75934a648d4ab76341193ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c43461c4ee854ae1ac52b873ccada318" is_pre_election: true
I20260925 09:57:43.131464 11084 raft_consensus.cc:374] T 0a018bea4200474a9904f62aee37240b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:43.131685 11084 raft_consensus.cc:729] T 0a018bea4200474a9904f62aee37240b P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
W20260925 09:57:43.132505 10918 leader_election.cc:341] T bc1d7b25d6ba42bc91130927c32d4ae6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:57:43.132323 11084 consensus_queue.cc:260] T 0a018bea4200474a9904f62aee37240b P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:43.141031 12718 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } } }
I20260925 09:57:43.152998 12718 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 [sys.catalog]: This master's current role is: LEADER
I20260925 09:57:43.156256 10276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu with pid 10735
I20260925 09:57:43.163090 11084 ts_tablet_manager.cc:1428] T 0a018bea4200474a9904f62aee37240b P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.057s user 0.002s sys 0.003s
W20260925 09:57:43.164064 10380 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 71092.1us
I20260925 09:57:43.164999 11084 tablet_bootstrap.cc:492] T 8a1e9536f9af4ed299d1cc5e391f2500 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
W20260925 09:57:43.170078 10919 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.1:33197, user_credentials={real_user=slave}} blocked reactor thread for 77111.6us
I20260925 09:57:43.174927 11727 ts_tablet_manager.cc:1918] T 815b4b1bd914433798dcb5deec6ec79c P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:43.175375 11727 log.cc:1196] T 815b4b1bd914433798dcb5deec6ec79c P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/815b4b1bd914433798dcb5deec6ec79c
I20260925 09:57:43.176009 11727 ts_tablet_manager.cc:1939] T 815b4b1bd914433798dcb5deec6ec79c P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:43.181238 11084 tablet_bootstrap.cc:654] T 8a1e9536f9af4ed299d1cc5e391f2500 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:43.198113 10919 leader_election.cc:334] T 4a6b1284cddc4b2581931d8895212ca6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Network error: Client connection negotiation failed: client connection to 127.10.9.1:33197: connect: Connection refused (error 111)
I20260925 09:57:43.198732 10919 leader_election.cc:302] T 4a6b1284cddc4b2581931d8895212ca6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:43.199482 12688 raft_consensus.cc:2726] T 4a6b1284cddc4b2581931d8895212ca6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:43.200358 10919 leader_election.cc:334] T 5d6ebe0c2ccc48a09b616816953d83fe P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Network error: Client connection negotiation failed: client connection to 127.10.9.1:33197: connect: Connection refused (error 111)
I20260925 09:57:43.200778 10919 leader_election.cc:302] T 5d6ebe0c2ccc48a09b616816953d83fe P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:43.201411 12688 raft_consensus.cc:2726] T 5d6ebe0c2ccc48a09b616816953d83fe P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:43.199167 12721 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8af3f111e8da4d2dad4e4a98ef0fd5d3)
W20260925 09:57:43.202198 10919 leader_election.cc:334] T 30e04d90c962442cadf7bdec6f827dac P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Network error: Client connection negotiation failed: client connection to 127.10.9.1:33197: connect: Connection refused (error 111)
I20260925 09:57:43.203115 10919 leader_election.cc:302] T 30e04d90c962442cadf7bdec6f827dac P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:43.205891 12688 raft_consensus.cc:2726] T 30e04d90c962442cadf7bdec6f827dac P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:43.206944 10919 consensus_peers.cc:487] T e3b6b985a76b459dbd1de42ea20d3856 P be97ab5d75934a648d4ab76341193ec0 -> Peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Couldn't send request to peer b8850818e2ee42a082802d0095c08d21. Status: Network error: Client connection negotiation failed: client connection to 127.10.9.1:33197: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:43.207688 12721 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } }
W20260925 09:57:43.208763 10919 consensus_peers.cc:487] T ef6fefaa76d44fe798201aab2f21b9c7 P be97ab5d75934a648d4ab76341193ec0 -> Peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Couldn't send request to peer b8850818e2ee42a082802d0095c08d21. Status: Network error: Client connection negotiation failed: client connection to 127.10.9.1:33197: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:43.209698 11084 tablet_bootstrap.cc:492] T 8a1e9536f9af4ed299d1cc5e391f2500 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:43.210175 11084 ts_tablet_manager.cc:1397] T 8a1e9536f9af4ed299d1cc5e391f2500 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.045s user 0.006s sys 0.003s
I20260925 09:57:43.211470 10403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "54c8485a7dcc47a1a75d08ef57647ece" candidate_term: 5 candidate_status { last_received { term: 4 index: 493 } } ignore_live_leader: false dest_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" is_pre_election: true
W20260925 09:57:43.209157 10919 leader_election.cc:334] T 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Network error: Client connection negotiation failed: client connection to 127.10.9.1:33197: connect: Connection refused (error 111)
I20260925 09:57:43.212109 10919 leader_election.cc:302] T 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:43.212857 12688 raft_consensus.cc:2726] T 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:43.212954 12315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "54c8485a7dcc47a1a75d08ef57647ece" candidate_term: 5 candidate_status { last_received { term: 4 index: 493 } } ignore_live_leader: false dest_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" is_pre_election: true
W20260925 09:57:43.213730 10919 leader_election.cc:334] T 2d79538d6474433fbda26c91645a0619 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Network error: Client connection negotiation failed: client connection to 127.10.9.1:33197: connect: Connection refused (error 111)
I20260925 09:57:43.214126 10919 leader_election.cc:302] T 2d79538d6474433fbda26c91645a0619 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:43.214702 10304 leader_election.cc:302] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54c8485a7dcc47a1a75d08ef57647ece; no voters: 8af3f111e8da4d2dad4e4a98ef0fd5d3, b8be057e0b6d40a3a4b90f2b1f4a0fa2
I20260925 09:57:43.214905 12688 raft_consensus.cc:2726] T 2d79538d6474433fbda26c91645a0619 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:43.215695 10919 leader_election.cc:334] T ac8230d622214346bc7412bfc6cdb203 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Network error: Client connection negotiation failed: client connection to 127.10.9.1:33197: connect: Connection refused (error 111)
I20260925 09:57:43.217800 10919 leader_election.cc:302] T ac8230d622214346bc7412bfc6cdb203 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:43.227662 12688 raft_consensus.cc:2726] T ac8230d622214346bc7412bfc6cdb203 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:43.227620 10380 catalog_manager.cc:4584] TS b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 05e2c7f9820f41679c4d58343ff3f829: Network error: Client connection negotiation failed: client connection to 127.10.9.1:33197: connect: Connection refused (error 111)
W20260925 09:57:43.228839 10919 leader_election.cc:334] T d5f28d1c19eb4e849dfa6b0bf532b67e P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Network error: Client connection negotiation failed: client connection to 127.10.9.1:33197: connect: Connection refused (error 111)
I20260925 09:57:43.229421 10919 leader_election.cc:302] T d5f28d1c19eb4e849dfa6b0bf532b67e P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:43.229489 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 815b4b1bd914433798dcb5deec6ec79c (table default.Table_bc338e8ca7174647948301512850ce00 [id=7d130f66062344458770b2b943b4485a]) successfully deleted
I20260925 09:57:43.217103 11084 raft_consensus.cc:348] T 8a1e9536f9af4ed299d1cc5e391f2500 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:43.230171 11084 raft_consensus.cc:374] T 8a1e9536f9af4ed299d1cc5e391f2500 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260925 09:57:43.230420 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet 26fae3fbfe374f348a1729c21ec751b9 because the tablet was not found. No further retry: Not found: Tablet not found: 26fae3fbfe374f348a1729c21ec751b9
I20260925 09:57:43.230484 11084 raft_consensus.cc:729] T 8a1e9536f9af4ed299d1cc5e391f2500 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:43.231026 11727 tablet_replica.cc:330] T 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef P be97ab5d75934a648d4ab76341193ec0: stopping tablet replica
W20260925 09:57:43.231053 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet e9faf70be8ac43c79305980d685a28eb because the tablet was not found. No further retry: Not found: Tablet not found: e9faf70be8ac43c79305980d685a28eb
I20260925 09:57:43.231202 12688 raft_consensus.cc:2726] T d5f28d1c19eb4e849dfa6b0bf532b67e P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:43.231328 12726 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260925 09:57:43.231995 11727 raft_consensus.cc:2229] T 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:57:43.231961 12721 leader_election.cc:288] T 00000000000000000000000000000000 P 54c8485a7dcc47a1a75d08ef57647ece [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8af3f111e8da4d2dad4e4a98ef0fd5d3 (127.10.9.61:33047), b8be057e0b6d40a3a4b90f2b1f4a0fa2 (127.10.9.60:36985)
I20260925 09:57:43.232393 11727 raft_consensus.cc:2258] T 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260925 09:57:43.232744 10919 consensus_peers.cc:487] T 8aae639695f44d8c8a61b3ea42ab6ac8 P be97ab5d75934a648d4ab76341193ec0 -> Peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Couldn't send request to peer b8850818e2ee42a082802d0095c08d21. Status: Network error: Client connection negotiation failed: client connection to 127.10.9.1:33197: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260925 09:57:43.233733 10919 consensus_peers.cc:487] T c588b0c05a1a4f3ea545066183e76502 P be97ab5d75934a648d4ab76341193ec0 -> Peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Couldn't send request to peer b8850818e2ee42a082802d0095c08d21. Status: Network error: Client connection negotiation failed: client connection to 127.10.9.1:33197: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:43.234002 11727 ts_tablet_manager.cc:1905] T 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef P be97ab5d75934a648d4ab76341193ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:57:43.232502 11084 consensus_queue.cc:260] T 8a1e9536f9af4ed299d1cc5e391f2500 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:43.236050 11084 ts_tablet_manager.cc:1428] T 8a1e9536f9af4ed299d1cc5e391f2500 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.026s user 0.001s sys 0.004s
I20260925 09:57:43.246671 11084 tablet_bootstrap.cc:492] T 4c31466d0b8d474a8fd19fbf1b2f70cf P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
W20260925 09:57:43.250154 10919 leader_election.cc:334] T 76b1f79efa784662a0adf032c074ce4a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Network error: Client connection negotiation failed: client connection to 127.10.9.1:33197: connect: Connection refused (error 111)
I20260925 09:57:43.250871 10919 leader_election.cc:302] T 76b1f79efa784662a0adf032c074ce4a P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:43.263725 12727 raft_consensus.cc:2726] T 76b1f79efa784662a0adf032c074ce4a P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:43.261631 10919 leader_election.cc:334] T 476567ad4efa4043b6536f3f28d2ee28 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Network error: Client connection negotiation failed: client connection to 127.10.9.1:33197: connect: Connection refused (error 111)
I20260925 09:57:43.267838 10919 leader_election.cc:302] T 476567ad4efa4043b6536f3f28d2ee28 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:43.270362 11084 tablet_bootstrap.cc:654] T 4c31466d0b8d474a8fd19fbf1b2f70cf P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:43.271169 10919 leader_election.cc:334] T bc1d7b25d6ba42bc91130927c32d4ae6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Network error: Client connection negotiation failed: client connection to 127.10.9.1:33197: connect: Connection refused (error 111)
I20260925 09:57:43.274128 10919 leader_election.cc:302] T bc1d7b25d6ba42bc91130927c32d4ae6 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:43.271626 12727 raft_consensus.cc:2726] T 476567ad4efa4043b6536f3f28d2ee28 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:43.276665 12727 raft_consensus.cc:2726] T bc1d7b25d6ba42bc91130927c32d4ae6 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:43.280181 12727 raft_consensus.cc:482] T 451d1984f9b144b68f0681837f8eacf1 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:43.280654 12727 raft_consensus.cc:504] T 451d1984f9b144b68f0681837f8eacf1 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:43.281672 12728 raft_consensus.cc:482] T e329410c044c40a4933550730f46001e P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:43.283468 12728 raft_consensus.cc:504] T e329410c044c40a4933550730f46001e P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:43.285466 12728 leader_election.cc:288] T e329410c044c40a4933550730f46001e P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:43.286090 12727 leader_election.cc:288] T 451d1984f9b144b68f0681837f8eacf1 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
W20260925 09:57:43.298655 10919 leader_election.cc:334] T e329410c044c40a4933550730f46001e P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Network error: Client connection negotiation failed: client connection to 127.10.9.1:33197: connect: Connection refused (error 111)
W20260925 09:57:43.299494 10919 leader_election.cc:334] T 451d1984f9b144b68f0681837f8eacf1 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Network error: Client connection negotiation failed: client connection to 127.10.9.1:33197: connect: Connection refused (error 111)
I20260925 09:57:43.300863 11084 tablet_bootstrap.cc:492] T 4c31466d0b8d474a8fd19fbf1b2f70cf P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:43.301280 11084 ts_tablet_manager.cc:1397] T 4c31466d0b8d474a8fd19fbf1b2f70cf P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.055s user 0.007s sys 0.000s
I20260925 09:57:43.303642 11084 raft_consensus.cc:348] T 4c31466d0b8d474a8fd19fbf1b2f70cf P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:43.304327 11084 raft_consensus.cc:374] T 4c31466d0b8d474a8fd19fbf1b2f70cf P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:43.304584 11084 raft_consensus.cc:729] T 4c31466d0b8d474a8fd19fbf1b2f70cf P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:43.305205 11084 consensus_queue.cc:260] T 4c31466d0b8d474a8fd19fbf1b2f70cf P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:43.315284 11084 ts_tablet_manager.cc:1428] T 4c31466d0b8d474a8fd19fbf1b2f70cf P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.014s user 0.005s sys 0.001s
I20260925 09:57:43.316193 11084 tablet_bootstrap.cc:492] T f46095351ea64812a7fdcfc79cd92c1f P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:43.318478 11727 ts_tablet_manager.cc:1918] T 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef P be97ab5d75934a648d4ab76341193ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:57:43.318862 11727 log.cc:1196] T 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef P be97ab5d75934a648d4ab76341193ec0: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/05a6ea9a6a7449fe8a1a2f7d9ff0c5ef
I20260925 09:57:43.333946 11084 tablet_bootstrap.cc:654] T f46095351ea64812a7fdcfc79cd92c1f P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:43.334362 11727 ts_tablet_manager.cc:1939] T 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef P be97ab5d75934a648d4ab76341193ec0: Deleting consensus metadata
I20260925 09:57:43.336717 10377 catalog_manager.cc:4841] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): tablet 05a6ea9a6a7449fe8a1a2f7d9ff0c5ef (table default.Table_40181c2d583b42b89dbb7e1f589f5317 [id=cda0a13cf8024d98942bd16e2b9ecbe8]) successfully deleted
W20260925 09:57:43.337453 10377 catalog_manager.cc:4808] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553): delete failed for tablet a58cf1dc81454710a9ad1a8baf155c9f because the tablet was not found. No further retry: Not found: Tablet not found: a58cf1dc81454710a9ad1a8baf155c9f
I20260925 09:57:43.381556 11084 tablet_bootstrap.cc:492] T f46095351ea64812a7fdcfc79cd92c1f P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:43.382071 11084 ts_tablet_manager.cc:1397] T f46095351ea64812a7fdcfc79cd92c1f P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.066s user 0.007s sys 0.009s
I20260925 09:57:43.384322 11084 raft_consensus.cc:348] T f46095351ea64812a7fdcfc79cd92c1f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:43.386085 11084 raft_consensus.cc:374] T f46095351ea64812a7fdcfc79cd92c1f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:43.386710 11084 raft_consensus.cc:729] T f46095351ea64812a7fdcfc79cd92c1f P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:43.392606 11084 consensus_queue.cc:260] T f46095351ea64812a7fdcfc79cd92c1f P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:43.394629 11084 ts_tablet_manager.cc:1428] T f46095351ea64812a7fdcfc79cd92c1f P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.012s user 0.004s sys 0.000s
I20260925 09:57:43.395541 11084 tablet_bootstrap.cc:492] T 71fe7382bcb340d58a09f0b9ef255369 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:43.399879 12727 raft_consensus.cc:482] T c9fccada64d1411eb1f75480c8001c49 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:43.400360 12727 raft_consensus.cc:504] T c9fccada64d1411eb1f75480c8001c49 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:43.402204 12727 leader_election.cc:288] T c9fccada64d1411eb1f75480c8001c49 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
I20260925 09:57:43.409215 11084 tablet_bootstrap.cc:654] T 71fe7382bcb340d58a09f0b9ef255369 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:43.409422 10919 leader_election.cc:334] T c9fccada64d1411eb1f75480c8001c49 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Network error: Client connection negotiation failed: client connection to 127.10.9.1:33197: connect: Connection refused (error 111)
I20260925 09:57:43.446399 11084 tablet_bootstrap.cc:492] T 71fe7382bcb340d58a09f0b9ef255369 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:43.446801 11084 ts_tablet_manager.cc:1397] T 71fe7382bcb340d58a09f0b9ef255369 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.051s user 0.010s sys 0.000s
I20260925 09:57:43.449156 11084 raft_consensus.cc:348] T 71fe7382bcb340d58a09f0b9ef255369 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:43.450227 11084 raft_consensus.cc:374] T 71fe7382bcb340d58a09f0b9ef255369 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:43.450536 11084 raft_consensus.cc:729] T 71fe7382bcb340d58a09f0b9ef255369 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:43.451215 11084 consensus_queue.cc:260] T 71fe7382bcb340d58a09f0b9ef255369 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:43.453416 11084 ts_tablet_manager.cc:1428] T 71fe7382bcb340d58a09f0b9ef255369 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.006s user 0.004s sys 0.001s
I20260925 09:57:43.454295 11084 tablet_bootstrap.cc:492] T cdc80b43c43e4d3a9da4fb053d7939e5 P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
W20260925 09:57:43.455356 10378 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet c52745a8cdee461aba7485b0531245d6 (table default.Table_8e186a99ccba425b9b588041d2364a40 [id=2d6bc0e2aaee4b1483b01fc678505bb0], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet c52745a8cdee461aba7485b0531245d6 consensus state has no leader
I20260925 09:57:43.458518 12727 raft_consensus.cc:482] T d822e8ac13d9479aa9ce4f74ace55dd5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:43.458971 12727 raft_consensus.cc:504] T d822e8ac13d9479aa9ce4f74ace55dd5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:43.460770 12727 leader_election.cc:288] T d822e8ac13d9479aa9ce4f74ace55dd5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197), c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257)
W20260925 09:57:43.465294 10919 leader_election.cc:334] T d822e8ac13d9479aa9ce4f74ace55dd5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Network error: Client connection negotiation failed: client connection to 127.10.9.1:33197: connect: Connection refused (error 111)
I20260925 09:57:43.467825 12727 raft_consensus.cc:482] T a3edf29118e048c5a59b4ff2643ed206 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:57:43.468287 12727 raft_consensus.cc:504] T a3edf29118e048c5a59b4ff2643ed206 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:43.470898 12727 leader_election.cc:288] T a3edf29118e048c5a59b4ff2643ed206 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257), b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197)
W20260925 09:57:43.475049 10919 leader_election.cc:334] T a3edf29118e048c5a59b4ff2643ed206 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer b8850818e2ee42a082802d0095c08d21 (127.10.9.1:33197): Network error: Client connection negotiation failed: client connection to 127.10.9.1:33197: connect: Connection refused (error 111)
I20260925 09:57:43.479059 11084 tablet_bootstrap.cc:654] T cdc80b43c43e4d3a9da4fb053d7939e5 P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
I20260925 09:57:43.480093 10387 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet bc1d7b25d6ba42bc91130927c32d4ae6 (table default.Table_bac584c420284db7964cb9262e8a309d [id=0f92f8e3fdcc49cc906a25117f2fb4da]). Expected version 1 got 0
I20260925 09:57:43.482159 10387 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet f0bf545f70ae4cd6a1ad2540c7158ecc (table default.Table_25edfe533c64421da7700ac5d60265c3 [id=1469f44b239d454caa97056bd5803255]). Expected version 1 got 0
I20260925 09:57:43.497788 10387 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet c52745a8cdee461aba7485b0531245d6 (table default.Table_8e186a99ccba425b9b588041d2364a40 [id=2d6bc0e2aaee4b1483b01fc678505bb0]). Expected version 2 got 0
I20260925 09:57:43.501114 10387 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 572c83b4a7c84fdab1fdbb0fbdb0a53d (table default.Table_8398f7e9c7a64f688bd18521e99cb669 [id=b8b7198fdaa24d72ae13b7275685c555]). Expected version 1 got 0
I20260925 09:57:43.502725 10387 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet 25ac35fb43d34c9b85b8e61ad6e386a6 (table default.Table_2672598ccbce42eb8f11019f47a6fdda [id=887ce3cafef64210bb69d424ed5dae8a]). Expected version 2 got 0
I20260925 09:57:43.505872 10387 catalog_manager.cc:5587] TS be97ab5d75934a648d4ab76341193ec0 (127.10.9.3:44553) does not have the latest schema for tablet cc295f7a9169475f9a9235e657be629f (table default.Table_168840855e0746f7ab027c1900842751 [id=99ba81c4d0dd497398e048a05759419a]). Expected version 1 got 0
I20260925 09:57:43.512586 11084 tablet_bootstrap.cc:492] T cdc80b43c43e4d3a9da4fb053d7939e5 P be97ab5d75934a648d4ab76341193ec0: No bootstrap required, opened a new log
I20260925 09:57:43.513028 11084 ts_tablet_manager.cc:1397] T cdc80b43c43e4d3a9da4fb053d7939e5 P be97ab5d75934a648d4ab76341193ec0: Time spent bootstrapping tablet: real 0.059s user 0.008s sys 0.005s
I20260925 09:57:43.515440 11084 raft_consensus.cc:348] T cdc80b43c43e4d3a9da4fb053d7939e5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:43.519680 11084 raft_consensus.cc:374] T cdc80b43c43e4d3a9da4fb053d7939e5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:57:43.520018 11084 raft_consensus.cc:729] T cdc80b43c43e4d3a9da4fb053d7939e5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be97ab5d75934a648d4ab76341193ec0, State: Initialized, Role: FOLLOWER
I20260925 09:57:43.520706 11084 consensus_queue.cc:260] T cdc80b43c43e4d3a9da4fb053d7939e5 P be97ab5d75934a648d4ab76341193ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8850818e2ee42a082802d0095c08d21" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 33197 } } peers { permanent_uuid: "c43461c4ee854ae1ac52b873ccada318" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 43257 } } peers { permanent_uuid: "be97ab5d75934a648d4ab76341193ec0" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 44553 } }
I20260925 09:57:43.530303 11084 ts_tablet_manager.cc:1428] T cdc80b43c43e4d3a9da4fb053d7939e5 P be97ab5d75934a648d4ab76341193ec0: Time spent starting tablet: real 0.017s user 0.003s sys 0.002s
I20260925 09:57:43.531272 11084 tablet_bootstrap.cc:492] T 038c9962a12a48089300b655c56f368d P be97ab5d75934a648d4ab76341193ec0: Bootstrap starting.
I20260925 09:57:43.536509 11084 tablet_bootstrap.cc:654] T 038c9962a12a48089300b655c56f368d P be97ab5d75934a648d4ab76341193ec0: Neither blocks nor log segments found. Creating new log.
W20260925 09:57:43.563611 10918 connection.cc:535] client connection to 127.10.9.2:43257 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 09:57:43.569444 12722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:57:43.565610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.9.2:43257 (local address 127.10.9.3:35167)
0925 09:57:43.566951 (+ 1341us) negotiation.cc:107] Waiting for socket to connect
0925 09:57:43.566991 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 09:57:43.567236 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:57:43.568705 (+ 1469us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.9.2:43257: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":947}
W20260925 09:57:43.570150 10918 leader_election.cc:334] T 451d1984f9b144b68f0681837f8eacf1 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Network error: Client connection negotiation failed: client connection to 127.10.9.2:43257: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 09:57:43.570576 10918 leader_election.cc:302] T 451d1984f9b144b68f0681837f8eacf1 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:43.571221 12727 raft_consensus.cc:2726] T 451d1984f9b144b68f0681837f8eacf1 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:43.572858 10918 leader_election.cc:334] T e329410c044c40a4933550730f46001e P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Network error: Client connection negotiation failed: client connection to 127.10.9.2:43257: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 09:57:43.573259 10918 leader_election.cc:302] T e329410c044c40a4933550730f46001e P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:43.573909 12727 raft_consensus.cc:2726] T e329410c044c40a4933550730f46001e P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:43.574681 10918 consensus_peers.cc:487] T c588b0c05a1a4f3ea545066183e76502 P be97ab5d75934a648d4ab76341193ec0 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Status: Network error: Client connection negotiation failed: client connection to 127.10.9.2:43257: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
W20260925 09:57:43.575157 10918 consensus_peers.cc:487] T e3b6b985a76b459dbd1de42ea20d3856 P be97ab5d75934a648d4ab76341193ec0 -> Peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Couldn't send request to peer c43461c4ee854ae1ac52b873ccada318. Status: Network error: Client connection negotiation failed: client connection to 127.10.9.2:43257: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
W20260925 09:57:43.575552 10918 leader_election.cc:334] T c9fccada64d1411eb1f75480c8001c49 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Network error: Client connection negotiation failed: client connection to 127.10.9.2:43257: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 09:57:43.575904 10918 leader_election.cc:302] T c9fccada64d1411eb1f75480c8001c49 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:43.576530 12727 raft_consensus.cc:2726] T c9fccada64d1411eb1f75480c8001c49 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:43.577277 10918 leader_election.cc:334] T d822e8ac13d9479aa9ce4f74ace55dd5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Network error: Client connection negotiation failed: client connection to 127.10.9.2:43257: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 09:57:43.577637 10918 leader_election.cc:302] T d822e8ac13d9479aa9ce4f74ace55dd5 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:43.578240 12727 raft_consensus.cc:2726] T d822e8ac13d9479aa9ce4f74ace55dd5 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:57:43.578938 10918 leader_election.cc:334] T a3edf29118e048c5a59b4ff2643ed206 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer c43461c4ee854ae1ac52b873ccada318 (127.10.9.2:43257): Network error: Client connection negotiation failed: client connection to 127.10.9.2:43257: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 09:57:43.579284 10918 leader_election.cc:302] T a3edf29118e048c5a59b4ff2643ed206 P be97ab5d75934a648d4ab76341193ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be97ab5d75934a648d4ab76341193ec0; no voters: b8850818e2ee42a082802d0095c08d21, c43461c4ee854ae1ac52b873ccada318
I20260925 09:57:43.579953 12727 raft_consensus.cc:2726] T a3edf29118e048c5a59b4ff2643ed206 P be97ab5d75934a648d4ab76341193ec0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:57:43.597131 10276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu with pid 10895
I20260925 09:57:43.675933 12736 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8af3f111e8da4d2dad4e4a98ef0fd5d3)
I20260925 09:57:43.677537 12736 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54c8485a7dcc47a1a75d08ef57647ece" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 34455 } } peers { permanent_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 33047 } } peers { permanent_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 36985 } }
I20260925 09:57:43.692646 12736 leader_election.cc:288] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 54c8485a7dcc47a1a75d08ef57647ece (127.10.9.62:34455), 8af3f111e8da4d2dad4e4a98ef0fd5d3 (127.10.9.61:33047)
I20260925 09:57:43.693470 10327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" candidate_term: 5 candidate_status { last_received { term: 4 index: 493 } } ignore_live_leader: false dest_uuid: "54c8485a7dcc47a1a75d08ef57647ece" is_pre_election: true
I20260925 09:57:43.694590 10403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8be057e0b6d40a3a4b90f2b1f4a0fa2" candidate_term: 5 candidate_status { last_received { term: 4 index: 493 } } ignore_live_leader: false dest_uuid: "8af3f111e8da4d2dad4e4a98ef0fd5d3" is_pre_election: true
I20260925 09:57:43.696703 12199 leader_election.cc:302] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8be057e0b6d40a3a4b90f2b1f4a0fa2; no voters: 54c8485a7dcc47a1a75d08ef57647ece, 8af3f111e8da4d2dad4e4a98ef0fd5d3
I20260925 09:57:43.697432 12736 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b8be057e0b6d40a3a4b90f2b1f4a0fa2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260925 09:57:43.803604 12250 connection.cc:535] server connection from 127.10.9.3:33067 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 09:57:43.804440 10304 connection.cc:535] server connection from 127.10.9.3:37239 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 09:57:43.811611 10377 connection.cc:535] client connection to 127.10.9.3:44553 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 09:57:43.813460 10276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu with pid 10285
W20260925 09:57:43.826808 10379 connection.cc:535] server connection from 127.10.9.3:41469 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 09:57:43.920356 10379 connection.cc:535] client connection to 127.10.9.62:34455 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 09:57:43.931509 10379 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8af3f111e8da4d2dad4e4a98ef0fd5d3 -> Peer 54c8485a7dcc47a1a75d08ef57647ece (127.10.9.62:34455): Couldn't send request to peer 54c8485a7dcc47a1a75d08ef57647ece. Status: Network error: Client connection negotiation failed: client connection to 127.10.9.62:34455: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 09:57:43.939370 10276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu with pid 10355
I20260925 09:57:44.022024 10276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu with pid 11435
2026-09-25T09:57:44Z chronyd exiting
I20260925 09:57:44.083712 10276 master-stress-test.cc:586] Tables created: 211
I20260925 09:57:44.083909 10276 master-stress-test.cc:587] Tables altered: 71
I20260925 09:57:44.084017 10276 master-stress-test.cc:588] Tables deleted: 74
I20260925 09:57:44.084128 10276 master-stress-test.cc:589] Tablets replaced: 29
I20260925 09:57:44.084233 10276 master-stress-test.cc:590] Masters restarted: 1
[ OK ] CatalogManagerConfigurations/MasterStressTest.Test/0 (53072 ms)
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/3
I20260925 09:57:44.192876 10276 test_util.cc:268] Using random seed: -1083340971
2026-09-25T09:57:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T09:57:44Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskdR3P3Z/thirdparty/src/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskdR3P3Z/thirdparty/src/hadoop-3.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/security.properties
2026-09-25 09:57:51: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskdR3P3Z/thirdparty/src/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskdR3P3Z/thirdparty/src/hadoop-3.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
I20260925 09:57:58.477824 10276 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu
/tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.9.62:43437
--webserver_interface=127.10.9.62
--webserver_port=0
--builtin_ntp_servers=127.10.9.20:35467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.9.62:43437,127.10.9.61:40425,127.10.9.60:42083
--hive_metastore_uris=thrift://127.0.0.1:42723
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260925 09:57:58.798508 12974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:57:58.799117 12974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:57:58.799638 12974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:57:58.831101 12974 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260925 09:57:58.831445 12974 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260925 09:57:58.831732 12974 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:57:58.832006 12974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:57:58.832258 12974 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:57:58.832502 12974 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:57:58.868316 12974 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.9.20:35467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:42723
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.10.9.62:43437,127.10.9.61:40425,127.10.9.60:42083
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.9.62:43437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.9.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:06:08 UTC on ccycloud.quasar-lntfnh.root.comops.site
build id 4672078
TSAN enabled
I20260925 09:57:58.869612 12974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:57:58.871194 12974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:57:58.890645 12981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:57:58.890657 12980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:57:58.891101 12983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:57:58.893404 12974 server_base.cc:1034] running on GCE node
I20260925 09:58:00.103261 12974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:58:00.105971 12974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:58:00.107317 12974 hybrid_clock.cc:648] HybridClock initialized: now 1790330280107292 us; error 46 us; skew 500 ppm
I20260925 09:58:00.108179 12974 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:58:00.114440 12974 webserver.cc:460] Webserver started at http://127.10.9.62:38583/ using document root <none> and password file <none>
I20260925 09:58:00.115341 12974 fs_manager.cc:359] Metadata directory not provided
I20260925 09:58:00.115640 12974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:58:00.116094 12974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:58:00.122079 12974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-0/data/instance:
uuid: "70718e3f6f9f489089c0d8354380ed6c"
format_stamp: "Formatted at 2026-09-25 09:58:00 on dist-test-slave-4xvl"
I20260925 09:58:00.123188 12974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-0/wal/instance:
uuid: "70718e3f6f9f489089c0d8354380ed6c"
format_stamp: "Formatted at 2026-09-25 09:58:00 on dist-test-slave-4xvl"
I20260925 09:58:00.130903 12974 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260925 09:58:00.136484 12990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:58:00.137462 12974 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260925 09:58:00.137785 12974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-0/data,/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-0/wal
uuid: "70718e3f6f9f489089c0d8354380ed6c"
format_stamp: "Formatted at 2026-09-25 09:58:00 on dist-test-slave-4xvl"
I20260925 09:58:00.138146 12974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:58:00.194412 12974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:58:00.196141 12974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:58:00.196635 12974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:58:00.268021 12974 rpc_server.cc:307] RPC server started. Bound to: 127.10.9.62:43437
I20260925 09:58:00.268134 13041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.9.62:43437 every 8 connection(s)
I20260925 09:58:00.270715 12974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-0/data/info.pb
I20260925 09:58:00.275601 13042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:00.277261 10276 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu as pid 12974
I20260925 09:58:00.277657 10276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-0/wal/instance
I20260925 09:58:00.280146 10276 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu
/tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.9.61:40425
--webserver_interface=127.10.9.61
--webserver_port=0
--builtin_ntp_servers=127.10.9.20:35467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.9.62:43437,127.10.9.61:40425,127.10.9.60:42083
--hive_metastore_uris=thrift://127.0.0.1:42723
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260925 09:58:00.287215 13042 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } has no permanent_uuid. Determining permanent_uuid...
I20260925 09:58:00.314049 13042 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } has no permanent_uuid. Determining permanent_uuid...
W20260925 09:58:00.316113 12993 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.9.61:40425: connect: Connection refused (error 111)
W20260925 09:58:00.320468 13042 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.61:40425: Network error: Client connection negotiation failed: client connection to 127.10.9.61:40425: connect: Connection refused (error 111)
I20260925 09:58:00.370029 13042 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } attempt: 1
W20260925 09:58:00.373878 13042 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.61:40425: Network error: Client connection negotiation failed: client connection to 127.10.9.61:40425: connect: Connection refused (error 111)
I20260925 09:58:00.442373 13042 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } attempt: 2
W20260925 09:58:00.446091 13042 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.61:40425: Network error: Client connection negotiation failed: client connection to 127.10.9.61:40425: connect: Connection refused (error 111)
I20260925 09:58:00.537580 13042 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } attempt: 3
W20260925 09:58:00.541410 13042 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.61:40425: Network error: Client connection negotiation failed: client connection to 127.10.9.61:40425: connect: Connection refused (error 111)
W20260925 09:58:00.590716 13043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:58:00.591470 13043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:58:00.591948 13043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:58:00.622327 13043 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260925 09:58:00.622558 13043 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260925 09:58:00.622869 13043 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:58:00.623123 13043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:58:00.623381 13043 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:58:00.623651 13043 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:58:00.658442 13043 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.9.20:35467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:42723
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.10.9.62:43437,127.10.9.61:40425,127.10.9.60:42083
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.9.61:40425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.9.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:06:08 UTC on ccycloud.quasar-lntfnh.root.comops.site
build id 4672078
TSAN enabled
I20260925 09:58:00.659737 13043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:58:00.661302 13043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:58:00.679878 13055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:58:00.684813 13042 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } attempt: 4
W20260925 09:58:00.688771 13042 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.61:40425: Network error: Client connection negotiation failed: client connection to 127.10.9.61:40425: connect: Connection refused (error 111)
I20260925 09:58:00.988183 13042 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } attempt: 5
W20260925 09:58:00.992393 13042 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.61:40425: Network error: Client connection negotiation failed: client connection to 127.10.9.61:40425: connect: Connection refused (error 111)
I20260925 09:58:01.539760 13042 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } attempt: 6
W20260925 09:58:01.554308 13042 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.61:40425: Network error: Client connection negotiation failed: client connection to 127.10.9.61:40425: connect: Connection refused (error 111)
W20260925 09:58:00.679890 13053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:58:00.679903 13052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:58:00.681278 13043 server_base.cc:1034] running on GCE node
I20260925 09:58:01.875264 13043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:58:01.878259 13043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:58:01.879673 13043 hybrid_clock.cc:648] HybridClock initialized: now 1790330281879650 us; error 58 us; skew 500 ppm
I20260925 09:58:01.880549 13043 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:58:01.894191 13043 webserver.cc:460] Webserver started at http://127.10.9.61:44995/ using document root <none> and password file <none>
I20260925 09:58:01.895200 13043 fs_manager.cc:359] Metadata directory not provided
I20260925 09:58:01.895483 13043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:58:01.895982 13043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:58:01.901018 13043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-1/data/instance:
uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0"
format_stamp: "Formatted at 2026-09-25 09:58:01 on dist-test-slave-4xvl"
I20260925 09:58:01.902220 13043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-1/wal/instance:
uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0"
format_stamp: "Formatted at 2026-09-25 09:58:01 on dist-test-slave-4xvl"
I20260925 09:58:01.909339 13043 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.003s
I20260925 09:58:01.914922 13065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:58:01.915959 13043 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:58:01.916299 13043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-1/data,/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-1/wal
uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0"
format_stamp: "Formatted at 2026-09-25 09:58:01 on dist-test-slave-4xvl"
I20260925 09:58:01.916707 13043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:58:01.963201 13043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:58:01.964725 13043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:58:01.965191 13043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:58:02.036593 13043 rpc_server.cc:307] RPC server started. Bound to: 127.10.9.61:40425
I20260925 09:58:02.036716 13116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.9.61:40425 every 8 connection(s)
I20260925 09:58:02.039453 13043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-1/data/info.pb
I20260925 09:58:02.044685 13117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:02.050009 10276 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu as pid 13043
I20260925 09:58:02.050616 10276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-1/wal/instance
I20260925 09:58:02.053521 10276 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu
/tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.9.60:42083
--webserver_interface=127.10.9.60
--webserver_port=0
--builtin_ntp_servers=127.10.9.20:35467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.9.62:43437,127.10.9.61:40425,127.10.9.60:42083
--hive_metastore_uris=thrift://127.0.0.1:42723
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260925 09:58:02.057237 13117 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } has no permanent_uuid. Determining permanent_uuid...
I20260925 09:58:02.085543 13117 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } has no permanent_uuid. Determining permanent_uuid...
I20260925 09:58:02.101665 13117 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } has no permanent_uuid. Determining permanent_uuid...
W20260925 09:58:02.103915 13067 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.9.60:42083: connect: Connection refused (error 111)
W20260925 09:58:02.108384 13117 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.60:42083: Network error: Client connection negotiation failed: client connection to 127.10.9.60:42083: connect: Connection refused (error 111)
I20260925 09:58:02.157840 13117 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } attempt: 1
W20260925 09:58:02.162087 13117 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.60:42083: Network error: Client connection negotiation failed: client connection to 127.10.9.60:42083: connect: Connection refused (error 111)
I20260925 09:58:02.230568 13117 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } attempt: 2
W20260925 09:58:02.234736 13117 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.60:42083: Network error: Client connection negotiation failed: client connection to 127.10.9.60:42083: connect: Connection refused (error 111)
I20260925 09:58:02.326237 13117 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } attempt: 3
W20260925 09:58:02.329921 13117 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.60:42083: Network error: Client connection negotiation failed: client connection to 127.10.9.60:42083: connect: Connection refused (error 111)
W20260925 09:58:02.399363 13118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:58:02.399986 13118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:58:02.400461 13118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:58:02.431136 13118 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260925 09:58:02.431483 13118 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260925 09:58:02.431746 13118 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:58:02.432001 13118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:58:02.432258 13118 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:58:02.432510 13118 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:58:02.467280 13118 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.9.20:35467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:42723
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.10.9.62:43437,127.10.9.61:40425,127.10.9.60:42083
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.9.60:42083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.9.60
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:06:08 UTC on ccycloud.quasar-lntfnh.root.comops.site
build id 4672078
TSAN enabled
I20260925 09:58:02.468626 13118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:58:02.470265 13118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 09:58:02.473258 13117 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } attempt: 4
W20260925 09:58:02.476776 13117 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.60:42083: Network error: Client connection negotiation failed: client connection to 127.10.9.60:42083: connect: Connection refused (error 111)
W20260925 09:58:02.488683 13129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:58:02.614959 13042 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } attempt: 7
I20260925 09:58:02.628603 13042 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } has no permanent_uuid. Determining permanent_uuid...
W20260925 09:58:02.634675 13042 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.60:42083: Network error: Client connection negotiation failed: client connection to 127.10.9.60:42083: connect: Connection refused (error 111)
I20260925 09:58:02.693135 13042 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } attempt: 1
W20260925 09:58:02.697530 13042 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.60:42083: Network error: Client connection negotiation failed: client connection to 127.10.9.60:42083: connect: Connection refused (error 111)
I20260925 09:58:02.776125 13117 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } attempt: 5
I20260925 09:58:02.778915 13042 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } attempt: 2
W20260925 09:58:02.780682 13117 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.60:42083: Network error: Client connection negotiation failed: client connection to 127.10.9.60:42083: connect: Connection refused (error 111)
W20260925 09:58:02.783430 13042 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.60:42083: Network error: Client connection negotiation failed: client connection to 127.10.9.60:42083: connect: Connection refused (error 111)
I20260925 09:58:02.868927 13042 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } attempt: 3
W20260925 09:58:02.873680 13042 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.60:42083: Network error: Client connection negotiation failed: client connection to 127.10.9.60:42083: connect: Connection refused (error 111)
I20260925 09:58:03.014134 13042 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } attempt: 4
W20260925 09:58:03.018750 13042 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.60:42083: Network error: Client connection negotiation failed: client connection to 127.10.9.60:42083: connect: Connection refused (error 111)
I20260925 09:58:03.302153 13042 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } attempt: 5
W20260925 09:58:03.306741 13042 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.60:42083: Network error: Client connection negotiation failed: client connection to 127.10.9.60:42083: connect: Connection refused (error 111)
I20260925 09:58:03.328087 13117 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } attempt: 6
W20260925 09:58:03.334841 13117 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.9.60:42083: Network error: Client connection negotiation failed: client connection to 127.10.9.60:42083: connect: Connection refused (error 111)
W20260925 09:58:02.488695 13131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:58:02.488991 13128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:58:02.490372 13118 server_base.cc:1034] running on GCE node
I20260925 09:58:03.682993 13118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:58:03.685933 13118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:58:03.687305 13118 hybrid_clock.cc:648] HybridClock initialized: now 1790330283687268 us; error 46 us; skew 500 ppm
I20260925 09:58:03.688336 13118 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:58:03.695207 13118 webserver.cc:460] Webserver started at http://127.10.9.60:36655/ using document root <none> and password file <none>
I20260925 09:58:03.696240 13118 fs_manager.cc:359] Metadata directory not provided
I20260925 09:58:03.696427 13118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:58:03.696894 13118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:58:03.701712 13118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-2/data/instance:
uuid: "61e29afab0cf46819cbffc91248bf761"
format_stamp: "Formatted at 2026-09-25 09:58:03 on dist-test-slave-4xvl"
I20260925 09:58:03.702786 13118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-2/wal/instance:
uuid: "61e29afab0cf46819cbffc91248bf761"
format_stamp: "Formatted at 2026-09-25 09:58:03 on dist-test-slave-4xvl"
I20260925 09:58:03.709949 13118 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.003s
I20260925 09:58:03.715274 13143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:58:03.716305 13118 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 09:58:03.716581 13118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-2/data,/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-2/wal
uuid: "61e29afab0cf46819cbffc91248bf761"
format_stamp: "Formatted at 2026-09-25 09:58:03 on dist-test-slave-4xvl"
I20260925 09:58:03.716876 13118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:58:03.766024 13118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:58:03.767589 13118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:58:03.768024 13118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:58:03.840214 13118 rpc_server.cc:307] RPC server started. Bound to: 127.10.9.60:42083
I20260925 09:58:03.840305 13194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.9.60:42083 every 8 connection(s)
I20260925 09:58:03.842999 13118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-2/data/info.pb
I20260925 09:58:03.845979 10276 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu as pid 13118
I20260925 09:58:03.846427 10276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-2/wal/instance
I20260925 09:58:03.848057 13195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:03.859165 13042 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } attempt: 6
I20260925 09:58:03.864789 13195 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } has no permanent_uuid. Determining permanent_uuid...
I20260925 09:58:03.896744 13195 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } has no permanent_uuid. Determining permanent_uuid...
I20260925 09:58:03.907112 13042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c: Bootstrap starting.
I20260925 09:58:03.910048 13195 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } has no permanent_uuid. Determining permanent_uuid...
I20260925 09:58:03.914153 13042 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:03.916320 13042 log.cc:824] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c: Log is configured to *not* fsync() on all Append() calls
I20260925 09:58:03.921753 13042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c: No bootstrap required, opened a new log
I20260925 09:58:03.929332 13195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761: Bootstrap starting.
I20260925 09:58:03.937592 13195 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:03.940133 13195 log.cc:824] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761: Log is configured to *not* fsync() on all Append() calls
I20260925 09:58:03.945364 13195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761: No bootstrap required, opened a new log
I20260925 09:58:03.947855 13042 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:03.948853 13042 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:03.949204 13042 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70718e3f6f9f489089c0d8354380ed6c, State: Initialized, Role: FOLLOWER
I20260925 09:58:03.950134 13042 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:03.952867 13206 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } } }
I20260925 09:58:03.953706 13206 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:58:03.954614 13042 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [sys.catalog]: configured and running, proceeding with master startup.
W20260925 09:58:03.955479 12996 tablet.cc:2369] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:58:03.970129 13195 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:03.971156 13195 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:03.971623 13195 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61e29afab0cf46819cbffc91248bf761, State: Initialized, Role: FOLLOWER
I20260925 09:58:03.972802 13195 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:03.976295 13215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } } }
I20260925 09:58:03.977240 13215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:58:03.978107 13195 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [sys.catalog]: configured and running, proceeding with master startup.
W20260925 09:58:04.003042 13225 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.003412 13225 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 09:58:04.017756 13225 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.018093 13225 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 09:58:04.023494 13149 tablet.cc:2369] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 09:58:04.032195 13225 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.032500 13225 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 09:58:04.046671 13225 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.047000 13225 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 09:58:03.999491 12974 master_runner.cc:186] Error getting master registration for 127.10.9.61:40425: OK, instance_id { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" instance_seqno: 1790330282016557 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 09:58:04.051218 12974 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260925 09:58:04.061086 13225 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.061740 13225 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 09:58:04.075950 13225 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.076329 13225 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 09:58:04.091470 13225 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.091886 13225 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 09:58:04.100060 13238 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.100472 13238 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 09:58:04.108309 13225 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.108692 13225 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 09:58:04.123131 13225 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.131414 13225 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 09:58:04.133052 13238 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.133415 13238 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 09:58:04.092514 13118 master_runner.cc:186] Error getting master registration for 127.10.9.61:40425: OK, instance_id { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" instance_seqno: 1790330282016557 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 09:58:04.145201 13118 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260925 09:58:04.145999 13225 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.146374 13225 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 09:58:04.149170 13238 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.149596 13238 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 09:58:04.160624 13225 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.161031 13225 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 09:58:04.165303 13238 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.165716 13238 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 09:58:04.175282 13225 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.175917 13225 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 09:58:04.179960 13238 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.180310 13238 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 09:58:04.190184 13225 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.191414 13225 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 09:58:04.194403 13238 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.194703 13238 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 09:58:04.205641 13225 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.208901 13238 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.209209 13238 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 09:58:04.206063 13225 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 09:58:04.223446 13238 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.224433 13238 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 09:58:04.228513 13225 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.228823 13225 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 09:58:04.238765 13238 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.239217 13238 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 09:58:04.244673 13225 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.245043 13225 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 09:58:04.253482 13238 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.253971 13238 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 09:58:04.259480 13225 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.259855 13225 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 09:58:04.276593 13238 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.276997 13238 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 09:58:04.281344 13225 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.281741 13225 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 09:58:04.292263 13238 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.293853 13238 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 09:58:04.305581 13225 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.309870 13238 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.310319 13225 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 09:58:04.310731 13238 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 09:58:04.324697 13225 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.325249 13225 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 09:58:04.343928 13238 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.344349 13238 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 09:58:04.344846 13225 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.345193 13225 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 09:58:04.358623 13238 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.362936 13225 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260925 09:58:04.369697 13206 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:04.372306 13206 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:04.374785 13206 leader_election.cc:288] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7d0c1c2f1ed4e7a9ed54e73869d53b0 (127.10.9.61:40425), 61e29afab0cf46819cbffc91248bf761 (127.10.9.60:42083)
W20260925 09:58:04.377197 13225 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 09:58:04.377290 13238 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 09:58:04.377951 13170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70718e3f6f9f489089c0d8354380ed6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61e29afab0cf46819cbffc91248bf761" is_pre_election: true
I20260925 09:58:04.378974 13170 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70718e3f6f9f489089c0d8354380ed6c in term 0.
I20260925 09:58:04.380538 12992 leader_election.cc:302] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61e29afab0cf46819cbffc91248bf761, 70718e3f6f9f489089c0d8354380ed6c; no voters:
I20260925 09:58:04.381480 13206 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:04.381812 13206 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:04.382140 13206 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:04.388957 13206 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
W20260925 09:58:04.392629 13238 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.392921 13238 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 09:58:04.396588 13170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70718e3f6f9f489089c0d8354380ed6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61e29afab0cf46819cbffc91248bf761"
I20260925 09:58:04.397190 13170 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:04.399091 13117 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } attempt: 7
I20260925 09:58:04.407519 13206 leader_election.cc:288] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [CANDIDATE]: Term 1 election: Requested vote from peers b7d0c1c2f1ed4e7a9ed54e73869d53b0 (127.10.9.61:40425), 61e29afab0cf46819cbffc91248bf761 (127.10.9.60:42083)
I20260925 09:58:04.407438 13170 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70718e3f6f9f489089c0d8354380ed6c in term 1.
I20260925 09:58:04.414202 12992 leader_election.cc:302] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61e29afab0cf46819cbffc91248bf761, 70718e3f6f9f489089c0d8354380ed6c; no voters:
I20260925 09:58:04.415198 13206 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:04.427692 13206 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 1 LEADER]: Becoming Leader. State: Replica: 70718e3f6f9f489089c0d8354380ed6c, State: Running, Role: LEADER
I20260925 09:58:04.428750 13206 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:04.441599 13241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70718e3f6f9f489089c0d8354380ed6c. Latest consensus state: current_term: 1 leader_uuid: "70718e3f6f9f489089c0d8354380ed6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } } }
I20260925 09:58:04.442502 13241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [sys.catalog]: This master's current role is: LEADER
W20260925 09:58:04.449168 13238 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.449486 13238 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260925 09:58:04.451087 13225 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.451442 13225 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 09:58:04.455593 13243 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:58:04.462704 13243 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:58:04.486356 13170 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 1 FOLLOWER]: Refusing update from remote peer 70718e3f6f9f489089c0d8354380ed6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260925 09:58:04.481048 13238 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260925 09:58:04.490855 13241 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 09:58:04.490671 13238 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 09:58:04.510655 13117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Bootstrap starting.
I20260925 09:58:04.535175 13245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70718e3f6f9f489089c0d8354380ed6c. Latest consensus state: current_term: 1 leader_uuid: "70718e3f6f9f489089c0d8354380ed6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } } }
I20260925 09:58:04.536020 13245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 09:58:04.538569 13238 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 09:58:04.539687 13238 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 09:58:04.540146 13244 mvcc.cc:205] Tried to move back new op lower bound from 7333192845236236288 to 7333192845031018496. Current Snapshot: MvccSnapshot[applied={T|T < 7333192845236236288}]
I20260925 09:58:04.544312 13117 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:04.547176 13206 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "70718e3f6f9f489089c0d8354380ed6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } } }
I20260925 09:58:04.547964 13206 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [sys.catalog]: This master's current role is: LEADER
I20260925 09:58:04.549196 13246 mvcc.cc:205] Tried to move back new op lower bound from 7333192845236236288 to 7333192845031018496. Current Snapshot: MvccSnapshot[applied={T|T < 7333192845236236288}]
I20260925 09:58:04.551767 13117 log.cc:824] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Log is configured to *not* fsync() on all Append() calls
I20260925 09:58:04.555652 13245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "70718e3f6f9f489089c0d8354380ed6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } } }
I20260925 09:58:04.557178 13243 catalog_manager.cc:1347] Generated new cluster ID: 85565ad29e6546888a40444865b055de
I20260925 09:58:04.557461 13243 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:58:04.556342 13245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:58:04.565917 13117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: No bootstrap required, opened a new log
I20260925 09:58:04.597304 13117 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:04.598171 13117 raft_consensus.cc:374] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:04.598462 13117 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7d0c1c2f1ed4e7a9ed54e73869d53b0, State: Initialized, Role: FOLLOWER
I20260925 09:58:04.599376 13117 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:04.606118 13238 catalog_manager.cc:1259] Loaded cluster ID: 85565ad29e6546888a40444865b055de
I20260925 09:58:04.607741 13238 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761: loading cluster ID for follower catalog manager: success
I20260925 09:58:04.613314 13117 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:58:04.613497 13092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70718e3f6f9f489089c0d8354380ed6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" is_pre_election: true
I20260925 09:58:04.614419 13092 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70718e3f6f9f489089c0d8354380ed6c in term 0.
I20260925 09:58:04.615676 13090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:04.607596 13251 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } } }
I20260925 09:58:04.616935 13251 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:58:04.622670 13090 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 1 FOLLOWER]: Refusing update from remote peer 70718e3f6f9f489089c0d8354380ed6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:58:04.623620 13091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70718e3f6f9f489089c0d8354380ed6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0"
I20260925 09:58:04.628384 13091 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70718e3f6f9f489089c0d8354380ed6c in term 1.
I20260925 09:58:04.629079 13243 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 09:58:04.631052 13243 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:58:04.632246 13206 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 09:58:04.640148 13238 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260925 09:58:04.643482 13238 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260925 09:58:04.673695 13238 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761: acquiring CA information for follower catalog manager: success
W20260925 09:58:04.678043 13238 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260925 09:58:04.680992 13243 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c: Generated new TSK 0
I20260925 09:58:04.682202 13243 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:58:04.686622 13243 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260925 09:58:04.699919 13243 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260925 09:58:04.709352 13238 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 09:58:04.712221 13241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "70718e3f6f9f489089c0d8354380ed6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } } }
I20260925 09:58:04.712980 13241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [sys.catalog]: This master's current role is: LEADER
I20260925 09:58:04.717970 13251 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70718e3f6f9f489089c0d8354380ed6c. Latest consensus state: current_term: 1 leader_uuid: "70718e3f6f9f489089c0d8354380ed6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } } }
I20260925 09:58:04.718050 13262 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "70718e3f6f9f489089c0d8354380ed6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } } }
I20260925 09:58:04.718964 13262 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:58:04.718914 13251 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:58:04.747772 10276 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu
/tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.9.1:0
--local_ip_for_outbound_sockets=127.10.9.1
--webserver_interface=127.10.9.1
--webserver_port=0
--tserver_master_addrs=127.10.9.62:43437,127.10.9.61:40425,127.10.9.60:42083
--builtin_ntp_servers=127.10.9.20:35467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260925 09:58:04.770578 13272 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 09:58:04.807328 13272 catalog_manager.cc:1259] Loaded cluster ID: 85565ad29e6546888a40444865b055de
I20260925 09:58:04.807603 13272 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: loading cluster ID for follower catalog manager: success
I20260925 09:58:04.815208 13272 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: acquiring CA information for follower catalog manager: success
I20260925 09:58:04.820797 13272 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260925 09:58:05.530660 13271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:58:05.531440 13271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:58:05.532440 13271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:58:05.593825 13271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:58:05.595700 13271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.9.1
I20260925 09:58:05.628163 13278 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 70718e3f6f9f489089c0d8354380ed6c)
I20260925 09:58:05.630138 13278 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:05.639176 13017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70718e3f6f9f489089c0d8354380ed6c" is_pre_election: true
I20260925 09:58:05.639317 13170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "61e29afab0cf46819cbffc91248bf761" is_pre_election: true
I20260925 09:58:05.641217 13278 leader_election.cc:288] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 70718e3f6f9f489089c0d8354380ed6c (127.10.9.62:43437), 61e29afab0cf46819cbffc91248bf761 (127.10.9.60:42083)
I20260925 09:58:05.641351 13069 leader_election.cc:302] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b7d0c1c2f1ed4e7a9ed54e73869d53b0; no voters: 61e29afab0cf46819cbffc91248bf761, 70718e3f6f9f489089c0d8354380ed6c
I20260925 09:58:05.645115 13278 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 09:58:05.681959 13271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.9.20:35467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.9.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.9.1
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.10.9.62:43437,127.10.9.61:40425,127.10.9.60:42083
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.9.1
--log_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:06:08 UTC on ccycloud.quasar-lntfnh.root.comops.site
build id 4672078
TSAN enabled
I20260925 09:58:05.684257 13271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:58:05.686637 13271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:58:05.762151 13281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:58:06.489614 13284 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 70718e3f6f9f489089c0d8354380ed6c)
I20260925 09:58:06.490242 13284 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:06.494127 13017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "61e29afab0cf46819cbffc91248bf761" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70718e3f6f9f489089c0d8354380ed6c" is_pre_election: true
I20260925 09:58:06.511736 13284 leader_election.cc:288] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 70718e3f6f9f489089c0d8354380ed6c (127.10.9.62:43437), b7d0c1c2f1ed4e7a9ed54e73869d53b0 (127.10.9.61:40425)
I20260925 09:58:06.511891 13091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "61e29afab0cf46819cbffc91248bf761" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" is_pre_election: true
I20260925 09:58:06.513569 13146 leader_election.cc:302] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61e29afab0cf46819cbffc91248bf761; no voters: 70718e3f6f9f489089c0d8354380ed6c, b7d0c1c2f1ed4e7a9ed54e73869d53b0
I20260925 09:58:06.514364 13284 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 09:58:06.781565 13285 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 70718e3f6f9f489089c0d8354380ed6c)
I20260925 09:58:06.782099 13285 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:06.784919 13017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70718e3f6f9f489089c0d8354380ed6c" is_pre_election: true
I20260925 09:58:06.785913 13285 leader_election.cc:288] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 70718e3f6f9f489089c0d8354380ed6c (127.10.9.62:43437), 61e29afab0cf46819cbffc91248bf761 (127.10.9.60:42083)
I20260925 09:58:06.786262 13170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "61e29afab0cf46819cbffc91248bf761" is_pre_election: true
I20260925 09:58:06.789503 13067 leader_election.cc:302] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b7d0c1c2f1ed4e7a9ed54e73869d53b0; no voters: 61e29afab0cf46819cbffc91248bf761, 70718e3f6f9f489089c0d8354380ed6c
I20260925 09:58:06.790447 13285 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 09:58:07.345175 13286 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 70718e3f6f9f489089c0d8354380ed6c)
I20260925 09:58:07.347281 13286 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:07.353353 13017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70718e3f6f9f489089c0d8354380ed6c" is_pre_election: true
I20260925 09:58:07.356355 13170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "61e29afab0cf46819cbffc91248bf761" is_pre_election: true
I20260925 09:58:07.355995 13286 leader_election.cc:288] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 70718e3f6f9f489089c0d8354380ed6c (127.10.9.62:43437), 61e29afab0cf46819cbffc91248bf761 (127.10.9.60:42083)
I20260925 09:58:07.357858 13067 leader_election.cc:302] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b7d0c1c2f1ed4e7a9ed54e73869d53b0; no voters: 61e29afab0cf46819cbffc91248bf761, 70718e3f6f9f489089c0d8354380ed6c
I20260925 09:58:07.358879 13286 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 09:58:07.912616 13287 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 70718e3f6f9f489089c0d8354380ed6c)
I20260925 09:58:07.918051 13287 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:07.921823 13017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "61e29afab0cf46819cbffc91248bf761" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70718e3f6f9f489089c0d8354380ed6c" is_pre_election: true
I20260925 09:58:07.923894 13287 leader_election.cc:288] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 70718e3f6f9f489089c0d8354380ed6c (127.10.9.62:43437), b7d0c1c2f1ed4e7a9ed54e73869d53b0 (127.10.9.61:40425)
I20260925 09:58:07.925333 13091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "61e29afab0cf46819cbffc91248bf761" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" is_pre_election: true
I20260925 09:58:07.928287 13146 leader_election.cc:302] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61e29afab0cf46819cbffc91248bf761; no voters: 70718e3f6f9f489089c0d8354380ed6c, b7d0c1c2f1ed4e7a9ed54e73869d53b0
I20260925 09:58:07.929246 13287 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260925 09:58:07.135560 13279 debug-util.cc:398] Leaking SignalData structure 0x7b08000148c0 after lost signal to thread 13271
W20260925 09:58:08.140844 13271 thread.cc:636] GCE (cloud detector) Time spent creating pthread: real 2.381s user 0.841s sys 1.231s
W20260925 09:58:08.147812 13271 thread.cc:602] GCE (cloud detector) Time spent starting thread: real 2.388s user 0.841s sys 1.232s
W20260925 09:58:05.769097 13280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:58:08.153112 13289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:58:08.158129 13271 server_base.cc:1034] running on GCE node
I20260925 09:58:08.159549 13271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:58:08.162256 13271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:58:08.163721 13271 hybrid_clock.cc:648] HybridClock initialized: now 1790330288163663 us; error 61 us; skew 500 ppm
I20260925 09:58:08.164808 13271 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:58:08.180120 13271 webserver.cc:460] Webserver started at http://127.10.9.1:44799/ using document root <none> and password file <none>
I20260925 09:58:08.181385 13271 fs_manager.cc:359] Metadata directory not provided
I20260925 09:58:08.182420 13271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:58:08.182998 13271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:58:08.191080 13271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/data/instance:
uuid: "b3082cf15dc74c0480e5d33ecf592843"
format_stamp: "Formatted at 2026-09-25 09:58:08 on dist-test-slave-4xvl"
I20260925 09:58:08.192878 13271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/instance:
uuid: "b3082cf15dc74c0480e5d33ecf592843"
format_stamp: "Formatted at 2026-09-25 09:58:08 on dist-test-slave-4xvl"
I20260925 09:58:08.202955 13271 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.008s sys 0.000s
I20260925 09:58:08.210644 13294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:58:08.212075 13271 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.007s sys 0.000s
I20260925 09:58:08.212451 13271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/data,/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal
uuid: "b3082cf15dc74c0480e5d33ecf592843"
format_stamp: "Formatted at 2026-09-25 09:58:08 on dist-test-slave-4xvl"
I20260925 09:58:08.212878 13271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:58:08.290584 13299 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 70718e3f6f9f489089c0d8354380ed6c)
I20260925 09:58:08.291256 13299 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:08.294273 13017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70718e3f6f9f489089c0d8354380ed6c" is_pre_election: true
I20260925 09:58:08.296100 13299 leader_election.cc:288] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 70718e3f6f9f489089c0d8354380ed6c (127.10.9.62:43437), 61e29afab0cf46819cbffc91248bf761 (127.10.9.60:42083)
I20260925 09:58:08.297031 13170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "61e29afab0cf46819cbffc91248bf761" is_pre_election: true
I20260925 09:58:08.298373 13067 leader_election.cc:302] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b7d0c1c2f1ed4e7a9ed54e73869d53b0; no voters: 61e29afab0cf46819cbffc91248bf761, 70718e3f6f9f489089c0d8354380ed6c
I20260925 09:58:08.299209 13299 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 09:58:08.303334 13271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:58:08.305267 13271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:58:08.305800 13271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:58:08.309183 13271 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:58:08.314908 13271 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:58:08.315187 13271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:58:08.315514 13271 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:58:08.315812 13271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:58:08.461220 13396 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 70718e3f6f9f489089c0d8354380ed6c)
I20260925 09:58:08.461876 13396 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:08.466207 13017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "61e29afab0cf46819cbffc91248bf761" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70718e3f6f9f489089c0d8354380ed6c" is_pre_election: true
I20260925 09:58:08.467612 13091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "61e29afab0cf46819cbffc91248bf761" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" is_pre_election: true
I20260925 09:58:08.467836 13396 leader_election.cc:288] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 70718e3f6f9f489089c0d8354380ed6c (127.10.9.62:43437), b7d0c1c2f1ed4e7a9ed54e73869d53b0 (127.10.9.61:40425)
I20260925 09:58:08.469125 13146 leader_election.cc:302] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61e29afab0cf46819cbffc91248bf761; no voters: 70718e3f6f9f489089c0d8354380ed6c, b7d0c1c2f1ed4e7a9ed54e73869d53b0
I20260925 09:58:08.470312 13396 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 09:58:08.480499 13271 rpc_server.cc:307] RPC server started. Bound to: 127.10.9.1:42161
I20260925 09:58:08.480575 13408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.9.1:42161 every 8 connection(s)
I20260925 09:58:08.484122 13271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/data/info.pb
I20260925 09:58:08.492614 10276 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu as pid 13271
I20260925 09:58:08.493243 10276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/instance
I20260925 09:58:08.527917 10276 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu
/tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.9.2:0
--local_ip_for_outbound_sockets=127.10.9.2
--webserver_interface=127.10.9.2
--webserver_port=0
--tserver_master_addrs=127.10.9.62:43437,127.10.9.61:40425,127.10.9.60:42083
--builtin_ntp_servers=127.10.9.20:35467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260925 09:58:08.553508 13410 heartbeater.cc:347] Connected to a master server at 127.10.9.61:40425
I20260925 09:58:08.554036 13410 heartbeater.cc:464] Registering TS with master...
I20260925 09:58:08.555315 13410 heartbeater.cc:511] Master 127.10.9.61:40425 requested a full tablet report, sending...
I20260925 09:58:08.556923 13411 heartbeater.cc:347] Connected to a master server at 127.10.9.62:43437
I20260925 09:58:08.557255 13411 heartbeater.cc:464] Registering TS with master...
I20260925 09:58:08.558044 13411 heartbeater.cc:511] Master 127.10.9.62:43437 requested a full tablet report, sending...
I20260925 09:58:08.559168 13082 ts_manager.cc:194] Registered new tserver with Master: b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161)
I20260925 09:58:08.560570 13006 ts_manager.cc:194] Registered new tserver with Master: b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161)
I20260925 09:58:08.562610 13409 heartbeater.cc:347] Connected to a master server at 127.10.9.60:42083
I20260925 09:58:08.562987 13409 heartbeater.cc:464] Registering TS with master...
I20260925 09:58:08.563485 13006 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.9.1:40535
I20260925 09:58:08.564296 13409 heartbeater.cc:511] Master 127.10.9.60:42083 requested a full tablet report, sending...
I20260925 09:58:08.567657 13160 ts_manager.cc:194] Registered new tserver with Master: b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161)
W20260925 09:58:08.884599 13419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:58:08.885140 13419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:58:08.885751 13419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:58:08.918963 13419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:58:08.919968 13419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.9.2
I20260925 09:58:08.956641 13419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.9.20:35467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.9.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.9.2
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.10.9.62:43437,127.10.9.61:40425,127.10.9.60:42083
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.9.2
--log_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:06:08 UTC on ccycloud.quasar-lntfnh.root.comops.site
build id 4672078
TSAN enabled
I20260925 09:58:08.958243 13419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:58:08.960263 13419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:58:08.980677 13427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:58:09.068394 13411 heartbeater.cc:503] Master 127.10.9.62:43437 was elected leader, sending a full tablet report...
I20260925 09:58:09.998088 13432 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 70718e3f6f9f489089c0d8354380ed6c)
I20260925 09:58:09.998476 13432 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:10.000191 13432 leader_election.cc:288] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 70718e3f6f9f489089c0d8354380ed6c (127.10.9.62:43437), b7d0c1c2f1ed4e7a9ed54e73869d53b0 (127.10.9.61:40425)
I20260925 09:58:10.001266 13091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "61e29afab0cf46819cbffc91248bf761" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" is_pre_election: true
I20260925 09:58:10.001214 13017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "61e29afab0cf46819cbffc91248bf761" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70718e3f6f9f489089c0d8354380ed6c" is_pre_election: true
I20260925 09:58:10.002934 13147 leader_election.cc:302] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61e29afab0cf46819cbffc91248bf761; no voters: 70718e3f6f9f489089c0d8354380ed6c, b7d0c1c2f1ed4e7a9ed54e73869d53b0
I20260925 09:58:10.003652 13432 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 09:58:08.983204 13419 server_base.cc:1034] running on GCE node
W20260925 09:58:08.981179 13426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:58:08.981308 13429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:58:10.219678 13419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:58:10.222465 13419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:58:10.223914 13419 hybrid_clock.cc:648] HybridClock initialized: now 1790330290223847 us; error 77 us; skew 500 ppm
I20260925 09:58:10.224713 13419 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:58:10.231029 13419 webserver.cc:460] Webserver started at http://127.10.9.2:43329/ using document root <none> and password file <none>
I20260925 09:58:10.232052 13419 fs_manager.cc:359] Metadata directory not provided
I20260925 09:58:10.232283 13419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:58:10.232745 13419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:58:10.238508 13419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/data/instance:
uuid: "c4d724d4b83e4d1e817a3a505dc24eb9"
format_stamp: "Formatted at 2026-09-25 09:58:10 on dist-test-slave-4xvl"
I20260925 09:58:10.239951 13419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/instance:
uuid: "c4d724d4b83e4d1e817a3a505dc24eb9"
format_stamp: "Formatted at 2026-09-25 09:58:10 on dist-test-slave-4xvl"
I20260925 09:58:10.247159 13419 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260925 09:58:10.254199 13437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:58:10.255368 13419 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260925 09:58:10.255762 13419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/data,/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal
uuid: "c4d724d4b83e4d1e817a3a505dc24eb9"
format_stamp: "Formatted at 2026-09-25 09:58:10 on dist-test-slave-4xvl"
I20260925 09:58:10.256135 13419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:58:10.308862 13419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:58:10.310271 13419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:58:10.310667 13419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:58:10.313522 13419 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:58:10.317945 13419 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:58:10.318130 13419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:58:10.318430 13419 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:58:10.318635 13419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:58:10.468207 13419 rpc_server.cc:307] RPC server started. Bound to: 127.10.9.2:44327
I20260925 09:58:10.468374 13549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.9.2:44327 every 8 connection(s)
I20260925 09:58:10.470818 13419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/data/info.pb
I20260925 09:58:10.480829 10276 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu as pid 13419
I20260925 09:58:10.481374 10276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/instance
I20260925 09:58:10.505668 10276 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu
/tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.9.3:0
--local_ip_for_outbound_sockets=127.10.9.3
--webserver_interface=127.10.9.3
--webserver_port=0
--tserver_master_addrs=127.10.9.62:43437,127.10.9.61:40425,127.10.9.60:42083
--builtin_ntp_servers=127.10.9.20:35467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260925 09:58:10.529203 13560 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 70718e3f6f9f489089c0d8354380ed6c)
I20260925 09:58:10.529728 13560 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:10.531858 13560 leader_election.cc:288] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 70718e3f6f9f489089c0d8354380ed6c (127.10.9.62:43437), 61e29afab0cf46819cbffc91248bf761 (127.10.9.60:42083)
I20260925 09:58:10.533277 13017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70718e3f6f9f489089c0d8354380ed6c" is_pre_election: true
I20260925 09:58:10.533572 13170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "61e29afab0cf46819cbffc91248bf761" is_pre_election: true
I20260925 09:58:10.535199 13069 leader_election.cc:302] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b7d0c1c2f1ed4e7a9ed54e73869d53b0; no voters: 61e29afab0cf46819cbffc91248bf761, 70718e3f6f9f489089c0d8354380ed6c
I20260925 09:58:10.536321 13560 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 09:58:10.543861 13551 heartbeater.cc:347] Connected to a master server at 127.10.9.61:40425
I20260925 09:58:10.544346 13551 heartbeater.cc:464] Registering TS with master...
I20260925 09:58:10.545682 13551 heartbeater.cc:511] Master 127.10.9.61:40425 requested a full tablet report, sending...
I20260925 09:58:10.548805 13082 ts_manager.cc:194] Registered new tserver with Master: c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
I20260925 09:58:10.549110 13550 heartbeater.cc:347] Connected to a master server at 127.10.9.60:42083
I20260925 09:58:10.549420 13550 heartbeater.cc:464] Registering TS with master...
I20260925 09:58:10.550251 13550 heartbeater.cc:511] Master 127.10.9.60:42083 requested a full tablet report, sending...
I20260925 09:58:10.552526 13160 ts_manager.cc:194] Registered new tserver with Master: c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
I20260925 09:58:10.554687 13552 heartbeater.cc:347] Connected to a master server at 127.10.9.62:43437
I20260925 09:58:10.555001 13552 heartbeater.cc:464] Registering TS with master...
I20260925 09:58:10.555775 13552 heartbeater.cc:511] Master 127.10.9.62:43437 requested a full tablet report, sending...
I20260925 09:58:10.557662 13006 ts_manager.cc:194] Registered new tserver with Master: c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
I20260925 09:58:10.559127 13006 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.9.2:35073
I20260925 09:58:10.574443 13562 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 70718e3f6f9f489089c0d8354380ed6c)
I20260925 09:58:10.575024 13562 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:10.578351 13017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "61e29afab0cf46819cbffc91248bf761" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70718e3f6f9f489089c0d8354380ed6c" is_pre_election: true
I20260925 09:58:10.579526 13091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "61e29afab0cf46819cbffc91248bf761" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" is_pre_election: true
I20260925 09:58:10.580032 13562 leader_election.cc:288] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 70718e3f6f9f489089c0d8354380ed6c (127.10.9.62:43437), b7d0c1c2f1ed4e7a9ed54e73869d53b0 (127.10.9.61:40425)
I20260925 09:58:10.580091 13091 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61e29afab0cf46819cbffc91248bf761 in term 1.
I20260925 09:58:10.581195 13146 leader_election.cc:302] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61e29afab0cf46819cbffc91248bf761, b7d0c1c2f1ed4e7a9ed54e73869d53b0; no voters: 70718e3f6f9f489089c0d8354380ed6c
I20260925 09:58:10.586300 13562 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 09:58:10.586694 13562 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 70718e3f6f9f489089c0d8354380ed6c)
I20260925 09:58:10.587098 13562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:58:10.595724 13562 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:10.597189 13170 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 2 FOLLOWER]: Rejecting Update request from peer 70718e3f6f9f489089c0d8354380ed6c for earlier term 1. Current term is 2. Ops: []
I20260925 09:58:10.599764 13562 leader_election.cc:288] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [CANDIDATE]: Term 2 election: Requested vote from peers 70718e3f6f9f489089c0d8354380ed6c (127.10.9.62:43437), b7d0c1c2f1ed4e7a9ed54e73869d53b0 (127.10.9.61:40425)
I20260925 09:58:10.600788 13564 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260925 09:58:10.601466 13017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "61e29afab0cf46819cbffc91248bf761" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "70718e3f6f9f489089c0d8354380ed6c"
I20260925 09:58:10.602330 13565 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 1 LEADER]: Stepping down as leader of term 1
I20260925 09:58:10.602721 13565 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 70718e3f6f9f489089c0d8354380ed6c, State: Running, Role: LEADER
I20260925 09:58:10.603551 13565 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:10.604887 13565 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:58:10.608402 13091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "61e29afab0cf46819cbffc91248bf761" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0"
I20260925 09:58:10.609229 13091 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 1 FOLLOWER]: Advancing to term 2
I20260925 09:58:10.617233 13225 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 09:58:10.617905 13091 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61e29afab0cf46819cbffc91248bf761 in term 2.
I20260925 09:58:10.618722 13090 raft_consensus.cc:1226] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 2 FOLLOWER]: Rejecting Update request from peer 70718e3f6f9f489089c0d8354380ed6c for earlier term 1. Current term is 2. Ops: []
I20260925 09:58:10.619825 13146 leader_election.cc:302] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61e29afab0cf46819cbffc91248bf761, b7d0c1c2f1ed4e7a9ed54e73869d53b0; no voters: 70718e3f6f9f489089c0d8354380ed6c
I20260925 09:58:10.620481 13562 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 09:58:10.622426 13562 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 2 LEADER]: Becoming Leader. State: Replica: 61e29afab0cf46819cbffc91248bf761, State: Running, Role: LEADER
I20260925 09:58:10.623369 13562 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:10.627817 13566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61e29afab0cf46819cbffc91248bf761. Latest consensus state: current_term: 2 leader_uuid: "61e29afab0cf46819cbffc91248bf761" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } } }
I20260925 09:58:10.628597 13566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [sys.catalog]: This master's current role is: LEADER
I20260925 09:58:10.630695 13568 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:58:10.636175 13568 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:58:10.639438 13568 catalog_manager.cc:1259] Loaded cluster ID: 85565ad29e6546888a40444865b055de
I20260925 09:58:10.639669 13568 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:58:10.642961 13568 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:58:10.645699 13568 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761: Loaded TSK: 0
I20260925 09:58:10.646751 13568 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:58:10.649333 13568 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260925 09:58:10.652220 13568 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
W20260925 09:58:10.893018 13561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 09:58:10.893565 13561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:58:10.894176 13561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 09:58:10.927597 13561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:58:10.928534 13561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.9.3
I20260925 09:58:10.965022 13561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.9.20:35467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.9.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.9.3
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.10.9.62:43437,127.10.9.61:40425,127.10.9.60:42083
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.9.3
--log_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:06:08 UTC on ccycloud.quasar-lntfnh.root.comops.site
build id 4672078
TSAN enabled
I20260925 09:58:10.966259 13561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:58:10.967933 13561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 09:58:10.987890 13577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:58:11.062721 13552 heartbeater.cc:503] Master 127.10.9.62:43437 was elected leader, sending a full tablet report...
I20260925 09:58:11.082154 13017 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 2 FOLLOWER]: Refusing update from remote peer 61e29afab0cf46819cbffc91248bf761: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260925 09:58:11.083851 13566 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260925 09:58:11.094573 13090 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 2 FOLLOWER]: Refusing update from remote peer 61e29afab0cf46819cbffc91248bf761: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260925 09:58:11.096163 13565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61e29afab0cf46819cbffc91248bf761. Latest consensus state: current_term: 2 leader_uuid: "61e29afab0cf46819cbffc91248bf761" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } } }
I20260925 09:58:11.096881 13565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:58:11.096937 13566 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260925 09:58:11.112427 13566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "61e29afab0cf46819cbffc91248bf761" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } } }
I20260925 09:58:11.113423 13566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [sys.catalog]: This master's current role is: LEADER
I20260925 09:58:11.115247 13582 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61e29afab0cf46819cbffc91248bf761. Latest consensus state: current_term: 2 leader_uuid: "61e29afab0cf46819cbffc91248bf761" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } } }
I20260925 09:58:11.116322 13582 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:58:11.119645 13565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "61e29afab0cf46819cbffc91248bf761" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } } }
I20260925 09:58:11.120358 13565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:58:11.122371 13562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "61e29afab0cf46819cbffc91248bf761" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } } }
I20260925 09:58:11.123173 13562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [sys.catalog]: This master's current role is: LEADER
I20260925 09:58:11.126045 13582 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "61e29afab0cf46819cbffc91248bf761" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } } }
I20260925 09:58:11.126920 13582 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:58:11.558787 13550 heartbeater.cc:503] Master 127.10.9.60:42083 was elected leader, sending a full tablet report...
I20260925 09:58:11.598506 13409 heartbeater.cc:503] Master 127.10.9.60:42083 was elected leader, sending a full tablet report...
I20260925 09:58:11.604101 13565 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 61e29afab0cf46819cbffc91248bf761)
I20260925 09:58:11.604538 13565 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:11.606287 13565 leader_election.cc:288] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b7d0c1c2f1ed4e7a9ed54e73869d53b0 (127.10.9.61:40425), 61e29afab0cf46819cbffc91248bf761 (127.10.9.60:42083)
I20260925 09:58:11.607677 13170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70718e3f6f9f489089c0d8354380ed6c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "61e29afab0cf46819cbffc91248bf761" is_pre_election: true
I20260925 09:58:11.607537 13090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70718e3f6f9f489089c0d8354380ed6c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" is_pre_election: true
I20260925 09:58:11.609376 12993 leader_election.cc:302] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70718e3f6f9f489089c0d8354380ed6c; no voters: 61e29afab0cf46819cbffc91248bf761, b7d0c1c2f1ed4e7a9ed54e73869d53b0
I20260925 09:58:11.610042 13565 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260925 09:58:11.623318 13582 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 61e29afab0cf46819cbffc91248bf761)
I20260925 09:58:11.623764 13582 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:11.625238 13582 leader_election.cc:288] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 70718e3f6f9f489089c0d8354380ed6c (127.10.9.62:43437), 61e29afab0cf46819cbffc91248bf761 (127.10.9.60:42083)
I20260925 09:58:11.626261 13170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "61e29afab0cf46819cbffc91248bf761" is_pre_election: true
I20260925 09:58:11.626364 13017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70718e3f6f9f489089c0d8354380ed6c" is_pre_election: true
I20260925 09:58:11.626914 13017 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7d0c1c2f1ed4e7a9ed54e73869d53b0 in term 2.
I20260925 09:58:11.628191 13069 leader_election.cc:302] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 70718e3f6f9f489089c0d8354380ed6c, b7d0c1c2f1ed4e7a9ed54e73869d53b0; no voters: 61e29afab0cf46819cbffc91248bf761
I20260925 09:58:11.629190 13582 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260925 09:58:11.629487 13582 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 61e29afab0cf46819cbffc91248bf761)
I20260925 09:58:11.629804 13582 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 2 FOLLOWER]: Advancing to term 3
I20260925 09:58:11.635938 13582 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:11.638020 13582 leader_election.cc:288] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [CANDIDATE]: Term 3 election: Requested vote from peers 70718e3f6f9f489089c0d8354380ed6c (127.10.9.62:43437), 61e29afab0cf46819cbffc91248bf761 (127.10.9.60:42083)
I20260925 09:58:11.638682 13170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "61e29afab0cf46819cbffc91248bf761"
I20260925 09:58:11.640062 13017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70718e3f6f9f489089c0d8354380ed6c"
I20260925 09:58:11.641716 13090 raft_consensus.cc:1226] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 3 FOLLOWER]: Rejecting Update request from peer 61e29afab0cf46819cbffc91248bf761 for earlier term 2. Current term is 3. Ops: []
I20260925 09:58:11.641686 13069 leader_election.cc:302] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b7d0c1c2f1ed4e7a9ed54e73869d53b0; no voters: 61e29afab0cf46819cbffc91248bf761, 70718e3f6f9f489089c0d8354380ed6c
I20260925 09:58:11.642977 13585 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260925 09:58:11.643474 13582 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260925 09:58:11.644398 13588 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 2 LEADER]: Stepping down as leader of term 2
I20260925 09:58:11.644629 13588 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 61e29afab0cf46819cbffc91248bf761, State: Running, Role: LEADER
I20260925 09:58:11.645330 13588 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:11.646267 13588 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 2 FOLLOWER]: Advancing to term 3
I20260925 09:58:12.133033 13590 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 61e29afab0cf46819cbffc91248bf761)
I20260925 09:58:12.133430 13590 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:12.135779 13590 leader_election.cc:288] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b7d0c1c2f1ed4e7a9ed54e73869d53b0 (127.10.9.61:40425), 61e29afab0cf46819cbffc91248bf761 (127.10.9.60:42083)
I20260925 09:58:12.136204 13090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70718e3f6f9f489089c0d8354380ed6c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" is_pre_election: true
I20260925 09:58:12.136466 13170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70718e3f6f9f489089c0d8354380ed6c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "61e29afab0cf46819cbffc91248bf761" is_pre_election: true
I20260925 09:58:12.137042 13090 raft_consensus.cc:2379] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70718e3f6f9f489089c0d8354380ed6c in current term 3: Already voted for candidate b7d0c1c2f1ed4e7a9ed54e73869d53b0 in this term.
I20260925 09:58:12.137146 13170 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70718e3f6f9f489089c0d8354380ed6c in term 3.
I20260925 09:58:12.138437 12992 leader_election.cc:302] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61e29afab0cf46819cbffc91248bf761, 70718e3f6f9f489089c0d8354380ed6c; no voters: b7d0c1c2f1ed4e7a9ed54e73869d53b0
I20260925 09:58:12.139183 13590 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260925 09:58:12.139674 13590 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 61e29afab0cf46819cbffc91248bf761)
I20260925 09:58:12.140045 13590 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 2 FOLLOWER]: Advancing to term 3
I20260925 09:58:12.145731 13590 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:12.147702 13590 leader_election.cc:288] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [CANDIDATE]: Term 3 election: Requested vote from peers b7d0c1c2f1ed4e7a9ed54e73869d53b0 (127.10.9.61:40425), 61e29afab0cf46819cbffc91248bf761 (127.10.9.60:42083)
I20260925 09:58:12.148490 13090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70718e3f6f9f489089c0d8354380ed6c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0"
I20260925 09:58:12.149252 13090 raft_consensus.cc:2379] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 70718e3f6f9f489089c0d8354380ed6c in current term 3: Already voted for candidate b7d0c1c2f1ed4e7a9ed54e73869d53b0 in this term.
I20260925 09:58:12.149979 13170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70718e3f6f9f489089c0d8354380ed6c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "61e29afab0cf46819cbffc91248bf761"
I20260925 09:58:12.156802 13170 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70718e3f6f9f489089c0d8354380ed6c in term 3.
I20260925 09:58:12.157738 12992 leader_election.cc:302] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61e29afab0cf46819cbffc91248bf761, 70718e3f6f9f489089c0d8354380ed6c; no voters: b7d0c1c2f1ed4e7a9ed54e73869d53b0
I20260925 09:58:12.158452 13590 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 3 FOLLOWER]: Leader election won for term 3
I20260925 09:58:12.159780 13590 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 3 LEADER]: Becoming Leader. State: Replica: 70718e3f6f9f489089c0d8354380ed6c, State: Running, Role: LEADER
I20260925 09:58:12.160399 13590 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:12.163645 13591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70718e3f6f9f489089c0d8354380ed6c. Latest consensus state: current_term: 3 leader_uuid: "70718e3f6f9f489089c0d8354380ed6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } } }
I20260925 09:58:12.164373 13591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [sys.catalog]: This master's current role is: LEADER
I20260925 09:58:12.165539 13593 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:58:12.172963 13593 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:58:12.176959 13593 catalog_manager.cc:1259] Loaded cluster ID: 85565ad29e6546888a40444865b055de
I20260925 09:58:12.177209 13593 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:58:12.182569 13593 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:58:12.185575 13593 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c: Loaded TSK: 0
I20260925 09:58:12.186890 13593 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:58:12.189657 13593 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260925 09:58:12.192134 13593 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260925 09:58:12.234352 13594 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:12.234948 13594 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:12.237329 13594 leader_election.cc:288] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 70718e3f6f9f489089c0d8354380ed6c (127.10.9.62:43437), 61e29afab0cf46819cbffc91248bf761 (127.10.9.60:42083)
I20260925 09:58:12.238219 13017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70718e3f6f9f489089c0d8354380ed6c" is_pre_election: true
I20260925 09:58:12.238516 13170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "61e29afab0cf46819cbffc91248bf761" is_pre_election: true
I20260925 09:58:12.239229 13170 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7d0c1c2f1ed4e7a9ed54e73869d53b0 in term 3.
I20260925 09:58:12.240520 13067 leader_election.cc:302] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61e29afab0cf46819cbffc91248bf761, b7d0c1c2f1ed4e7a9ed54e73869d53b0; no voters: 70718e3f6f9f489089c0d8354380ed6c
I20260925 09:58:12.241654 13594 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260925 09:58:12.242003 13594 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:12.242404 13594 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 3 FOLLOWER]: Advancing to term 4
I20260925 09:58:12.249920 13594 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:12.252986 13594 leader_election.cc:288] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [CANDIDATE]: Term 4 election: Requested vote from peers 70718e3f6f9f489089c0d8354380ed6c (127.10.9.62:43437), 61e29afab0cf46819cbffc91248bf761 (127.10.9.60:42083)
I20260925 09:58:12.252908 13017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "70718e3f6f9f489089c0d8354380ed6c"
I20260925 09:58:12.253684 13170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "61e29afab0cf46819cbffc91248bf761"
I20260925 09:58:12.254349 13170 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 3 FOLLOWER]: Advancing to term 4
W20260925 09:58:10.987950 13575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:58:10.990607 13561 server_base.cc:1034] running on GCE node
W20260925 09:58:10.988335 13574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:58:12.261116 13561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:58:12.261253 13170 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7d0c1c2f1ed4e7a9ed54e73869d53b0 in term 4.
I20260925 09:58:12.263247 13067 leader_election.cc:302] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61e29afab0cf46819cbffc91248bf761, b7d0c1c2f1ed4e7a9ed54e73869d53b0; no voters: 70718e3f6f9f489089c0d8354380ed6c
I20260925 09:58:12.264119 13594 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 4 FOLLOWER]: Leader election won for term 4
I20260925 09:58:12.265646 13594 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 4 LEADER]: Becoming Leader. State: Replica: b7d0c1c2f1ed4e7a9ed54e73869d53b0, State: Running, Role: LEADER
I20260925 09:58:12.267117 13594 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:12.272328 13561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:58:12.273862 13561 hybrid_clock.cc:648] HybridClock initialized: now 1790330292273801 us; error 50 us; skew 500 ppm
I20260925 09:58:12.274725 13561 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:58:12.276173 13596 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7d0c1c2f1ed4e7a9ed54e73869d53b0. Latest consensus state: current_term: 4 leader_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } } }
I20260925 09:58:12.276870 13596 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [sys.catalog]: This master's current role is: LEADER
I20260925 09:58:12.278713 13598 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:58:12.283267 13561 webserver.cc:460] Webserver started at http://127.10.9.3:36861/ using document root <none> and password file <none>
I20260925 09:58:12.284554 13561 fs_manager.cc:359] Metadata directory not provided
I20260925 09:58:12.284848 13561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:58:12.285450 13561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 09:58:12.287604 13598 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:58:12.291669 13598 catalog_manager.cc:1259] Loaded cluster ID: 85565ad29e6546888a40444865b055de
I20260925 09:58:12.291963 13598 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:58:12.292873 13561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/data/instance:
uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
format_stamp: "Formatted at 2026-09-25 09:58:12 on dist-test-slave-4xvl"
I20260925 09:58:12.294394 13561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/instance:
uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
format_stamp: "Formatted at 2026-09-25 09:58:12 on dist-test-slave-4xvl"
I20260925 09:58:12.297461 13598 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:58:12.301529 13598 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Loaded TSK: 0
I20260925 09:58:12.303160 13598 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:58:12.304064 13561 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.010s sys 0.001s
I20260925 09:58:12.306165 13598 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260925 09:58:12.308703 13598 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260925 09:58:12.309731 13602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:58:12.310839 13561 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.004s
I20260925 09:58:12.311175 13561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/data,/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal
uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
format_stamp: "Formatted at 2026-09-25 09:58:12 on dist-test-slave-4xvl"
I20260925 09:58:12.311583 13561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:58:12.357066 13561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:58:12.358536 13561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:58:12.359021 13561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:58:12.361636 13561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 09:58:12.365788 13561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 09:58:12.366051 13561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 09:58:12.366319 13561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 09:58:12.366454 13561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 09:58:12.527601 13561 rpc_server.cc:307] RPC server started. Bound to: 127.10.9.3:40731
I20260925 09:58:12.527701 13715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.9.3:40731 every 8 connection(s)
I20260925 09:58:12.530304 13561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/data/info.pb
I20260925 09:58:12.535562 10276 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu as pid 13561
I20260925 09:58:12.536536 10276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/instance
I20260925 09:58:12.547621 13170 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 4 FOLLOWER]: Rejecting Update request from peer 70718e3f6f9f489089c0d8354380ed6c for earlier term 3. Current term is 4. Ops: []
I20260925 09:58:12.550087 13591 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260925 09:58:12.550904 13591 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 3 LEADER]: Stepping down as leader of term 3
I20260925 09:58:12.551203 13591 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 70718e3f6f9f489089c0d8354380ed6c, State: Running, Role: LEADER
I20260925 09:58:12.551930 13591 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:12.553035 13591 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 3 FOLLOWER]: Advancing to term 4
I20260925 09:58:12.599376 13717 heartbeater.cc:347] Connected to a master server at 127.10.9.61:40425
I20260925 09:58:12.599900 13717 heartbeater.cc:464] Registering TS with master...
I20260925 09:58:12.600849 13718 heartbeater.cc:347] Connected to a master server at 127.10.9.62:43437
I20260925 09:58:12.600841 13716 heartbeater.cc:347] Connected to a master server at 127.10.9.60:42083
I20260925 09:58:12.601022 13717 heartbeater.cc:511] Master 127.10.9.61:40425 requested a full tablet report, sending...
I20260925 09:58:12.601362 13718 heartbeater.cc:464] Registering TS with master...
I20260925 09:58:12.601387 13716 heartbeater.cc:464] Registering TS with master...
I20260925 09:58:12.602558 13716 heartbeater.cc:511] Master 127.10.9.60:42083 requested a full tablet report, sending...
I20260925 09:58:12.602608 13718 heartbeater.cc:511] Master 127.10.9.62:43437 requested a full tablet report, sending...
I20260925 09:58:12.603592 13081 ts_manager.cc:194] Registered new tserver with Master: a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:12.605623 13006 ts_manager.cc:194] Registered new tserver with Master: a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:12.605660 13160 ts_manager.cc:194] Registered new tserver with Master: a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:12.606751 13081 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.9.3:39435
I20260925 09:58:12.620728 10276 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 09:58:12.682345 13017 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 4 FOLLOWER]: Refusing update from remote peer b7d0c1c2f1ed4e7a9ed54e73869d53b0: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260925 09:58:12.682690 13017 pending_rounds.cc:76] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c: Aborting all ops after (but not including) 5
I20260925 09:58:12.682906 13017 pending_rounds.cc:97] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260925 09:58:12.683163 13017 raft_consensus.cc:2866] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260925 09:58:12.685005 13596 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260925 09:58:12.689050 13591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7d0c1c2f1ed4e7a9ed54e73869d53b0. Latest consensus state: current_term: 4 leader_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } } }
I20260925 09:58:12.689811 13591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:58:12.667686 13082 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50556:
name: "default.Table_2bcc4dad40e04c2fad5d7739718f0e29"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:58:12.679184 13080 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50556:
name: "default.Table_7be114188e08442db683500d68eafdfd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:58:12.682866 13079 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50556:
name: "default.Table_d170c346194441ee8bff277382a7aea3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:58:12.666046 13081 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50556:
name: "default.Table_4ca061cf765540c096d1d9aeaa2dc63d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:58:12.708954 13591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } } }
I20260925 09:58:12.709656 13591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:58:12.713099 13170 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 4 FOLLOWER]: Refusing update from remote peer b7d0c1c2f1ed4e7a9ed54e73869d53b0: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260925 09:58:12.717221 13596 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } } }
I20260925 09:58:12.717949 13596 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [sys.catalog]: This master's current role is: LEADER
I20260925 09:58:12.718412 13596 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260925 09:58:12.733129 13082 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_2bcc4dad40e04c2fad5d7739718f0e29 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 09:58:12.737959 13079 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_d170c346194441ee8bff277382a7aea3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 09:58:12.738250 13080 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_7be114188e08442db683500d68eafdfd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 09:58:12.740734 13081 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_4ca061cf765540c096d1d9aeaa2dc63d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:58:12.749853 10276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu with pid 13118
W20260925 09:58:12.774072 13067 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.9.60:42083 (error 108) [suppressed 6 similar messages]
W20260925 09:58:12.783246 13757 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:58:12.778585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.9.60:42083 (local address 127.0.0.1:59380)
0925 09:58:12.780985 (+ 2400us) negotiation.cc:107] Waiting for socket to connect
0925 09:58:12.781045 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0925 09:58:12.781352 (+ 307us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 09:58:12.781826 (+ 474us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.9.60:42083: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":1887,"thread_start_us":1516,"threads_started":1}
W20260925 09:58:12.783700 13067 consensus_peers.cc:487] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 -> Peer 61e29afab0cf46819cbffc91248bf761 (127.10.9.60:42083): Couldn't send request to peer 61e29afab0cf46819cbffc91248bf761. Status: Network error: Client connection negotiation failed: client connection to 127.10.9.60:42083: BlockingWrite error: write error: Connection reset by peer (error 104). This is attempt 1: this message will repeat every 5th retry.
W20260925 09:58:13.077425 13550 heartbeater.cc:644] Failed to heartbeat to 127.10.9.60:42083 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.9.60:42083: connect: Connection refused (error 111)
I20260925 09:58:13.079823 13551 heartbeater.cc:503] Master 127.10.9.61:40425 was elected leader, sending a full tablet report...
I20260925 09:58:13.096114 13410 heartbeater.cc:503] Master 127.10.9.61:40425 was elected leader, sending a full tablet report...
I20260925 09:58:13.111379 13717 heartbeater.cc:503] Master 127.10.9.61:40425 was elected leader, sending a full tablet report...
W20260925 09:58:13.116398 13409 heartbeater.cc:644] Failed to heartbeat to 127.10.9.60:42083 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.9.60:42083: connect: Connection refused (error 111)
W20260925 09:58:13.120759 13716 heartbeater.cc:644] Failed to heartbeat to 127.10.9.60:42083 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.9.60:42083: connect: Connection refused (error 111)
I20260925 09:58:13.191555 13082 catalog_manager.cc:2148] created HMS catalog entry for table [id=2f774783b25a4b0e88449cd7cedaed2e]
I20260925 09:58:13.230394 13082 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:50556:
table { table_name: "default.Table_2bcc4dad40e04c2fad5d7739718f0e29" } new_table_name: "default.Table_fb4b236f93c04bb08319a6915fa8796c" modify_external_catalogs: true
I20260925 09:58:13.232511 13079 catalog_manager.cc:2148] created HMS catalog entry for table [id=ac75711cba134e34919b4106c864f477]
I20260925 09:58:13.288344 13081 catalog_manager.cc:2148] created HMS catalog entry for table [id=8c90bf002b144971986f1085cd102118]
I20260925 09:58:13.298826 13079 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:50556:
table { table_name: "default.Table_d170c346194441ee8bff277382a7aea3" } new_table_name: "default.Table_832367e542164f49b92f562d7a77699f" modify_external_catalogs: true
I20260925 09:58:13.317854 13081 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50556:
table { table_name: "default.Table_4ca061cf765540c096d1d9aeaa2dc63d" } modify_external_catalogs: true
I20260925 09:58:13.327925 13651 tablet_service.cc:1467] Processing CreateTablet for tablet fa643a5c8e8241aaaac8b68004df0d86 (DEFAULT_TABLE table=default.table_2bcc4dad40e04c2fad5d7739718f0e29 [id=2f774783b25a4b0e88449cd7cedaed2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:13.328763 13343 tablet_service.cc:1467] Processing CreateTablet for tablet fa643a5c8e8241aaaac8b68004df0d86 (DEFAULT_TABLE table=default.table_2bcc4dad40e04c2fad5d7739718f0e29 [id=2f774783b25a4b0e88449cd7cedaed2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:13.331866 13343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa643a5c8e8241aaaac8b68004df0d86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:13.319116 13081 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50556:
table { table_name: "default.Table_4ca061cf765540c096d1d9aeaa2dc63d" } modify_external_catalogs: true
I20260925 09:58:13.329856 13651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa643a5c8e8241aaaac8b68004df0d86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:13.342288 13485 tablet_service.cc:1467] Processing CreateTablet for tablet fa643a5c8e8241aaaac8b68004df0d86 (DEFAULT_TABLE table=default.table_2bcc4dad40e04c2fad5d7739718f0e29 [id=2f774783b25a4b0e88449cd7cedaed2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:13.344134 13485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa643a5c8e8241aaaac8b68004df0d86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:13.348722 13342 tablet_service.cc:1467] Processing CreateTablet for tablet 3e3942049d1244398152c2ee17f28160 (DEFAULT_TABLE table=default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:13.349891 13342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e3942049d1244398152c2ee17f28160. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:13.350898 13484 tablet_service.cc:1467] Processing CreateTablet for tablet 3e3942049d1244398152c2ee17f28160 (DEFAULT_TABLE table=default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:13.352006 13484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e3942049d1244398152c2ee17f28160. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:13.353960 13650 tablet_service.cc:1467] Processing CreateTablet for tablet 3e3942049d1244398152c2ee17f28160 (DEFAULT_TABLE table=default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:13.355139 13650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e3942049d1244398152c2ee17f28160. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:13.381896 13341 tablet_service.cc:1467] Processing CreateTablet for tablet 185afa49bad5423999a9c04485aceeb9 (DEFAULT_TABLE table=default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:13.383176 13341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 185afa49bad5423999a9c04485aceeb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:13.388449 13649 tablet_service.cc:1467] Processing CreateTablet for tablet 185afa49bad5423999a9c04485aceeb9 (DEFAULT_TABLE table=default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:13.389652 13649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 185afa49bad5423999a9c04485aceeb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:13.397687 13483 tablet_service.cc:1467] Processing CreateTablet for tablet 185afa49bad5423999a9c04485aceeb9 (DEFAULT_TABLE table=default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:13.398887 13483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 185afa49bad5423999a9c04485aceeb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:13.431185 13078 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50556:
name: "default.Table_3823faf045944cc1a828ce929b8a7275"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:58:13.444377 13772 tablet_bootstrap.cc:492] T 3e3942049d1244398152c2ee17f28160 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:13.450016 13770 tablet_bootstrap.cc:492] T 3e3942049d1244398152c2ee17f28160 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:13.453603 13772 tablet_bootstrap.cc:654] T 3e3942049d1244398152c2ee17f28160 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:13.463819 13770 tablet_bootstrap.cc:654] T 3e3942049d1244398152c2ee17f28160 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:13.466159 13770 log.cc:824] T 3e3942049d1244398152c2ee17f28160 P a06c3ef7a6a94f7cbcc03e7cd232f237: Log is configured to *not* fsync() on all Append() calls
I20260925 09:58:13.484911 13770 tablet_bootstrap.cc:492] T 3e3942049d1244398152c2ee17f28160 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:13.485522 13770 ts_tablet_manager.cc:1397] T 3e3942049d1244398152c2ee17f28160 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.036s user 0.011s sys 0.009s
I20260925 09:58:13.487270 13773 tablet_bootstrap.cc:492] T fa643a5c8e8241aaaac8b68004df0d86 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:13.489928 13772 log.cc:824] T 3e3942049d1244398152c2ee17f28160 P b3082cf15dc74c0480e5d33ecf592843: Log is configured to *not* fsync() on all Append() calls
I20260925 09:58:13.493458 13077 master_service.cc:946] ReplaceTablet: received request to replace tablet fa643a5c8e8241aaaac8b68004df0d86 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:13.497704 13080 catalog_manager.cc:2148] created HMS catalog entry for table [id=07bcde2a154b4eb2b4ee526d680b5ceb]
I20260925 09:58:13.504419 13076 master_service.cc:946] ReplaceTablet: received request to replace tablet 3e3942049d1244398152c2ee17f28160 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:13.512785 13773 tablet_bootstrap.cc:654] T fa643a5c8e8241aaaac8b68004df0d86 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:13.514282 13075 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50556:
name: "default.Table_a29aa5b33fd644a2ab80242e289ed25d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:58:13.517045 13773 log.cc:824] T fa643a5c8e8241aaaac8b68004df0d86 P c4d724d4b83e4d1e817a3a505dc24eb9: Log is configured to *not* fsync() on all Append() calls
I20260925 09:58:13.519531 13074 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50556:
name: "default.Table_4c3437905d9e4b05ac1fb4c6e3d0a417"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:58:13.520537 13772 tablet_bootstrap.cc:492] T 3e3942049d1244398152c2ee17f28160 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:13.521031 13772 ts_tablet_manager.cc:1397] T 3e3942049d1244398152c2ee17f28160 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.077s user 0.024s sys 0.014s
I20260925 09:58:13.548758 13080 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50556:
table { table_name: "default.Table_7be114188e08442db683500d68eafdfd" } modify_external_catalogs: true
I20260925 09:58:13.551743 13080 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50556:
table { table_name: "default.Table_7be114188e08442db683500d68eafdfd" } modify_external_catalogs: true
I20260925 09:58:13.551959 13773 tablet_bootstrap.cc:492] T fa643a5c8e8241aaaac8b68004df0d86 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:13.553512 13773 ts_tablet_manager.cc:1397] T fa643a5c8e8241aaaac8b68004df0d86 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.067s user 0.011s sys 0.011s
I20260925 09:58:13.575671 13077 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet fa643a5c8e8241aaaac8b68004df0d86
I20260925 09:58:13.577483 13077 catalog_manager.cc:6565] ReplaceTablet: tablet fa643a5c8e8241aaaac8b68004df0d86 deleted and replaced by tablet 39854cd5a1834ade80ed37a7f181eb25
I20260925 09:58:13.578418 13341 tablet_service.cc:1514] Processing DeleteTablet for tablet fa643a5c8e8241aaaac8b68004df0d86 with delete_type TABLET_DATA_DELETED (replaced by tablet 39854cd5a1834ade80ed37a7f181eb25) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:13.578280 13770 raft_consensus.cc:348] T 3e3942049d1244398152c2ee17f28160 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:13.579226 13770 raft_consensus.cc:374] T 3e3942049d1244398152c2ee17f28160 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:13.579540 13770 raft_consensus.cc:729] T 3e3942049d1244398152c2ee17f28160 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:13.583360 13772 raft_consensus.cc:348] T 3e3942049d1244398152c2ee17f28160 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:13.582386 13770 consensus_queue.cc:260] T 3e3942049d1244398152c2ee17f28160 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:13.584789 13649 tablet_service.cc:1514] Processing DeleteTablet for tablet fa643a5c8e8241aaaac8b68004df0d86 with delete_type TABLET_DATA_DELETED (replaced by tablet 39854cd5a1834ade80ed37a7f181eb25) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:13.587419 13485 tablet_service.cc:1514] Processing DeleteTablet for tablet fa643a5c8e8241aaaac8b68004df0d86 with delete_type TABLET_DATA_DELETED (replaced by tablet 39854cd5a1834ade80ed37a7f181eb25) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:13.584285 13772 raft_consensus.cc:374] T 3e3942049d1244398152c2ee17f28160 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:13.591329 13772 raft_consensus.cc:729] T 3e3942049d1244398152c2ee17f28160 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:13.592399 13772 consensus_queue.cc:260] T 3e3942049d1244398152c2ee17f28160 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
W20260925 09:58:13.599025 13068 catalog_manager.cc:4821] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): delete failed for tablet fa643a5c8e8241aaaac8b68004df0d86 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fa643a5c8e8241aaaac8b68004df0d86 already in progress: creating tablet
I20260925 09:58:13.603046 13772 ts_tablet_manager.cc:1428] T 3e3942049d1244398152c2ee17f28160 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.082s user 0.033s sys 0.007s
I20260925 09:58:13.604025 13772 tablet_bootstrap.cc:492] T fa643a5c8e8241aaaac8b68004df0d86 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:13.610060 13772 tablet_bootstrap.cc:654] T fa643a5c8e8241aaaac8b68004df0d86 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
W20260925 09:58:13.644083 13069 catalog_manager.cc:4821] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): delete failed for tablet fa643a5c8e8241aaaac8b68004df0d86 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fa643a5c8e8241aaaac8b68004df0d86 already in progress: creating tablet
I20260925 09:58:13.649712 13770 ts_tablet_manager.cc:1428] T 3e3942049d1244398152c2ee17f28160 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.164s user 0.040s sys 0.003s
I20260925 09:58:13.664878 13076 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet 3e3942049d1244398152c2ee17f28160
I20260925 09:58:13.669929 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e3942049d1244398152c2ee17f28160 with delete_type TABLET_DATA_DELETED (replaced by tablet 2086e4e50cc84e2e9ecf8a44c194e8d4) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:13.672353 13782 raft_consensus.cc:482] T 3e3942049d1244398152c2ee17f28160 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:13.672927 13782 raft_consensus.cc:504] T 3e3942049d1244398152c2ee17f28160 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:13.674269 13649 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e3942049d1244398152c2ee17f28160 with delete_type TABLET_DATA_DELETED (replaced by tablet 2086e4e50cc84e2e9ecf8a44c194e8d4) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:13.675431 13770 tablet_bootstrap.cc:492] T fa643a5c8e8241aaaac8b68004df0d86 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:13.676519 13780 tablet_replica.cc:330] T 3e3942049d1244398152c2ee17f28160 P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:13.677299 13780 raft_consensus.cc:2229] T 3e3942049d1244398152c2ee17f28160 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:13.677840 13780 raft_consensus.cc:2258] T 3e3942049d1244398152c2ee17f28160 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:13.678601 10276 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu
/tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.9.60:42083
--webserver_interface=127.10.9.60
--webserver_port=36655
--builtin_ntp_servers=127.10.9.20:35467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.9.62:43437,127.10.9.61:40425,127.10.9.60:42083
--hive_metastore_uris=thrift://127.0.0.1:42723
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260925 09:58:13.682917 13076 catalog_manager.cc:6565] ReplaceTablet: tablet 3e3942049d1244398152c2ee17f28160 deleted and replaced by tablet 2086e4e50cc84e2e9ecf8a44c194e8d4
I20260925 09:58:13.689242 13780 ts_tablet_manager.cc:1905] T 3e3942049d1244398152c2ee17f28160 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:13.694845 13782 leader_election.cc:288] T 3e3942049d1244398152c2ee17f28160 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:13.696069 13485 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e3942049d1244398152c2ee17f28160 with delete_type TABLET_DATA_DELETED (replaced by tablet 2086e4e50cc84e2e9ecf8a44c194e8d4) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:13.699317 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e3942049d1244398152c2ee17f28160" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
W20260925 09:58:13.701486 13298 leader_election.cc:341] T 3e3942049d1244398152c2ee17f28160 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:58:13.713109 13069 catalog_manager.cc:4821] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): delete failed for tablet 3e3942049d1244398152c2ee17f28160 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3e3942049d1244398152c2ee17f28160 already in progress: creating tablet
I20260925 09:58:13.715134 13770 tablet_bootstrap.cc:654] T fa643a5c8e8241aaaac8b68004df0d86 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:13.715975 13772 tablet_bootstrap.cc:492] T fa643a5c8e8241aaaac8b68004df0d86 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:13.716434 13772 ts_tablet_manager.cc:1397] T fa643a5c8e8241aaaac8b68004df0d86 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.113s user 0.015s sys 0.033s
W20260925 09:58:13.719430 13066 catalog_manager.cc:4821] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): delete failed for tablet fa643a5c8e8241aaaac8b68004df0d86 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fa643a5c8e8241aaaac8b68004df0d86 already in progress: creating tablet
I20260925 09:58:13.729223 13773 raft_consensus.cc:348] T fa643a5c8e8241aaaac8b68004df0d86 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:13.730355 13773 raft_consensus.cc:374] T fa643a5c8e8241aaaac8b68004df0d86 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:13.730741 13773 raft_consensus.cc:729] T fa643a5c8e8241aaaac8b68004df0d86 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:13.732026 13773 consensus_queue.cc:260] T fa643a5c8e8241aaaac8b68004df0d86 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:13.736588 13770 tablet_bootstrap.cc:492] T fa643a5c8e8241aaaac8b68004df0d86 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:13.736997 13770 ts_tablet_manager.cc:1397] T fa643a5c8e8241aaaac8b68004df0d86 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.062s user 0.010s sys 0.000s
I20260925 09:58:13.719385 13772 raft_consensus.cc:348] T fa643a5c8e8241aaaac8b68004df0d86 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:13.739744 13772 raft_consensus.cc:374] T fa643a5c8e8241aaaac8b68004df0d86 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:13.739426 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e3942049d1244398152c2ee17f28160" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:13.740016 13772 raft_consensus.cc:729] T fa643a5c8e8241aaaac8b68004df0d86 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:13.740758 13772 consensus_queue.cc:260] T fa643a5c8e8241aaaac8b68004df0d86 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:13.741070 13770 raft_consensus.cc:348] T fa643a5c8e8241aaaac8b68004df0d86 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
W20260925 09:58:13.741340 13297 leader_election.cc:341] T 3e3942049d1244398152c2ee17f28160 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Illegal state: Tablet not RUNNING: STOPPED
I20260925 09:58:13.741772 13297 leader_election.cc:302] T 3e3942049d1244398152c2ee17f28160 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:58:13.742692 13782 raft_consensus.cc:2726] T 3e3942049d1244398152c2ee17f28160 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:58:13.743832 13772 ts_tablet_manager.cc:1428] T fa643a5c8e8241aaaac8b68004df0d86 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.027s user 0.007s sys 0.000s
I20260925 09:58:13.744746 13772 tablet_bootstrap.cc:492] T 185afa49bad5423999a9c04485aceeb9 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:13.741734 13770 raft_consensus.cc:374] T fa643a5c8e8241aaaac8b68004df0d86 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:13.747440 13770 raft_consensus.cc:729] T fa643a5c8e8241aaaac8b68004df0d86 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:13.751675 13770 consensus_queue.cc:260] T fa643a5c8e8241aaaac8b68004df0d86 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:13.756172 13773 ts_tablet_manager.cc:1428] T fa643a5c8e8241aaaac8b68004df0d86 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.202s user 0.030s sys 0.009s
I20260925 09:58:13.758955 13485 tablet_service.cc:1467] Processing CreateTablet for tablet b3b185b9ce9d4da0a8eb36cc6e7dce08 (DEFAULT_TABLE table=default.table_7be114188e08442db683500d68eafdfd [id=07bcde2a154b4eb2b4ee526d680b5ceb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:13.759732 13773 tablet_bootstrap.cc:492] T 3e3942049d1244398152c2ee17f28160 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:13.760141 13485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3b185b9ce9d4da0a8eb36cc6e7dce08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:13.763628 13341 tablet_service.cc:1467] Processing CreateTablet for tablet b3b185b9ce9d4da0a8eb36cc6e7dce08 (DEFAULT_TABLE table=default.table_7be114188e08442db683500d68eafdfd [id=07bcde2a154b4eb2b4ee526d680b5ceb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:13.764747 13341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3b185b9ce9d4da0a8eb36cc6e7dce08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:13.768010 13773 tablet_bootstrap.cc:654] T 3e3942049d1244398152c2ee17f28160 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
W20260925 09:58:13.767970 13418 tablet.cc:2369] T 3e3942049d1244398152c2ee17f28160 P b3082cf15dc74c0480e5d33ecf592843: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:58:13.771183 13772 tablet_bootstrap.cc:654] T 185afa49bad5423999a9c04485aceeb9 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:13.771873 13780 ts_tablet_manager.cc:1918] T 3e3942049d1244398152c2ee17f28160 P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:13.764604 13649 tablet_service.cc:1467] Processing CreateTablet for tablet b3b185b9ce9d4da0a8eb36cc6e7dce08 (DEFAULT_TABLE table=default.table_7be114188e08442db683500d68eafdfd [id=07bcde2a154b4eb2b4ee526d680b5ceb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:13.772333 13780 log.cc:1196] T 3e3942049d1244398152c2ee17f28160 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/3e3942049d1244398152c2ee17f28160
W20260925 09:58:13.774534 13557 tablet.cc:2369] T fa643a5c8e8241aaaac8b68004df0d86 P c4d724d4b83e4d1e817a3a505dc24eb9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:58:13.776188 13770 ts_tablet_manager.cc:1428] T fa643a5c8e8241aaaac8b68004df0d86 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.039s user 0.008s sys 0.000s
I20260925 09:58:13.776468 13649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3b185b9ce9d4da0a8eb36cc6e7dce08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:13.780259 13770 tablet_bootstrap.cc:492] T 185afa49bad5423999a9c04485aceeb9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:13.781307 13076 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50556:
name: "default.Table_eadc20bcf90e4a72b8f26e3a4798e259"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:58:13.784466 13780 ts_tablet_manager.cc:1939] T 3e3942049d1244398152c2ee17f28160 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:13.787546 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet 3e3942049d1244398152c2ee17f28160 (table default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477]) successfully deleted
I20260925 09:58:13.791231 13483 tablet_service.cc:1514] Processing DeleteTablet for tablet fa643a5c8e8241aaaac8b68004df0d86 with delete_type TABLET_DATA_DELETED (replaced by tablet 39854cd5a1834ade80ed37a7f181eb25) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:13.792363 13781 tablet_replica.cc:330] T fa643a5c8e8241aaaac8b68004df0d86 P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:13.793211 13781 raft_consensus.cc:2229] T fa643a5c8e8241aaaac8b68004df0d86 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:13.783998 13777 tablet_replica.cc:330] T 3e3942049d1244398152c2ee17f28160 P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:13.796144 13770 tablet_bootstrap.cc:654] T 185afa49bad5423999a9c04485aceeb9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:13.796198 13650 tablet_service.cc:1467] Processing CreateTablet for tablet 39854cd5a1834ade80ed37a7f181eb25 (DEFAULT_TABLE table=default.table_2bcc4dad40e04c2fad5d7739718f0e29 [id=2f774783b25a4b0e88449cd7cedaed2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:13.797394 13650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39854cd5a1834ade80ed37a7f181eb25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:13.790407 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet fa643a5c8e8241aaaac8b68004df0d86 with delete_type TABLET_DATA_DELETED (replaced by tablet 39854cd5a1834ade80ed37a7f181eb25) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:13.807065 13077 master_service.cc:946] ReplaceTablet: received request to replace tablet 185afa49bad5423999a9c04485aceeb9 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:13.809677 13773 tablet_bootstrap.cc:492] T 3e3942049d1244398152c2ee17f28160 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:13.810184 13773 ts_tablet_manager.cc:1397] T 3e3942049d1244398152c2ee17f28160 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.051s user 0.003s sys 0.007s
W20260925 09:58:13.811352 13721 tablet.cc:2369] T fa643a5c8e8241aaaac8b68004df0d86 P a06c3ef7a6a94f7cbcc03e7cd232f237: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 09:58:13.812752 13781 raft_consensus.cc:2258] T fa643a5c8e8241aaaac8b68004df0d86 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:13.813189 13342 tablet_service.cc:1467] Processing CreateTablet for tablet 39854cd5a1834ade80ed37a7f181eb25 (DEFAULT_TABLE table=default.table_2bcc4dad40e04c2fad5d7739718f0e29 [id=2f774783b25a4b0e88449cd7cedaed2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:13.814682 13342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39854cd5a1834ade80ed37a7f181eb25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:13.796326 13777 raft_consensus.cc:2229] T 3e3942049d1244398152c2ee17f28160 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:13.813762 13773 raft_consensus.cc:348] T 3e3942049d1244398152c2ee17f28160 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:13.831503 13773 raft_consensus.cc:374] T 3e3942049d1244398152c2ee17f28160 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:13.831926 13773 raft_consensus.cc:729] T 3e3942049d1244398152c2ee17f28160 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:13.832755 13773 consensus_queue.cc:260] T 3e3942049d1244398152c2ee17f28160 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:13.834824 13773 ts_tablet_manager.cc:1428] T 3e3942049d1244398152c2ee17f28160 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.024s user 0.002s sys 0.006s
I20260925 09:58:13.835713 13773 tablet_bootstrap.cc:492] T 185afa49bad5423999a9c04485aceeb9 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:13.842675 13773 tablet_bootstrap.cc:654] T 185afa49bad5423999a9c04485aceeb9 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:13.846776 13777 raft_consensus.cc:2258] T 3e3942049d1244398152c2ee17f28160 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:13.850595 13770 tablet_bootstrap.cc:492] T 185afa49bad5423999a9c04485aceeb9 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:13.851042 13770 ts_tablet_manager.cc:1397] T 185afa49bad5423999a9c04485aceeb9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.071s user 0.013s sys 0.007s
I20260925 09:58:13.852275 13772 tablet_bootstrap.cc:492] T 185afa49bad5423999a9c04485aceeb9 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:13.852713 13772 ts_tablet_manager.cc:1397] T 185afa49bad5423999a9c04485aceeb9 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.108s user 0.015s sys 0.012s
I20260925 09:58:13.854153 13770 raft_consensus.cc:348] T 185afa49bad5423999a9c04485aceeb9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:13.855072 13772 raft_consensus.cc:348] T 185afa49bad5423999a9c04485aceeb9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:13.855793 13772 raft_consensus.cc:374] T 185afa49bad5423999a9c04485aceeb9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:13.856060 13772 raft_consensus.cc:729] T 185afa49bad5423999a9c04485aceeb9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:13.857968 13777 ts_tablet_manager.cc:1905] T 3e3942049d1244398152c2ee17f28160 P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:13.854854 13770 raft_consensus.cc:374] T 185afa49bad5423999a9c04485aceeb9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:13.863730 13770 raft_consensus.cc:729] T 185afa49bad5423999a9c04485aceeb9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:13.867095 13772 consensus_queue.cc:260] T 185afa49bad5423999a9c04485aceeb9 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:13.871022 13770 consensus_queue.cc:260] T 185afa49bad5423999a9c04485aceeb9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:13.872047 13772 ts_tablet_manager.cc:1428] T 185afa49bad5423999a9c04485aceeb9 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.019s user 0.007s sys 0.000s
I20260925 09:58:13.875757 13650 tablet_service.cc:1514] Processing DeleteTablet for tablet fa643a5c8e8241aaaac8b68004df0d86 with delete_type TABLET_DATA_DELETED (replaced by tablet 39854cd5a1834ade80ed37a7f181eb25) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:13.876753 13780 tablet_replica.cc:330] T fa643a5c8e8241aaaac8b68004df0d86 P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:13.877431 13780 raft_consensus.cc:2229] T fa643a5c8e8241aaaac8b68004df0d86 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:13.877911 13780 raft_consensus.cc:2258] T fa643a5c8e8241aaaac8b68004df0d86 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:13.880434 13780 ts_tablet_manager.cc:1905] T fa643a5c8e8241aaaac8b68004df0d86 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:13.880613 13770 ts_tablet_manager.cc:1428] T 185afa49bad5423999a9c04485aceeb9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.029s user 0.004s sys 0.004s
I20260925 09:58:13.884697 13077 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet 185afa49bad5423999a9c04485aceeb9
I20260925 09:58:13.886430 13341 tablet_service.cc:1514] Processing DeleteTablet for tablet 185afa49bad5423999a9c04485aceeb9 with delete_type TABLET_DATA_DELETED (replaced by tablet 20ccdde9c1e3476485f50e8361e5ae0b) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:13.887972 13772 tablet_bootstrap.cc:492] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:13.888109 13770 tablet_bootstrap.cc:492] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:13.888981 13077 catalog_manager.cc:6565] ReplaceTablet: tablet 185afa49bad5423999a9c04485aceeb9 deleted and replaced by tablet 20ccdde9c1e3476485f50e8361e5ae0b
I20260925 09:58:13.890041 13650 tablet_service.cc:1514] Processing DeleteTablet for tablet 185afa49bad5423999a9c04485aceeb9 with delete_type TABLET_DATA_DELETED (replaced by tablet 20ccdde9c1e3476485f50e8361e5ae0b) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:13.893864 13770 tablet_bootstrap.cc:654] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:13.900722 13772 tablet_bootstrap.cc:654] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:13.846468 13483 tablet_service.cc:1467] Processing CreateTablet for tablet 39854cd5a1834ade80ed37a7f181eb25 (DEFAULT_TABLE table=default.table_2bcc4dad40e04c2fad5d7739718f0e29 [id=2f774783b25a4b0e88449cd7cedaed2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:13.850106 13781 ts_tablet_manager.cc:1905] T fa643a5c8e8241aaaac8b68004df0d86 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:13.896469 13485 tablet_service.cc:1514] Processing DeleteTablet for tablet 185afa49bad5423999a9c04485aceeb9 with delete_type TABLET_DATA_DELETED (replaced by tablet 20ccdde9c1e3476485f50e8361e5ae0b) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:13.908229 13483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39854cd5a1834ade80ed37a7f181eb25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:13.909411 13779 raft_consensus.cc:482] T 185afa49bad5423999a9c04485aceeb9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:13.909947 13779 raft_consensus.cc:504] T 185afa49bad5423999a9c04485aceeb9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:13.916296 13780 ts_tablet_manager.cc:1918] T fa643a5c8e8241aaaac8b68004df0d86 P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:13.916777 13780 log.cc:1196] T fa643a5c8e8241aaaac8b68004df0d86 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/fa643a5c8e8241aaaac8b68004df0d86
I20260925 09:58:13.917497 13780 ts_tablet_manager.cc:1939] T fa643a5c8e8241aaaac8b68004df0d86 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:13.918154 13779 leader_election.cc:288] T 185afa49bad5423999a9c04485aceeb9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
I20260925 09:58:13.920403 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet fa643a5c8e8241aaaac8b68004df0d86 (table default.table_2bcc4dad40e04c2fad5d7739718f0e29 [id=2f774783b25a4b0e88449cd7cedaed2e]) successfully deleted
I20260925 09:58:13.921988 13782 raft_consensus.cc:482] T fa643a5c8e8241aaaac8b68004df0d86 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:13.922410 13782 raft_consensus.cc:504] T fa643a5c8e8241aaaac8b68004df0d86 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:13.924278 13782 leader_election.cc:288] T fa643a5c8e8241aaaac8b68004df0d86 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
W20260925 09:58:13.926612 13297 leader_election.cc:341] T fa643a5c8e8241aaaac8b68004df0d86 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Not found: Tablet not found: fa643a5c8e8241aaaac8b68004df0d86
I20260925 09:58:13.938359 13772 tablet_bootstrap.cc:492] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:13.938966 13772 ts_tablet_manager.cc:1397] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.051s user 0.011s sys 0.000s
I20260925 09:58:13.935760 13780 tablet_replica.cc:330] T 185afa49bad5423999a9c04485aceeb9 P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:13.942994 13780 raft_consensus.cc:2229] T 185afa49bad5423999a9c04485aceeb9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:13.943632 13780 raft_consensus.cc:2258] T 185afa49bad5423999a9c04485aceeb9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:13.944689 13772 raft_consensus.cc:348] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:13.946902 13780 ts_tablet_manager.cc:1905] T 185afa49bad5423999a9c04485aceeb9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:13.947237 13772 raft_consensus.cc:374] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:13.950699 13485 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e3942049d1244398152c2ee17f28160 with delete_type TABLET_DATA_DELETED (replaced by tablet 2086e4e50cc84e2e9ecf8a44c194e8d4) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:13.951027 13770 tablet_bootstrap.cc:492] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:13.959574 13788 raft_consensus.cc:482] T 3e3942049d1244398152c2ee17f28160 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:13.960119 13788 raft_consensus.cc:504] T 3e3942049d1244398152c2ee17f28160 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:13.965209 13788 leader_election.cc:288] T 3e3942049d1244398152c2ee17f28160 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:13.952327 13772 raft_consensus.cc:729] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:13.966960 13770 ts_tablet_manager.cc:1397] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.079s user 0.007s sys 0.004s
I20260925 09:58:13.966523 13772 consensus_queue.cc:260] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:13.974121 13777 ts_tablet_manager.cc:1918] T 3e3942049d1244398152c2ee17f28160 P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:13.975308 13772 ts_tablet_manager.cc:1428] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.036s user 0.006s sys 0.000s
I20260925 09:58:13.975328 13777 log.cc:1196] T 3e3942049d1244398152c2ee17f28160 P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/3e3942049d1244398152c2ee17f28160
I20260925 09:58:13.976286 13772 tablet_bootstrap.cc:492] T 39854cd5a1834ade80ed37a7f181eb25 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:13.982995 13777 ts_tablet_manager.cc:1939] T 3e3942049d1244398152c2ee17f28160 P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:13.985661 13773 tablet_bootstrap.cc:492] T 185afa49bad5423999a9c04485aceeb9 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:13.985889 13777 tablet_replica.cc:330] T fa643a5c8e8241aaaac8b68004df0d86 P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:13.986148 13773 ts_tablet_manager.cc:1397] T 185afa49bad5423999a9c04485aceeb9 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.151s user 0.009s sys 0.001s
I20260925 09:58:13.986706 13777 raft_consensus.cc:2229] T fa643a5c8e8241aaaac8b68004df0d86 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:13.988593 13773 raft_consensus.cc:348] T 185afa49bad5423999a9c04485aceeb9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:13.989315 13773 raft_consensus.cc:374] T 185afa49bad5423999a9c04485aceeb9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:13.989596 13773 raft_consensus.cc:729] T 185afa49bad5423999a9c04485aceeb9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:13.990238 13773 consensus_queue.cc:260] T 185afa49bad5423999a9c04485aceeb9 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:13.991360 13770 raft_consensus.cc:348] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:13.992093 13770 raft_consensus.cc:374] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:13.992434 13770 raft_consensus.cc:729] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:13.996083 13650 tablet_service.cc:1514] Processing DeleteTablet for tablet fa643a5c8e8241aaaac8b68004df0d86 with delete_type TABLET_DATA_DELETED (replaced by tablet 39854cd5a1834ade80ed37a7f181eb25) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:13.996233 13770 consensus_queue.cc:260] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:14.003432 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "185afa49bad5423999a9c04485aceeb9" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
I20260925 09:58:14.004186 13505 raft_consensus.cc:2454] T 185afa49bad5423999a9c04485aceeb9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 0.
I20260925 09:58:14.005384 13780 ts_tablet_manager.cc:1918] T 185afa49bad5423999a9c04485aceeb9 P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:14.005841 13780 log.cc:1196] T 185afa49bad5423999a9c04485aceeb9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/185afa49bad5423999a9c04485aceeb9
I20260925 09:58:14.005669 13606 leader_election.cc:302] T 185afa49bad5423999a9c04485aceeb9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:14.005995 13781 ts_tablet_manager.cc:1918] T fa643a5c8e8241aaaac8b68004df0d86 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:14.006397 13781 log.cc:1196] T fa643a5c8e8241aaaac8b68004df0d86 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/fa643a5c8e8241aaaac8b68004df0d86
I20260925 09:58:14.006507 13780 ts_tablet_manager.cc:1939] T 185afa49bad5423999a9c04485aceeb9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
W20260925 09:58:14.010946 13606 raft_consensus.cc:2670] T 185afa49bad5423999a9c04485aceeb9 P a06c3ef7a6a94f7cbcc03e7cd232f237: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260925 09:58:14.011161 13650 tablet_service.cc:1467] Processing CreateTablet for tablet 2086e4e50cc84e2e9ecf8a44c194e8d4 (DEFAULT_TABLE table=default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:14.012313 13650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2086e4e50cc84e2e9ecf8a44c194e8d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:14.014551 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet 185afa49bad5423999a9c04485aceeb9 (table default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]) successfully deleted
W20260925 09:58:14.017524 13068 catalog_manager.cc:4808] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): delete failed for tablet fa643a5c8e8241aaaac8b68004df0d86 because the tablet was not found. No further retry: Not found: Tablet not found: fa643a5c8e8241aaaac8b68004df0d86
I20260925 09:58:14.020017 13770 ts_tablet_manager.cc:1428] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.053s user 0.001s sys 0.007s
I20260925 09:58:14.022405 13772 tablet_bootstrap.cc:654] T 39854cd5a1834ade80ed37a7f181eb25 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:14.025606 13073 master_service.cc:946] ReplaceTablet: received request to replace tablet fa643a5c8e8241aaaac8b68004df0d86 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:13.987229 13777 raft_consensus.cc:2258] T fa643a5c8e8241aaaac8b68004df0d86 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260925 09:58:14.018921 13440 leader_election.cc:341] T 3e3942049d1244398152c2ee17f28160 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Not found: Tablet not found: 3e3942049d1244398152c2ee17f28160
I20260925 09:58:14.030074 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa643a5c8e8241aaaac8b68004df0d86" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
I20260925 09:58:14.025753 13483 tablet_service.cc:1467] Processing CreateTablet for tablet 2086e4e50cc84e2e9ecf8a44c194e8d4 (DEFAULT_TABLE table=default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:14.033620 13483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2086e4e50cc84e2e9ecf8a44c194e8d4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:58:14.034406 13298 leader_election.cc:341] T fa643a5c8e8241aaaac8b68004df0d86 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Illegal state: Tablet not RUNNING: STOPPED
I20260925 09:58:14.034816 13298 leader_election.cc:302] T fa643a5c8e8241aaaac8b68004df0d86 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
W20260925 09:58:14.035468 13298 raft_consensus.cc:2670] T fa643a5c8e8241aaaac8b68004df0d86 P b3082cf15dc74c0480e5d33ecf592843: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260925 09:58:14.039645 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet 3e3942049d1244398152c2ee17f28160 (table default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477]) successfully deleted
I20260925 09:58:14.041087 13342 tablet_service.cc:1467] Processing CreateTablet for tablet 2086e4e50cc84e2e9ecf8a44c194e8d4 (DEFAULT_TABLE table=default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:14.042330 13342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2086e4e50cc84e2e9ecf8a44c194e8d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:14.048205 13770 tablet_bootstrap.cc:492] T 39854cd5a1834ade80ed37a7f181eb25 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:14.052487 13777 ts_tablet_manager.cc:1905] T fa643a5c8e8241aaaac8b68004df0d86 P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:14.007282 13781 ts_tablet_manager.cc:1939] T fa643a5c8e8241aaaac8b68004df0d86 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:14.062637 13649 tablet_service.cc:1514] Processing DeleteTablet for tablet 185afa49bad5423999a9c04485aceeb9 with delete_type TABLET_DATA_DELETED (replaced by tablet 20ccdde9c1e3476485f50e8361e5ae0b) from {username='slave'} at 127.0.0.1:54548
W20260925 09:58:14.064661 13068 catalog_manager.cc:4808] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): delete failed for tablet 185afa49bad5423999a9c04485aceeb9 because the tablet was not found. No further retry: Not found: Tablet not found: 185afa49bad5423999a9c04485aceeb9
W20260925 09:58:14.065994 13438 leader_election.cc:341] T 3e3942049d1244398152c2ee17f28160 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Not found: Tablet not found: 3e3942049d1244398152c2ee17f28160
I20260925 09:58:14.066421 13438 leader_election.cc:302] T 3e3942049d1244398152c2ee17f28160 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4d724d4b83e4d1e817a3a505dc24eb9; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, b3082cf15dc74c0480e5d33ecf592843
I20260925 09:58:14.070514 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "185afa49bad5423999a9c04485aceeb9" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:14.071450 13363 raft_consensus.cc:2454] T 185afa49bad5423999a9c04485aceeb9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 0.
I20260925 09:58:14.076061 13770 tablet_bootstrap.cc:654] T 39854cd5a1834ade80ed37a7f181eb25 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:14.081728 13772 tablet_bootstrap.cc:492] T 39854cd5a1834ade80ed37a7f181eb25 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:14.082239 13772 ts_tablet_manager.cc:1397] T 39854cd5a1834ade80ed37a7f181eb25 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.106s user 0.011s sys 0.009s
I20260925 09:58:14.084704 13772 raft_consensus.cc:348] T 39854cd5a1834ade80ed37a7f181eb25 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } }
I20260925 09:58:14.091511 13772 raft_consensus.cc:374] T 39854cd5a1834ade80ed37a7f181eb25 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:14.091874 13772 raft_consensus.cc:729] T 39854cd5a1834ade80ed37a7f181eb25 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:14.097823 13073 master_service.cc:946] ReplaceTablet: received request to replace tablet fa643a5c8e8241aaaac8b68004df0d86 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:14.092623 13772 consensus_queue.cc:260] T 39854cd5a1834ade80ed37a7f181eb25 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } }
I20260925 09:58:14.105221 13770 tablet_bootstrap.cc:492] T 39854cd5a1834ade80ed37a7f181eb25 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:14.105643 13770 ts_tablet_manager.cc:1397] T 39854cd5a1834ade80ed37a7f181eb25 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.058s user 0.004s sys 0.006s
I20260925 09:58:14.107789 13770 raft_consensus.cc:348] T 39854cd5a1834ade80ed37a7f181eb25 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } }
I20260925 09:58:14.108299 13770 raft_consensus.cc:374] T 39854cd5a1834ade80ed37a7f181eb25 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:14.108559 13770 raft_consensus.cc:729] T 39854cd5a1834ade80ed37a7f181eb25 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:14.109135 13770 consensus_queue.cc:260] T 39854cd5a1834ade80ed37a7f181eb25 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } }
I20260925 09:58:14.111021 13770 ts_tablet_manager.cc:1428] T 39854cd5a1834ade80ed37a7f181eb25 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
W20260925 09:58:14.115208 13438 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.1:42161, user_credentials={real_user=slave}} blocked reactor thread for 49390.2us
I20260925 09:58:14.117623 13788 raft_consensus.cc:2726] T 3e3942049d1244398152c2ee17f28160 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:58:14.124395 13773 ts_tablet_manager.cc:1428] T 185afa49bad5423999a9c04485aceeb9 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.138s user 0.006s sys 0.001s
I20260925 09:58:14.125496 13770 tablet_bootstrap.cc:492] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:14.127843 13773 tablet_bootstrap.cc:492] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:14.131102 13770 tablet_bootstrap.cc:654] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:14.134138 13772 ts_tablet_manager.cc:1428] T 39854cd5a1834ade80ed37a7f181eb25 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.052s user 0.004s sys 0.000s
I20260925 09:58:14.136111 13777 ts_tablet_manager.cc:1918] T fa643a5c8e8241aaaac8b68004df0d86 P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:14.136480 13777 log.cc:1196] T fa643a5c8e8241aaaac8b68004df0d86 P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/fa643a5c8e8241aaaac8b68004df0d86
I20260925 09:58:14.137146 13777 ts_tablet_manager.cc:1939] T fa643a5c8e8241aaaac8b68004df0d86 P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:14.139698 13777 tablet_replica.cc:330] T 185afa49bad5423999a9c04485aceeb9 P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:14.139947 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet fa643a5c8e8241aaaac8b68004df0d86 (table default.table_2bcc4dad40e04c2fad5d7739718f0e29 [id=2f774783b25a4b0e88449cd7cedaed2e]) successfully deleted
I20260925 09:58:14.140407 13777 raft_consensus.cc:2229] T 185afa49bad5423999a9c04485aceeb9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:14.140935 13777 raft_consensus.cc:2258] T 185afa49bad5423999a9c04485aceeb9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260925 09:58:14.143500 13073 catalog_manager.cc:5328] Ignoring report from unknown tablet 185afa49bad5423999a9c04485aceeb9
I20260925 09:58:14.144161 13777 ts_tablet_manager.cc:1905] T 185afa49bad5423999a9c04485aceeb9 P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:14.152773 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet fa643a5c8e8241aaaac8b68004df0d86 (table default.table_2bcc4dad40e04c2fad5d7739718f0e29 [id=2f774783b25a4b0e88449cd7cedaed2e]) successfully deleted
I20260925 09:58:14.156175 13781 tablet_replica.cc:330] T 185afa49bad5423999a9c04485aceeb9 P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:14.156873 13781 raft_consensus.cc:2229] T 185afa49bad5423999a9c04485aceeb9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:14.157323 13781 raft_consensus.cc:2258] T 185afa49bad5423999a9c04485aceeb9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:14.159140 13781 ts_tablet_manager.cc:1905] T 185afa49bad5423999a9c04485aceeb9 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 09:58:14.170779 13073 catalog_manager.cc:5328] Ignoring report from unknown tablet 185afa49bad5423999a9c04485aceeb9
I20260925 09:58:14.172320 13770 tablet_bootstrap.cc:492] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:14.172825 13770 ts_tablet_manager.cc:1397] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.048s user 0.006s sys 0.005s
I20260925 09:58:14.175683 13772 tablet_bootstrap.cc:492] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:14.176594 13341 tablet_service.cc:1467] Processing CreateTablet for tablet 20ccdde9c1e3476485f50e8361e5ae0b (DEFAULT_TABLE table=default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:14.177624 13341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20ccdde9c1e3476485f50e8361e5ae0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:14.174171 13777 ts_tablet_manager.cc:1918] T 185afa49bad5423999a9c04485aceeb9 P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:14.182003 13777 log.cc:1196] T 185afa49bad5423999a9c04485aceeb9 P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/185afa49bad5423999a9c04485aceeb9
I20260925 09:58:14.182350 13772 tablet_bootstrap.cc:654] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:14.182720 13777 ts_tablet_manager.cc:1939] T 185afa49bad5423999a9c04485aceeb9 P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:14.185366 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet 185afa49bad5423999a9c04485aceeb9 (table default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]) successfully deleted
I20260925 09:58:14.176661 13770 raft_consensus.cc:348] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:14.188046 13770 raft_consensus.cc:374] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:14.188328 13770 raft_consensus.cc:729] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:14.189100 13770 consensus_queue.cc:260] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:14.193449 13650 tablet_service.cc:1467] Processing CreateTablet for tablet 20ccdde9c1e3476485f50e8361e5ae0b (DEFAULT_TABLE table=default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:14.194648 13650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20ccdde9c1e3476485f50e8361e5ae0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:14.198738 13770 ts_tablet_manager.cc:1428] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.026s user 0.004s sys 0.001s
I20260925 09:58:14.216377 13781 ts_tablet_manager.cc:1918] T 185afa49bad5423999a9c04485aceeb9 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:14.216795 13781 log.cc:1196] T 185afa49bad5423999a9c04485aceeb9 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/185afa49bad5423999a9c04485aceeb9
I20260925 09:58:14.217438 13781 ts_tablet_manager.cc:1939] T 185afa49bad5423999a9c04485aceeb9 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:14.219743 13781 tablet_replica.cc:330] T 3e3942049d1244398152c2ee17f28160 P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:14.220367 13781 raft_consensus.cc:2229] T 3e3942049d1244398152c2ee17f28160 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:14.220824 13781 raft_consensus.cc:2258] T 3e3942049d1244398152c2ee17f28160 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:14.221724 13773 tablet_bootstrap.cc:654] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:14.224334 13781 ts_tablet_manager.cc:1905] T 3e3942049d1244398152c2ee17f28160 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:14.226230 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet 185afa49bad5423999a9c04485aceeb9 (table default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]) successfully deleted
I20260925 09:58:14.234215 13077 master_service.cc:946] ReplaceTablet: received request to replace tablet 2086e4e50cc84e2e9ecf8a44c194e8d4 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:14.234962 13483 tablet_service.cc:1467] Processing CreateTablet for tablet 20ccdde9c1e3476485f50e8361e5ae0b (DEFAULT_TABLE table=default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:14.236577 13772 tablet_bootstrap.cc:492] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:14.237224 13772 ts_tablet_manager.cc:1397] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.062s user 0.004s sys 0.008s
I20260925 09:58:14.238396 13781 ts_tablet_manager.cc:1918] T 3e3942049d1244398152c2ee17f28160 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:14.238763 13781 log.cc:1196] T 3e3942049d1244398152c2ee17f28160 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/3e3942049d1244398152c2ee17f28160
I20260925 09:58:14.239451 13781 ts_tablet_manager.cc:1939] T 3e3942049d1244398152c2ee17f28160 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:14.243639 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet 3e3942049d1244398152c2ee17f28160 (table default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477]) successfully deleted
I20260925 09:58:14.244400 13772 raft_consensus.cc:348] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:14.245193 13772 raft_consensus.cc:374] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:14.245505 13772 raft_consensus.cc:729] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:14.246304 13772 consensus_queue.cc:260] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:14.248974 13483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20ccdde9c1e3476485f50e8361e5ae0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:14.255702 13770 tablet_bootstrap.cc:492] T 20ccdde9c1e3476485f50e8361e5ae0b P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:14.264091 13773 tablet_bootstrap.cc:492] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:14.264542 13773 ts_tablet_manager.cc:1397] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.137s user 0.010s sys 0.001s
I20260925 09:58:14.267025 13773 raft_consensus.cc:348] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:14.267726 13773 raft_consensus.cc:374] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:14.271946 13770 tablet_bootstrap.cc:654] T 20ccdde9c1e3476485f50e8361e5ae0b P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:14.268008 13773 raft_consensus.cc:729] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:14.283792 13773 consensus_queue.cc:260] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:14.289582 13773 ts_tablet_manager.cc:1428] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.025s user 0.006s sys 0.000s
I20260925 09:58:14.290484 13773 tablet_bootstrap.cc:492] T 39854cd5a1834ade80ed37a7f181eb25 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:14.296087 13773 tablet_bootstrap.cc:654] T 39854cd5a1834ade80ed37a7f181eb25 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:14.300113 13770 tablet_bootstrap.cc:492] T 20ccdde9c1e3476485f50e8361e5ae0b P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:14.300525 13770 ts_tablet_manager.cc:1397] T 20ccdde9c1e3476485f50e8361e5ae0b P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.045s user 0.006s sys 0.004s
I20260925 09:58:14.312175 13770 raft_consensus.cc:348] T 20ccdde9c1e3476485f50e8361e5ae0b P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:14.313028 13770 raft_consensus.cc:374] T 20ccdde9c1e3476485f50e8361e5ae0b P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:14.313274 13770 raft_consensus.cc:729] T 20ccdde9c1e3476485f50e8361e5ae0b P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:14.315059 13073 master_service.cc:946] ReplaceTablet: received request to replace tablet b3b185b9ce9d4da0a8eb36cc6e7dce08 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:14.315132 13770 consensus_queue.cc:260] T 20ccdde9c1e3476485f50e8361e5ae0b P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:14.319911 13770 ts_tablet_manager.cc:1428] T 20ccdde9c1e3476485f50e8361e5ae0b P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.019s user 0.007s sys 0.000s
I20260925 09:58:14.325407 13773 tablet_bootstrap.cc:492] T 39854cd5a1834ade80ed37a7f181eb25 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:14.325830 13773 ts_tablet_manager.cc:1397] T 39854cd5a1834ade80ed37a7f181eb25 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.036s user 0.011s sys 0.012s
I20260925 09:58:14.330565 13773 raft_consensus.cc:348] T 39854cd5a1834ade80ed37a7f181eb25 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } }
I20260925 09:58:14.334101 13773 raft_consensus.cc:374] T 39854cd5a1834ade80ed37a7f181eb25 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:14.334390 13773 raft_consensus.cc:729] T 39854cd5a1834ade80ed37a7f181eb25 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:14.335031 13773 consensus_queue.cc:260] T 39854cd5a1834ade80ed37a7f181eb25 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } }
I20260925 09:58:14.339725 13773 ts_tablet_manager.cc:1428] T 39854cd5a1834ade80ed37a7f181eb25 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.014s user 0.008s sys 0.000s
I20260925 09:58:14.340618 13773 tablet_bootstrap.cc:492] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:14.356783 13773 tablet_bootstrap.cc:654] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:14.363499 13077 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet 2086e4e50cc84e2e9ecf8a44c194e8d4
I20260925 09:58:14.365401 13772 ts_tablet_manager.cc:1428] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.128s user 0.004s sys 0.001s
I20260925 09:58:14.369328 13650 tablet_service.cc:1514] Processing DeleteTablet for tablet 2086e4e50cc84e2e9ecf8a44c194e8d4 with delete_type TABLET_DATA_DELETED (replaced by tablet 20dd5e4109ff4e7ab838363bd1d0a12b) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:14.370065 13780 tablet_replica.cc:330] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:14.370811 13780 raft_consensus.cc:2229] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:14.371265 13780 raft_consensus.cc:2258] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:14.373438 13780 ts_tablet_manager.cc:1905] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:14.377481 13773 tablet_bootstrap.cc:492] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:14.377658 13073 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet b3b185b9ce9d4da0a8eb36cc6e7dce08
I20260925 09:58:14.377916 13773 ts_tablet_manager.cc:1397] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.037s user 0.009s sys 0.005s
I20260925 09:58:14.380050 13650 tablet_service.cc:1514] Processing DeleteTablet for tablet b3b185b9ce9d4da0a8eb36cc6e7dce08 with delete_type TABLET_DATA_DELETED (replaced by tablet 7c004b586c60494fae35ebdbe2308f4c) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:14.380438 13773 raft_consensus.cc:348] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:14.381109 13773 raft_consensus.cc:374] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:14.381351 13773 raft_consensus.cc:729] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:14.382061 13773 consensus_queue.cc:260] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:14.383702 13073 catalog_manager.cc:6565] ReplaceTablet: tablet b3b185b9ce9d4da0a8eb36cc6e7dce08 deleted and replaced by tablet 7c004b586c60494fae35ebdbe2308f4c
I20260925 09:58:14.384133 13773 ts_tablet_manager.cc:1428] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260925 09:58:14.393380 13773 tablet_bootstrap.cc:492] T 20ccdde9c1e3476485f50e8361e5ae0b P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:14.399336 13773 tablet_bootstrap.cc:654] T 20ccdde9c1e3476485f50e8361e5ae0b P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:14.401155 13483 tablet_service.cc:1514] Processing DeleteTablet for tablet 2086e4e50cc84e2e9ecf8a44c194e8d4 with delete_type TABLET_DATA_DELETED (replaced by tablet 20dd5e4109ff4e7ab838363bd1d0a12b) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:14.401889 13781 tablet_replica.cc:330] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:14.402622 13781 raft_consensus.cc:2229] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:14.403092 13781 raft_consensus.cc:2258] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:14.403316 13483 tablet_service.cc:1514] Processing DeleteTablet for tablet b3b185b9ce9d4da0a8eb36cc6e7dce08 with delete_type TABLET_DATA_DELETED (replaced by tablet 7c004b586c60494fae35ebdbe2308f4c) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:14.405735 13781 ts_tablet_manager.cc:1905] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:14.406126 13077 catalog_manager.cc:6565] ReplaceTablet: tablet 2086e4e50cc84e2e9ecf8a44c194e8d4 deleted and replaced by tablet 20dd5e4109ff4e7ab838363bd1d0a12b
I20260925 09:58:14.367319 13341 tablet_service.cc:1514] Processing DeleteTablet for tablet 2086e4e50cc84e2e9ecf8a44c194e8d4 with delete_type TABLET_DATA_DELETED (replaced by tablet 20dd5e4109ff4e7ab838363bd1d0a12b) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:14.383292 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet b3b185b9ce9d4da0a8eb36cc6e7dce08 with delete_type TABLET_DATA_DELETED (replaced by tablet 7c004b586c60494fae35ebdbe2308f4c) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:14.409477 13777 tablet_replica.cc:330] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:14.410169 13777 raft_consensus.cc:2229] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:14.410600 13777 raft_consensus.cc:2258] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:14.413226 13777 ts_tablet_manager.cc:1905] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:14.416160 13780 ts_tablet_manager.cc:1918] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:14.416540 13780 log.cc:1196] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/2086e4e50cc84e2e9ecf8a44c194e8d4
I20260925 09:58:14.417186 13780 ts_tablet_manager.cc:1939] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:14.419559 13780 tablet_replica.cc:330] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:14.420265 13780 raft_consensus.cc:2229] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:14.420708 13780 raft_consensus.cc:2258] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:14.423590 13773 tablet_bootstrap.cc:492] T 20ccdde9c1e3476485f50e8361e5ae0b P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:14.424011 13773 ts_tablet_manager.cc:1397] T 20ccdde9c1e3476485f50e8361e5ae0b P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.031s user 0.010s sys 0.007s
I20260925 09:58:14.426630 13773 raft_consensus.cc:348] T 20ccdde9c1e3476485f50e8361e5ae0b P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
W20260925 09:58:14.408622 13066 catalog_manager.cc:4821] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): delete failed for tablet 2086e4e50cc84e2e9ecf8a44c194e8d4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2086e4e50cc84e2e9ecf8a44c194e8d4 already in progress: creating tablet
I20260925 09:58:14.419728 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet 2086e4e50cc84e2e9ecf8a44c194e8d4 (table default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477]) successfully deleted
I20260925 09:58:14.427279 13773 raft_consensus.cc:374] T 20ccdde9c1e3476485f50e8361e5ae0b P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:14.430130 13773 raft_consensus.cc:729] T 20ccdde9c1e3476485f50e8361e5ae0b P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:14.430836 13773 consensus_queue.cc:260] T 20ccdde9c1e3476485f50e8361e5ae0b P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:14.433240 13781 ts_tablet_manager.cc:1918] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:14.433642 13781 log.cc:1196] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/2086e4e50cc84e2e9ecf8a44c194e8d4
I20260925 09:58:14.434324 13781 ts_tablet_manager.cc:1939] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:14.436399 13772 tablet_bootstrap.cc:492] T 20ccdde9c1e3476485f50e8361e5ae0b P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:14.433523 13773 ts_tablet_manager.cc:1428] T 20ccdde9c1e3476485f50e8361e5ae0b P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260925 09:58:14.455940 13781 tablet_replica.cc:330] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:14.463850 13781 raft_consensus.cc:2229] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:14.464356 13781 raft_consensus.cc:2258] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:14.466441 13781 ts_tablet_manager.cc:1905] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:14.471287 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet 2086e4e50cc84e2e9ecf8a44c194e8d4 (table default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477]) successfully deleted
I20260925 09:58:14.477834 13342 tablet_service.cc:1467] Processing CreateTablet for tablet 7c004b586c60494fae35ebdbe2308f4c (DEFAULT_TABLE table=default.table_7be114188e08442db683500d68eafdfd [id=07bcde2a154b4eb2b4ee526d680b5ceb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:14.478899 13342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c004b586c60494fae35ebdbe2308f4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:14.481465 13781 ts_tablet_manager.cc:1918] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:14.481817 13781 log.cc:1196] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/b3b185b9ce9d4da0a8eb36cc6e7dce08
I20260925 09:58:14.482453 13781 ts_tablet_manager.cc:1939] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:14.486348 13650 tablet_service.cc:1467] Processing CreateTablet for tablet 7c004b586c60494fae35ebdbe2308f4c (DEFAULT_TABLE table=default.table_7be114188e08442db683500d68eafdfd [id=07bcde2a154b4eb2b4ee526d680b5ceb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:14.487479 13650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c004b586c60494fae35ebdbe2308f4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:14.493183 13780 ts_tablet_manager.cc:1905] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:14.504798 13483 tablet_service.cc:1467] Processing CreateTablet for tablet 7c004b586c60494fae35ebdbe2308f4c (DEFAULT_TABLE table=default.table_7be114188e08442db683500d68eafdfd [id=07bcde2a154b4eb2b4ee526d680b5ceb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:14.505877 13483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c004b586c60494fae35ebdbe2308f4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:14.510581 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet b3b185b9ce9d4da0a8eb36cc6e7dce08 (table default.table_7be114188e08442db683500d68eafdfd [id=07bcde2a154b4eb2b4ee526d680b5ceb]) successfully deleted
I20260925 09:58:14.515151 13485 tablet_service.cc:1467] Processing CreateTablet for tablet 20dd5e4109ff4e7ab838363bd1d0a12b (DEFAULT_TABLE table=default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:14.516230 13485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20dd5e4109ff4e7ab838363bd1d0a12b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:14.519886 13649 tablet_service.cc:1467] Processing CreateTablet for tablet 20dd5e4109ff4e7ab838363bd1d0a12b (DEFAULT_TABLE table=default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:14.520953 13649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20dd5e4109ff4e7ab838363bd1d0a12b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:14.530978 13780 ts_tablet_manager.cc:1918] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:14.535743 13780 log.cc:1196] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/b3b185b9ce9d4da0a8eb36cc6e7dce08
I20260925 09:58:14.536520 13780 ts_tablet_manager.cc:1939] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:14.541417 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet b3b185b9ce9d4da0a8eb36cc6e7dce08 (table default.table_7be114188e08442db683500d68eafdfd [id=07bcde2a154b4eb2b4ee526d680b5ceb]) successfully deleted
I20260925 09:58:14.547524 13772 tablet_bootstrap.cc:654] T 20ccdde9c1e3476485f50e8361e5ae0b P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:14.548630 13770 tablet_bootstrap.cc:492] T 20dd5e4109ff4e7ab838363bd1d0a12b P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:14.553838 13770 tablet_bootstrap.cc:654] T 20dd5e4109ff4e7ab838363bd1d0a12b P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:14.558416 13343 tablet_service.cc:1467] Processing CreateTablet for tablet 20dd5e4109ff4e7ab838363bd1d0a12b (DEFAULT_TABLE table=default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:14.560192 13341 tablet_service.cc:1514] Processing DeleteTablet for tablet 2086e4e50cc84e2e9ecf8a44c194e8d4 with delete_type TABLET_DATA_DELETED (replaced by tablet 20dd5e4109ff4e7ab838363bd1d0a12b) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:14.564577 13773 tablet_bootstrap.cc:492] T 20dd5e4109ff4e7ab838363bd1d0a12b P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:14.565349 13343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20dd5e4109ff4e7ab838363bd1d0a12b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:14.569485 13773 tablet_bootstrap.cc:654] T 20dd5e4109ff4e7ab838363bd1d0a12b P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:14.580614 13782 raft_consensus.cc:482] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:14.581188 13782 raft_consensus.cc:504] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:14.583262 13782 leader_election.cc:288] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
W20260925 09:58:14.585047 13298 leader_election.cc:341] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: 2086e4e50cc84e2e9ecf8a44c194e8d4
W20260925 09:58:14.587580 13297 leader_election.cc:341] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Not found: Tablet not found: 2086e4e50cc84e2e9ecf8a44c194e8d4
I20260925 09:58:14.587961 13297 leader_election.cc:302] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:58:14.589080 13782 raft_consensus.cc:2726] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:58:14.592459 13777 ts_tablet_manager.cc:1918] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:14.593011 13777 log.cc:1196] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/b3b185b9ce9d4da0a8eb36cc6e7dce08
I20260925 09:58:14.593845 13777 ts_tablet_manager.cc:1939] T b3b185b9ce9d4da0a8eb36cc6e7dce08 P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:14.595216 13077 master_service.cc:946] ReplaceTablet: received request to replace tablet 20ccdde9c1e3476485f50e8361e5ae0b from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:14.600150 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet b3b185b9ce9d4da0a8eb36cc6e7dce08 (table default.table_7be114188e08442db683500d68eafdfd [id=07bcde2a154b4eb2b4ee526d680b5ceb]) successfully deleted
I20260925 09:58:14.612373 13773 tablet_bootstrap.cc:492] T 20dd5e4109ff4e7ab838363bd1d0a12b P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:14.612957 13773 ts_tablet_manager.cc:1397] T 20dd5e4109ff4e7ab838363bd1d0a12b P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.049s user 0.016s sys 0.000s
I20260925 09:58:14.615520 13773 raft_consensus.cc:348] T 20dd5e4109ff4e7ab838363bd1d0a12b P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:14.616264 13773 raft_consensus.cc:374] T 20dd5e4109ff4e7ab838363bd1d0a12b P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:14.616531 13773 raft_consensus.cc:729] T 20dd5e4109ff4e7ab838363bd1d0a12b P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:14.617182 13773 consensus_queue.cc:260] T 20dd5e4109ff4e7ab838363bd1d0a12b P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:14.619623 13773 ts_tablet_manager.cc:1428] T 20dd5e4109ff4e7ab838363bd1d0a12b P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260925 09:58:14.620434 13773 tablet_bootstrap.cc:492] T 7c004b586c60494fae35ebdbe2308f4c P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:14.625603 13788 raft_consensus.cc:482] T 39854cd5a1834ade80ed37a7f181eb25 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:14.626058 13788 raft_consensus.cc:504] T 39854cd5a1834ade80ed37a7f181eb25 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } }
I20260925 09:58:14.628093 13788 leader_election.cc:288] T 39854cd5a1834ade80ed37a7f181eb25 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731), b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161)
I20260925 09:58:14.629114 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39854cd5a1834ade80ed37a7f181eb25" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:14.629212 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39854cd5a1834ade80ed37a7f181eb25" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:14.629809 13671 raft_consensus.cc:2454] T 39854cd5a1834ade80ed37a7f181eb25 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 0.
I20260925 09:58:14.629922 13363 raft_consensus.cc:2454] T 39854cd5a1834ade80ed37a7f181eb25 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 0.
I20260925 09:58:14.631150 13440 leader_election.cc:302] T 39854cd5a1834ade80ed37a7f181eb25 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:14.632112 13773 tablet_bootstrap.cc:654] T 7c004b586c60494fae35ebdbe2308f4c P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:14.633829 13788 raft_consensus.cc:2781] T 39854cd5a1834ade80ed37a7f181eb25 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:14.634174 13788 raft_consensus.cc:482] T 39854cd5a1834ade80ed37a7f181eb25 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:14.634464 13788 raft_consensus.cc:3037] T 39854cd5a1834ade80ed37a7f181eb25 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:14.640936 13788 raft_consensus.cc:504] T 39854cd5a1834ade80ed37a7f181eb25 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } }
I20260925 09:58:14.642313 13073 master_service.cc:946] ReplaceTablet: received request to replace tablet 20dd5e4109ff4e7ab838363bd1d0a12b from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:14.643654 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39854cd5a1834ade80ed37a7f181eb25" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
I20260925 09:58:14.644207 13671 raft_consensus.cc:3037] T 39854cd5a1834ade80ed37a7f181eb25 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:14.648245 13077 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet 20ccdde9c1e3476485f50e8361e5ae0b
I20260925 09:58:14.650492 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet 20ccdde9c1e3476485f50e8361e5ae0b with delete_type TABLET_DATA_DELETED (replaced by tablet 7e4fe7762300435995b06413fa84abdf) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:14.652357 13077 catalog_manager.cc:6565] ReplaceTablet: tablet 20ccdde9c1e3476485f50e8361e5ae0b deleted and replaced by tablet 7e4fe7762300435995b06413fa84abdf
I20260925 09:58:14.653462 13650 tablet_service.cc:1514] Processing DeleteTablet for tablet 20ccdde9c1e3476485f50e8361e5ae0b with delete_type TABLET_DATA_DELETED (replaced by tablet 7e4fe7762300435995b06413fa84abdf) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:14.653573 13485 tablet_service.cc:1514] Processing DeleteTablet for tablet 20ccdde9c1e3476485f50e8361e5ae0b with delete_type TABLET_DATA_DELETED (replaced by tablet 7e4fe7762300435995b06413fa84abdf) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:14.654242 13781 tablet_replica.cc:330] T 20ccdde9c1e3476485f50e8361e5ae0b P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:14.654400 13780 tablet_replica.cc:330] T 20ccdde9c1e3476485f50e8361e5ae0b P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:14.654933 13781 raft_consensus.cc:2229] T 20ccdde9c1e3476485f50e8361e5ae0b P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:14.655089 13780 raft_consensus.cc:2229] T 20ccdde9c1e3476485f50e8361e5ae0b P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:14.655380 13781 raft_consensus.cc:2258] T 20ccdde9c1e3476485f50e8361e5ae0b P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:14.655596 13780 raft_consensus.cc:2258] T 20ccdde9c1e3476485f50e8361e5ae0b P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:14.657248 13770 tablet_bootstrap.cc:492] T 20dd5e4109ff4e7ab838363bd1d0a12b P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:14.657612 13770 ts_tablet_manager.cc:1397] T 20dd5e4109ff4e7ab838363bd1d0a12b P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.109s user 0.004s sys 0.003s
I20260925 09:58:14.657667 13781 ts_tablet_manager.cc:1905] T 20ccdde9c1e3476485f50e8361e5ae0b P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:14.659772 13770 raft_consensus.cc:348] T 20dd5e4109ff4e7ab838363bd1d0a12b P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:14.660392 13770 raft_consensus.cc:374] T 20dd5e4109ff4e7ab838363bd1d0a12b P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:14.660620 13770 raft_consensus.cc:729] T 20dd5e4109ff4e7ab838363bd1d0a12b P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:14.662940 13780 ts_tablet_manager.cc:1905] T 20ccdde9c1e3476485f50e8361e5ae0b P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:14.653909 13671 raft_consensus.cc:2454] T 39854cd5a1834ade80ed37a7f181eb25 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:14.670441 13440 leader_election.cc:302] T 39854cd5a1834ade80ed37a7f181eb25 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:14.671624 13770 consensus_queue.cc:260] T 20dd5e4109ff4e7ab838363bd1d0a12b P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:14.678467 13788 leader_election.cc:288] T 39854cd5a1834ade80ed37a7f181eb25 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Requested vote from peers a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731), b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161)
I20260925 09:58:14.678601 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39854cd5a1834ade80ed37a7f181eb25" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843"
I20260925 09:58:14.679189 13363 raft_consensus.cc:3037] T 39854cd5a1834ade80ed37a7f181eb25 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:14.679780 13788 raft_consensus.cc:2781] T 39854cd5a1834ade80ed37a7f181eb25 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:14.681317 13788 raft_consensus.cc:686] T 39854cd5a1834ade80ed37a7f181eb25 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: Becoming Leader. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Running, Role: LEADER
I20260925 09:58:14.683110 13788 consensus_queue.cc:237] T 39854cd5a1834ade80ed37a7f181eb25 P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } }
I20260925 09:58:14.690886 13363 raft_consensus.cc:2454] T 39854cd5a1834ade80ed37a7f181eb25 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:14.677708 13073 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet 20dd5e4109ff4e7ab838363bd1d0a12b
I20260925 09:58:14.692498 13770 ts_tablet_manager.cc:1428] T 20dd5e4109ff4e7ab838363bd1d0a12b P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.035s user 0.005s sys 0.001s
I20260925 09:58:14.693068 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet 20dd5e4109ff4e7ab838363bd1d0a12b with delete_type TABLET_DATA_DELETED (replaced by tablet 676f113a83ee4931b7652562f59034a8) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:14.693895 13485 tablet_service.cc:1514] Processing DeleteTablet for tablet 20dd5e4109ff4e7ab838363bd1d0a12b with delete_type TABLET_DATA_DELETED (replaced by tablet 676f113a83ee4931b7652562f59034a8) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:14.696849 13781 ts_tablet_manager.cc:1918] T 20ccdde9c1e3476485f50e8361e5ae0b P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:14.697229 13781 log.cc:1196] T 20ccdde9c1e3476485f50e8361e5ae0b P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/20ccdde9c1e3476485f50e8361e5ae0b
I20260925 09:58:14.697837 13781 ts_tablet_manager.cc:1939] T 20ccdde9c1e3476485f50e8361e5ae0b P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:14.699012 13073 catalog_manager.cc:6565] ReplaceTablet: tablet 20dd5e4109ff4e7ab838363bd1d0a12b deleted and replaced by tablet 676f113a83ee4931b7652562f59034a8
I20260925 09:58:14.700570 13650 tablet_service.cc:1514] Processing DeleteTablet for tablet 20dd5e4109ff4e7ab838363bd1d0a12b with delete_type TABLET_DATA_DELETED (replaced by tablet 676f113a83ee4931b7652562f59034a8) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:14.702888 13770 tablet_bootstrap.cc:492] T 7c004b586c60494fae35ebdbe2308f4c P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:14.703284 13772 tablet_bootstrap.cc:492] T 20ccdde9c1e3476485f50e8361e5ae0b P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:14.703696 13772 ts_tablet_manager.cc:1397] T 20ccdde9c1e3476485f50e8361e5ae0b P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.268s user 0.006s sys 0.004s
I20260925 09:58:14.705850 13772 raft_consensus.cc:348] T 20ccdde9c1e3476485f50e8361e5ae0b P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:14.706506 13772 raft_consensus.cc:374] T 20ccdde9c1e3476485f50e8361e5ae0b P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:14.706745 13772 raft_consensus.cc:729] T 20ccdde9c1e3476485f50e8361e5ae0b P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:14.707536 13773 tablet_bootstrap.cc:492] T 7c004b586c60494fae35ebdbe2308f4c P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:14.707937 13773 ts_tablet_manager.cc:1397] T 7c004b586c60494fae35ebdbe2308f4c P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.088s user 0.017s sys 0.010s
I20260925 09:58:14.712146 13777 tablet_replica.cc:330] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:14.713122 13777 raft_consensus.cc:2229] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:14.713704 13777 raft_consensus.cc:2258] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:14.717052 13781 tablet_replica.cc:330] T 20dd5e4109ff4e7ab838363bd1d0a12b P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:14.717151 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet 20ccdde9c1e3476485f50e8361e5ae0b (table default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]) successfully deleted
I20260925 09:58:14.717720 13781 raft_consensus.cc:2229] T 20dd5e4109ff4e7ab838363bd1d0a12b P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:14.718151 13781 raft_consensus.cc:2258] T 20dd5e4109ff4e7ab838363bd1d0a12b P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:14.719097 13773 raft_consensus.cc:348] T 7c004b586c60494fae35ebdbe2308f4c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:14.719808 13773 raft_consensus.cc:374] T 7c004b586c60494fae35ebdbe2308f4c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:14.720069 13773 raft_consensus.cc:729] T 7c004b586c60494fae35ebdbe2308f4c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:14.724123 13770 tablet_bootstrap.cc:654] T 7c004b586c60494fae35ebdbe2308f4c P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:14.728609 13780 ts_tablet_manager.cc:1918] T 20ccdde9c1e3476485f50e8361e5ae0b P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:14.729169 13780 log.cc:1196] T 20ccdde9c1e3476485f50e8361e5ae0b P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/20ccdde9c1e3476485f50e8361e5ae0b
I20260925 09:58:14.730235 13773 consensus_queue.cc:260] T 7c004b586c60494fae35ebdbe2308f4c P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:14.707353 13772 consensus_queue.cc:260] T 20ccdde9c1e3476485f50e8361e5ae0b P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:14.732725 13780 ts_tablet_manager.cc:1939] T 20ccdde9c1e3476485f50e8361e5ae0b P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:14.739064 13773 ts_tablet_manager.cc:1428] T 7c004b586c60494fae35ebdbe2308f4c P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.031s user 0.003s sys 0.007s
I20260925 09:58:14.745676 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet 20ccdde9c1e3476485f50e8361e5ae0b (table default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]) successfully deleted
I20260925 09:58:14.752276 13772 ts_tablet_manager.cc:1428] T 20ccdde9c1e3476485f50e8361e5ae0b P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.048s user 0.005s sys 0.000s
I20260925 09:58:14.764762 13781 ts_tablet_manager.cc:1905] T 20dd5e4109ff4e7ab838363bd1d0a12b P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:14.756098 13777 ts_tablet_manager.cc:1905] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:14.768234 13650 tablet_service.cc:1467] Processing CreateTablet for tablet 7e4fe7762300435995b06413fa84abdf (DEFAULT_TABLE table=default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:14.768260 13343 tablet_service.cc:1467] Processing CreateTablet for tablet 7e4fe7762300435995b06413fa84abdf (DEFAULT_TABLE table=default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:14.769294 13343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e4fe7762300435995b06413fa84abdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:14.769318 13650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e4fe7762300435995b06413fa84abdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:14.776224 13772 tablet_bootstrap.cc:492] T 7c004b586c60494fae35ebdbe2308f4c P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:14.784211 13780 tablet_replica.cc:330] T 20dd5e4109ff4e7ab838363bd1d0a12b P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:14.785125 13780 raft_consensus.cc:2229] T 20dd5e4109ff4e7ab838363bd1d0a12b P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:14.785704 13780 raft_consensus.cc:2258] T 20dd5e4109ff4e7ab838363bd1d0a12b P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:14.767561 13485 tablet_service.cc:1467] Processing CreateTablet for tablet 7e4fe7762300435995b06413fa84abdf (DEFAULT_TABLE table=default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:14.792894 13485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e4fe7762300435995b06413fa84abdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:14.751456 13077 catalog_manager.cc:5495] T 39854cd5a1834ade80ed37a7f181eb25 P c4d724d4b83e4d1e817a3a505dc24eb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2). New cstate: current_term: 1 leader_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: HEALTHY } } }
I20260925 09:58:14.808470 13773 tablet_bootstrap.cc:492] T 7e4fe7762300435995b06413fa84abdf P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:14.814488 13773 tablet_bootstrap.cc:654] T 7e4fe7762300435995b06413fa84abdf P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
W20260925 09:58:14.819145 13786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
I20260925 09:58:14.819379 13770 tablet_bootstrap.cc:492] T 7c004b586c60494fae35ebdbe2308f4c P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:14.819813 13770 ts_tablet_manager.cc:1397] T 7c004b586c60494fae35ebdbe2308f4c P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.117s user 0.011s sys 0.002s
W20260925 09:58:14.821141 13786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 09:58:14.821857 13786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260925 09:58:14.836364 13770 raft_consensus.cc:348] T 7c004b586c60494fae35ebdbe2308f4c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:14.837044 13770 raft_consensus.cc:374] T 7c004b586c60494fae35ebdbe2308f4c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:14.837292 13770 raft_consensus.cc:729] T 7c004b586c60494fae35ebdbe2308f4c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:14.837934 13770 consensus_queue.cc:260] T 7c004b586c60494fae35ebdbe2308f4c P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:14.848188 13773 tablet_bootstrap.cc:492] T 7e4fe7762300435995b06413fa84abdf P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:14.848634 13773 ts_tablet_manager.cc:1397] T 7e4fe7762300435995b06413fa84abdf P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.040s user 0.009s sys 0.000s
I20260925 09:58:14.852969 13773 raft_consensus.cc:348] T 7e4fe7762300435995b06413fa84abdf P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:14.853672 13773 raft_consensus.cc:374] T 7e4fe7762300435995b06413fa84abdf P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:14.853945 13773 raft_consensus.cc:729] T 7e4fe7762300435995b06413fa84abdf P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:14.854585 13773 consensus_queue.cc:260] T 7e4fe7762300435995b06413fa84abdf P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:14.858568 13772 tablet_bootstrap.cc:654] T 7c004b586c60494fae35ebdbe2308f4c P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:14.864679 13780 ts_tablet_manager.cc:1905] T 20dd5e4109ff4e7ab838363bd1d0a12b P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:14.873055 13773 ts_tablet_manager.cc:1428] T 7e4fe7762300435995b06413fa84abdf P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.024s user 0.006s sys 0.000s
I20260925 09:58:14.846164 13770 ts_tablet_manager.cc:1428] T 7c004b586c60494fae35ebdbe2308f4c P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.026s user 0.004s sys 0.001s
I20260925 09:58:14.887154 13770 tablet_bootstrap.cc:492] T 7e4fe7762300435995b06413fa84abdf P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
W20260925 09:58:14.897240 13786 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260925 09:58:14.897579 13786 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260925 09:58:14.897902 13786 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 09:58:14.898214 13786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 09:58:14.898537 13786 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 09:58:14.898833 13786 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 09:58:14.914011 13077 master_service.cc:946] ReplaceTablet: received request to replace tablet 20ccdde9c1e3476485f50e8361e5ae0b from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:14.915856 13077 master_service.cc:946] ReplaceTablet: received request to replace tablet 39854cd5a1834ade80ed37a7f181eb25 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:14.921890 13777 ts_tablet_manager.cc:1918] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:14.922312 13777 log.cc:1196] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/2086e4e50cc84e2e9ecf8a44c194e8d4
I20260925 09:58:14.922987 13777 ts_tablet_manager.cc:1939] T 2086e4e50cc84e2e9ecf8a44c194e8d4 P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:14.927330 13343 tablet_service.cc:1467] Processing CreateTablet for tablet 676f113a83ee4931b7652562f59034a8 (DEFAULT_TABLE table=default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:14.928807 13485 tablet_service.cc:1467] Processing CreateTablet for tablet 676f113a83ee4931b7652562f59034a8 (DEFAULT_TABLE table=default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:14.930034 13485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 676f113a83ee4931b7652562f59034a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:14.930058 13650 tablet_service.cc:1467] Processing CreateTablet for tablet 676f113a83ee4931b7652562f59034a8 (DEFAULT_TABLE table=default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:14.952721 13650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 676f113a83ee4931b7652562f59034a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:14.902187 13770 tablet_bootstrap.cc:654] T 7e4fe7762300435995b06413fa84abdf P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:14.960080 13772 tablet_bootstrap.cc:492] T 7c004b586c60494fae35ebdbe2308f4c P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:14.960515 13772 ts_tablet_manager.cc:1397] T 7c004b586c60494fae35ebdbe2308f4c P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.184s user 0.009s sys 0.000s
I20260925 09:58:14.962885 13786 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.9.20:35467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:42723
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.10.9.62:43437,127.10.9.61:40425,127.10.9.60:42083
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.9.60:42083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.9.60
--webserver_port=36655
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:06:08 UTC on ccycloud.quasar-lntfnh.root.comops.site
build id 4672078
TSAN enabled
I20260925 09:58:14.963647 13772 raft_consensus.cc:348] T 7c004b586c60494fae35ebdbe2308f4c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:14.964272 13772 raft_consensus.cc:374] T 7c004b586c60494fae35ebdbe2308f4c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:14.964481 13786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 09:58:14.964519 13772 raft_consensus.cc:729] T 7c004b586c60494fae35ebdbe2308f4c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:14.965059 13772 consensus_queue.cc:260] T 7c004b586c60494fae35ebdbe2308f4c P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:14.966598 13786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 09:58:14.967087 13772 ts_tablet_manager.cc:1428] T 7c004b586c60494fae35ebdbe2308f4c P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260925 09:58:14.970490 13077 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet 39854cd5a1834ade80ed37a7f181eb25
I20260925 09:58:14.967813 13772 tablet_bootstrap.cc:492] T 20dd5e4109ff4e7ab838363bd1d0a12b P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:14.974126 13779 raft_consensus.cc:482] T 7c004b586c60494fae35ebdbe2308f4c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:14.974542 13779 raft_consensus.cc:504] T 7c004b586c60494fae35ebdbe2308f4c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:14.976476 13779 leader_election.cc:288] T 7c004b586c60494fae35ebdbe2308f4c P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
I20260925 09:58:14.978283 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c004b586c60494fae35ebdbe2308f4c" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
I20260925 09:58:14.978330 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c004b586c60494fae35ebdbe2308f4c" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:14.978940 13363 raft_consensus.cc:2454] T 7c004b586c60494fae35ebdbe2308f4c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 0.
I20260925 09:58:14.978945 13505 raft_consensus.cc:2454] T 7c004b586c60494fae35ebdbe2308f4c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 0.
I20260925 09:58:14.980325 13606 leader_election.cc:302] T 7c004b586c60494fae35ebdbe2308f4c P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:14.982340 13649 tablet_service.cc:1514] Processing DeleteTablet for tablet 39854cd5a1834ade80ed37a7f181eb25 with delete_type TABLET_DATA_DELETED (replaced by tablet da32d2b6fdef415e9b704bc8b1eff532) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:14.984201 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet 39854cd5a1834ade80ed37a7f181eb25 with delete_type TABLET_DATA_DELETED (replaced by tablet da32d2b6fdef415e9b704bc8b1eff532) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:14.984810 13779 raft_consensus.cc:2781] T 7c004b586c60494fae35ebdbe2308f4c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:14.985165 13779 raft_consensus.cc:482] T 7c004b586c60494fae35ebdbe2308f4c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:14.985495 13779 raft_consensus.cc:3037] T 7c004b586c60494fae35ebdbe2308f4c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:14.991696 13779 raft_consensus.cc:504] T 7c004b586c60494fae35ebdbe2308f4c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:14.994930 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c004b586c60494fae35ebdbe2308f4c" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843"
I20260925 09:58:14.995590 13363 raft_consensus.cc:3037] T 7c004b586c60494fae35ebdbe2308f4c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:14.996919 13483 tablet_service.cc:1514] Processing DeleteTablet for tablet 39854cd5a1834ade80ed37a7f181eb25 with delete_type TABLET_DATA_DELETED (replaced by tablet da32d2b6fdef415e9b704bc8b1eff532) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:14.998817 13077 catalog_manager.cc:6565] ReplaceTablet: tablet 39854cd5a1834ade80ed37a7f181eb25 deleted and replaced by tablet da32d2b6fdef415e9b704bc8b1eff532
I20260925 09:58:15.001482 13082 catalog_manager.cc:3411] renamed table default.table_2bcc4dad40e04c2fad5d7739718f0e29 [id=2f774783b25a4b0e88449cd7cedaed2e] in HMS: new name default.table_fb4b236f93c04bb08319a6915fa8796c
I20260925 09:58:15.002075 13363 raft_consensus.cc:2454] T 7c004b586c60494fae35ebdbe2308f4c P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 1.
I20260925 09:58:15.003243 13603 leader_election.cc:302] T 7c004b586c60494fae35ebdbe2308f4c P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, b3082cf15dc74c0480e5d33ecf592843; no voters:
W20260925 09:58:15.011438 13827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:58:15.016839 13770 tablet_bootstrap.cc:492] T 7e4fe7762300435995b06413fa84abdf P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:15.021217 13343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 676f113a83ee4931b7652562f59034a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:15.032337 13342 tablet_service.cc:1467] Processing CreateTablet for tablet da32d2b6fdef415e9b704bc8b1eff532 (DEFAULT_TABLE table=default.table_2bcc4dad40e04c2fad5d7739718f0e29 [id=2f774783b25a4b0e88449cd7cedaed2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:15.033480 13342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da32d2b6fdef415e9b704bc8b1eff532. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:15.038584 13772 tablet_bootstrap.cc:654] T 20dd5e4109ff4e7ab838363bd1d0a12b P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:15.039054 13483 tablet_service.cc:1467] Processing CreateTablet for tablet da32d2b6fdef415e9b704bc8b1eff532 (DEFAULT_TABLE table=default.table_2bcc4dad40e04c2fad5d7739718f0e29 [id=2f774783b25a4b0e88449cd7cedaed2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:15.040166 13483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da32d2b6fdef415e9b704bc8b1eff532. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:15.055519 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c004b586c60494fae35ebdbe2308f4c" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9"
I20260925 09:58:15.056327 13505 raft_consensus.cc:3037] T 7c004b586c60494fae35ebdbe2308f4c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:58:15.059451 13269 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet da32d2b6fdef415e9b704bc8b1eff532 consensus state has no leader
I20260925 09:58:15.061008 13777 tablet_replica.cc:330] T 20ccdde9c1e3476485f50e8361e5ae0b P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:15.061865 13777 raft_consensus.cc:2229] T 20ccdde9c1e3476485f50e8361e5ae0b P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:15.062160 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet 2086e4e50cc84e2e9ecf8a44c194e8d4 (table default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477]) successfully deleted
I20260925 09:58:15.063275 13777 raft_consensus.cc:2258] T 20ccdde9c1e3476485f50e8361e5ae0b P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:15.063490 13505 raft_consensus.cc:2454] T 7c004b586c60494fae35ebdbe2308f4c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 1.
I20260925 09:58:15.064042 13770 ts_tablet_manager.cc:1397] T 7e4fe7762300435995b06413fa84abdf P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.177s user 0.008s sys 0.003s
I20260925 09:58:15.066311 13770 raft_consensus.cc:348] T 7e4fe7762300435995b06413fa84abdf P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:15.010195 13781 ts_tablet_manager.cc:1918] T 20dd5e4109ff4e7ab838363bd1d0a12b P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:15.067265 13781 log.cc:1196] T 20dd5e4109ff4e7ab838363bd1d0a12b P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/20dd5e4109ff4e7ab838363bd1d0a12b
I20260925 09:58:15.067978 13781 ts_tablet_manager.cc:1939] T 20dd5e4109ff4e7ab838363bd1d0a12b P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:15.078892 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet 20dd5e4109ff4e7ab838363bd1d0a12b (table default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477]) successfully deleted
I20260925 09:58:15.080168 13781 tablet_replica.cc:330] T 39854cd5a1834ade80ed37a7f181eb25 P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:15.080876 13781 raft_consensus.cc:2229] T 39854cd5a1834ade80ed37a7f181eb25 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: Raft consensus shutting down.
W20260925 09:58:15.087240 13078 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_3823faf045944cc1a828ce929b8a7275 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 09:58:15.088346 13074 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_4c3437905d9e4b05ac1fb4c6e3d0a417 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 09:58:15.087283 13075 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_a29aa5b33fd644a2ab80242e289ed25d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:58:15.066954 13770 raft_consensus.cc:374] T 7e4fe7762300435995b06413fa84abdf P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:15.093392 13770 raft_consensus.cc:729] T 7e4fe7762300435995b06413fa84abdf P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:15.094591 13770 consensus_queue.cc:260] T 7e4fe7762300435995b06413fa84abdf P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:15.096015 13777 ts_tablet_manager.cc:1905] T 20ccdde9c1e3476485f50e8361e5ae0b P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:15.087982 13773 tablet_bootstrap.cc:492] T 676f113a83ee4931b7652562f59034a8 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:15.016988 13780 ts_tablet_manager.cc:1918] T 20dd5e4109ff4e7ab838363bd1d0a12b P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:15.101991 13780 log.cc:1196] T 20dd5e4109ff4e7ab838363bd1d0a12b P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/20dd5e4109ff4e7ab838363bd1d0a12b
I20260925 09:58:15.102748 13780 ts_tablet_manager.cc:1939] T 20dd5e4109ff4e7ab838363bd1d0a12b P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:15.104624 13773 tablet_bootstrap.cc:654] T 676f113a83ee4931b7652562f59034a8 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:15.057807 13829 raft_consensus.cc:2781] T 7c004b586c60494fae35ebdbe2308f4c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election won for term 1
W20260925 09:58:15.058233 13603 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.1:42161, user_credentials={real_user=slave}} blocked reactor thread for 55202.8us
I20260925 09:58:15.108299 13649 tablet_service.cc:1467] Processing CreateTablet for tablet da32d2b6fdef415e9b704bc8b1eff532 (DEFAULT_TABLE table=default.table_2bcc4dad40e04c2fad5d7739718f0e29 [id=2f774783b25a4b0e88449cd7cedaed2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:15.109321 13649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da32d2b6fdef415e9b704bc8b1eff532. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:15.109815 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet 20dd5e4109ff4e7ab838363bd1d0a12b (table default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477]) successfully deleted
I20260925 09:58:15.112468 13780 tablet_replica.cc:330] T 39854cd5a1834ade80ed37a7f181eb25 P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:15.113142 13780 raft_consensus.cc:2229] T 39854cd5a1834ade80ed37a7f181eb25 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:15.113597 13780 raft_consensus.cc:2258] T 39854cd5a1834ade80ed37a7f181eb25 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:15.113888 13772 tablet_bootstrap.cc:492] T 20dd5e4109ff4e7ab838363bd1d0a12b P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:15.114599 13772 ts_tablet_manager.cc:1397] T 20dd5e4109ff4e7ab838363bd1d0a12b P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.147s user 0.016s sys 0.003s
W20260925 09:58:15.115865 13068 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet da32d2b6fdef415e9b704bc8b1eff532 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet da32d2b6fdef415e9b704bc8b1eff532 consensus state has no leader
I20260925 09:58:15.117894 13772 raft_consensus.cc:348] T 20dd5e4109ff4e7ab838363bd1d0a12b P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:15.058670 13779 leader_election.cc:288] T 7c004b586c60494fae35ebdbe2308f4c P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 election: Requested vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
I20260925 09:58:15.118834 13772 raft_consensus.cc:374] T 20dd5e4109ff4e7ab838363bd1d0a12b P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:15.119177 13772 raft_consensus.cc:729] T 20dd5e4109ff4e7ab838363bd1d0a12b P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:15.124541 13780 ts_tablet_manager.cc:1905] T 39854cd5a1834ade80ed37a7f181eb25 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:15.138898 13773 tablet_bootstrap.cc:492] T 676f113a83ee4931b7652562f59034a8 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:15.081650 13781 pending_rounds.cc:62] T 39854cd5a1834ade80ed37a7f181eb25 P c4d724d4b83e4d1e817a3a505dc24eb9: Trying to abort 1 pending ops.
I20260925 09:58:15.144769 13780 ts_tablet_manager.cc:1918] T 39854cd5a1834ade80ed37a7f181eb25 P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:15.145112 13780 log.cc:1196] T 39854cd5a1834ade80ed37a7f181eb25 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/39854cd5a1834ade80ed37a7f181eb25
I20260925 09:58:15.145701 13780 ts_tablet_manager.cc:1939] T 39854cd5a1834ade80ed37a7f181eb25 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:15.146154 13073 master_service.cc:946] ReplaceTablet: received request to replace tablet 39854cd5a1834ade80ed37a7f181eb25 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:15.147930 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet 39854cd5a1834ade80ed37a7f181eb25 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]) successfully deleted
I20260925 09:58:15.142510 13773 ts_tablet_manager.cc:1397] T 676f113a83ee4931b7652562f59034a8 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.052s user 0.010s sys 0.000s
I20260925 09:58:15.148578 13772 consensus_queue.cc:260] T 20dd5e4109ff4e7ab838363bd1d0a12b P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:15.150862 13772 ts_tablet_manager.cc:1428] T 20dd5e4109ff4e7ab838363bd1d0a12b P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.036s user 0.003s sys 0.003s
I20260925 09:58:15.151612 13772 tablet_bootstrap.cc:492] T 7e4fe7762300435995b06413fa84abdf P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:15.151798 13773 raft_consensus.cc:348] T 676f113a83ee4931b7652562f59034a8 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:15.152530 13773 raft_consensus.cc:374] T 676f113a83ee4931b7652562f59034a8 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:15.155650 13773 raft_consensus.cc:729] T 676f113a83ee4931b7652562f59034a8 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:15.156283 13773 consensus_queue.cc:260] T 676f113a83ee4931b7652562f59034a8 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:15.160315 13773 ts_tablet_manager.cc:1428] T 676f113a83ee4931b7652562f59034a8 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
W20260925 09:58:15.168404 13073 catalog_manager.cc:5328] Ignoring report from unknown tablet 20dd5e4109ff4e7ab838363bd1d0a12b
I20260925 09:58:15.160646 13781 pending_rounds.cc:68] T 39854cd5a1834ade80ed37a7f181eb25 P c4d724d4b83e4d1e817a3a505dc24eb9: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192887061979136 op_type: NO_OP noop_request { }
I20260925 09:58:15.170176 13781 raft_consensus.cc:2866] T 39854cd5a1834ade80ed37a7f181eb25 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:58:15.170562 13781 raft_consensus.cc:2258] T 39854cd5a1834ade80ed37a7f181eb25 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 09:58:15.174050 13077 catalog_manager.cc:5328] Ignoring report from unknown tablet 20dd5e4109ff4e7ab838363bd1d0a12b
I20260925 09:58:15.174651 13773 tablet_bootstrap.cc:492] T da32d2b6fdef415e9b704bc8b1eff532 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:15.179209 13781 ts_tablet_manager.cc:1905] T 39854cd5a1834ade80ed37a7f181eb25 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:15.184769 13772 tablet_bootstrap.cc:654] T 7e4fe7762300435995b06413fa84abdf P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:15.194480 13773 tablet_bootstrap.cc:654] T da32d2b6fdef415e9b704bc8b1eff532 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:15.200663 13829 raft_consensus.cc:686] T 7c004b586c60494fae35ebdbe2308f4c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 LEADER]: Becoming Leader. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Running, Role: LEADER
I20260925 09:58:15.201742 13829 consensus_queue.cc:237] T 7c004b586c60494fae35ebdbe2308f4c P a06c3ef7a6a94f7cbcc03e7cd232f237 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:15.210361 13772 tablet_bootstrap.cc:492] T 7e4fe7762300435995b06413fa84abdf P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:15.210753 13772 ts_tablet_manager.cc:1397] T 7e4fe7762300435995b06413fa84abdf P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.059s user 0.006s sys 0.002s
I20260925 09:58:15.211778 13073 master_service.cc:946] ReplaceTablet: received request to replace tablet 676f113a83ee4931b7652562f59034a8 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:15.213158 13772 raft_consensus.cc:348] T 7e4fe7762300435995b06413fa84abdf P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:15.213796 13772 raft_consensus.cc:374] T 7e4fe7762300435995b06413fa84abdf P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:15.214056 13772 raft_consensus.cc:729] T 7e4fe7762300435995b06413fa84abdf P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:15.214617 13772 consensus_queue.cc:260] T 7e4fe7762300435995b06413fa84abdf P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:15.229267 13781 ts_tablet_manager.cc:1918] T 39854cd5a1834ade80ed37a7f181eb25 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:15.229674 13781 log.cc:1196] T 39854cd5a1834ade80ed37a7f181eb25 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/39854cd5a1834ade80ed37a7f181eb25
I20260925 09:58:15.232913 13772 ts_tablet_manager.cc:1428] T 7e4fe7762300435995b06413fa84abdf P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.022s user 0.000s sys 0.004s
I20260925 09:58:15.233736 13772 tablet_bootstrap.cc:492] T 676f113a83ee4931b7652562f59034a8 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:15.235155 13782 raft_consensus.cc:482] T 20dd5e4109ff4e7ab838363bd1d0a12b P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:15.235620 13782 raft_consensus.cc:504] T 20dd5e4109ff4e7ab838363bd1d0a12b P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:15.237809 13782 leader_election.cc:288] T 20dd5e4109ff4e7ab838363bd1d0a12b P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:15.238456 13772 tablet_bootstrap.cc:654] T 676f113a83ee4931b7652562f59034a8 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
W20260925 09:58:15.247062 13297 leader_election.cc:341] T 20dd5e4109ff4e7ab838363bd1d0a12b P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Not found: Tablet not found: 20dd5e4109ff4e7ab838363bd1d0a12b
I20260925 09:58:15.248461 13781 ts_tablet_manager.cc:1939] T 39854cd5a1834ade80ed37a7f181eb25 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
W20260925 09:58:15.248319 13298 leader_election.cc:341] T 20dd5e4109ff4e7ab838363bd1d0a12b P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: 20dd5e4109ff4e7ab838363bd1d0a12b
I20260925 09:58:15.248822 13298 leader_election.cc:302] T 20dd5e4109ff4e7ab838363bd1d0a12b P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:58:15.256966 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet 39854cd5a1834ade80ed37a7f181eb25 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]) successfully deleted
I20260925 09:58:15.258850 13782 raft_consensus.cc:2726] T 20dd5e4109ff4e7ab838363bd1d0a12b P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:58:15.269784 13067 consensus_peers.cc:487] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 -> Peer 61e29afab0cf46819cbffc91248bf761 (127.10.9.60:42083): Couldn't send request to peer 61e29afab0cf46819cbffc91248bf761. Status: Network error: Client connection negotiation failed: client connection to 127.10.9.60:42083: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260925 09:58:15.273988 13770 ts_tablet_manager.cc:1428] T 7e4fe7762300435995b06413fa84abdf P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.210s user 0.007s sys 0.000s
I20260925 09:58:15.274451 13073 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet 676f113a83ee4931b7652562f59034a8
I20260925 09:58:15.276226 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet 676f113a83ee4931b7652562f59034a8 with delete_type TABLET_DATA_DELETED (replaced by tablet 22c600a879ed420b931f782012e41162) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:15.277352 13772 tablet_bootstrap.cc:492] T 676f113a83ee4931b7652562f59034a8 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:15.277962 13772 ts_tablet_manager.cc:1397] T 676f113a83ee4931b7652562f59034a8 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.044s user 0.006s sys 0.004s
I20260925 09:58:15.278622 13650 tablet_service.cc:1514] Processing DeleteTablet for tablet 676f113a83ee4931b7652562f59034a8 with delete_type TABLET_DATA_DELETED (replaced by tablet 22c600a879ed420b931f782012e41162) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:15.279264 13483 tablet_service.cc:1514] Processing DeleteTablet for tablet 676f113a83ee4931b7652562f59034a8 with delete_type TABLET_DATA_DELETED (replaced by tablet 22c600a879ed420b931f782012e41162) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:15.279433 13770 tablet_bootstrap.cc:492] T 676f113a83ee4931b7652562f59034a8 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:15.281164 13781 tablet_replica.cc:330] T 676f113a83ee4931b7652562f59034a8 P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
W20260925 09:58:15.280970 13068 catalog_manager.cc:4821] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): delete failed for tablet 676f113a83ee4931b7652562f59034a8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 676f113a83ee4931b7652562f59034a8 already in progress: creating tablet
I20260925 09:58:15.281931 13781 raft_consensus.cc:2229] T 676f113a83ee4931b7652562f59034a8 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:15.282516 13781 raft_consensus.cc:2258] T 676f113a83ee4931b7652562f59034a8 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:15.284030 13772 raft_consensus.cc:348] T 676f113a83ee4931b7652562f59034a8 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:15.285554 13781 ts_tablet_manager.cc:1905] T 676f113a83ee4931b7652562f59034a8 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:15.286005 13772 raft_consensus.cc:374] T 676f113a83ee4931b7652562f59034a8 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:15.288383 13772 raft_consensus.cc:729] T 676f113a83ee4931b7652562f59034a8 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:15.295814 13772 consensus_queue.cc:260] T 676f113a83ee4931b7652562f59034a8 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:15.298069 13772 ts_tablet_manager.cc:1428] T 676f113a83ee4931b7652562f59034a8 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.020s user 0.005s sys 0.000s
I20260925 09:58:15.299057 13772 tablet_bootstrap.cc:492] T da32d2b6fdef415e9b704bc8b1eff532 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:15.301333 13073 catalog_manager.cc:6565] ReplaceTablet: tablet 676f113a83ee4931b7652562f59034a8 deleted and replaced by tablet 22c600a879ed420b931f782012e41162
I20260925 09:58:15.304114 13772 tablet_bootstrap.cc:654] T da32d2b6fdef415e9b704bc8b1eff532 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:15.318362 13342 tablet_service.cc:1467] Processing CreateTablet for tablet 22c600a879ed420b931f782012e41162 (DEFAULT_TABLE table=default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:15.319443 13342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22c600a879ed420b931f782012e41162. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:15.324523 13772 tablet_bootstrap.cc:492] T da32d2b6fdef415e9b704bc8b1eff532 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:15.325004 13772 ts_tablet_manager.cc:1397] T da32d2b6fdef415e9b704bc8b1eff532 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.026s user 0.011s sys 0.001s
I20260925 09:58:15.326372 13483 tablet_service.cc:1467] Processing CreateTablet for tablet 22c600a879ed420b931f782012e41162 (DEFAULT_TABLE table=default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:15.327503 13483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22c600a879ed420b931f782012e41162. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:15.327333 13772 raft_consensus.cc:348] T da32d2b6fdef415e9b704bc8b1eff532 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:15.328080 13772 raft_consensus.cc:374] T da32d2b6fdef415e9b704bc8b1eff532 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:15.328377 13772 raft_consensus.cc:729] T da32d2b6fdef415e9b704bc8b1eff532 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:15.331122 13773 tablet_bootstrap.cc:492] T da32d2b6fdef415e9b704bc8b1eff532 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:15.331532 13773 ts_tablet_manager.cc:1397] T da32d2b6fdef415e9b704bc8b1eff532 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.157s user 0.014s sys 0.020s
I20260925 09:58:15.330888 13772 consensus_queue.cc:260] T da32d2b6fdef415e9b704bc8b1eff532 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:15.333335 13772 ts_tablet_manager.cc:1428] T da32d2b6fdef415e9b704bc8b1eff532 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.008s user 0.004s sys 0.000s
I20260925 09:58:15.333739 13773 raft_consensus.cc:348] T da32d2b6fdef415e9b704bc8b1eff532 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:15.334394 13773 raft_consensus.cc:374] T da32d2b6fdef415e9b704bc8b1eff532 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:15.334640 13773 raft_consensus.cc:729] T da32d2b6fdef415e9b704bc8b1eff532 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:15.335913 13773 consensus_queue.cc:260] T da32d2b6fdef415e9b704bc8b1eff532 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:15.338052 13773 ts_tablet_manager.cc:1428] T da32d2b6fdef415e9b704bc8b1eff532 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.006s user 0.004s sys 0.001s
I20260925 09:58:15.342402 13650 tablet_service.cc:1467] Processing CreateTablet for tablet 22c600a879ed420b931f782012e41162 (DEFAULT_TABLE table=default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:15.343537 13650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22c600a879ed420b931f782012e41162. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:15.360705 13781 ts_tablet_manager.cc:1918] T 676f113a83ee4931b7652562f59034a8 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:15.361232 13781 log.cc:1196] T 676f113a83ee4931b7652562f59034a8 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/676f113a83ee4931b7652562f59034a8
I20260925 09:58:15.362136 13781 ts_tablet_manager.cc:1939] T 676f113a83ee4931b7652562f59034a8 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:15.364934 13073 catalog_manager.cc:5587] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327) does not have the latest schema for tablet da32d2b6fdef415e9b704bc8b1eff532 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]). Expected version 1 got 0
I20260925 09:58:15.369377 13073 catalog_manager.cc:5587] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161) does not have the latest schema for tablet da32d2b6fdef415e9b704bc8b1eff532 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]). Expected version 1 got 0
I20260925 09:58:15.377465 13073 catalog_manager.cc:5587] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327) does not have the latest schema for tablet da32d2b6fdef415e9b704bc8b1eff532 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]). Expected version 1 got 0
I20260925 09:58:15.379851 13073 master_service.cc:946] ReplaceTablet: received request to replace tablet 39854cd5a1834ade80ed37a7f181eb25 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:15.383008 13077 catalog_manager.cc:5495] T 7c004b586c60494fae35ebdbe2308f4c P a06c3ef7a6a94f7cbcc03e7cd232f237 reported cstate change: term changed from 0 to 1, leader changed from <none> to a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3). New cstate: current_term: 1 leader_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: HEALTHY } } }
I20260925 09:58:15.389475 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet 676f113a83ee4931b7652562f59034a8 (table default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477]) successfully deleted
I20260925 09:58:15.391770 13773 tablet_bootstrap.cc:492] T 22c600a879ed420b931f782012e41162 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:15.396729 13773 tablet_bootstrap.cc:654] T 22c600a879ed420b931f782012e41162 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:15.413465 13782 raft_consensus.cc:482] T 7e4fe7762300435995b06413fa84abdf P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:15.413932 13782 raft_consensus.cc:504] T 7e4fe7762300435995b06413fa84abdf P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:15.415769 13782 leader_election.cc:288] T 7e4fe7762300435995b06413fa84abdf P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:15.417305 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e4fe7762300435995b06413fa84abdf" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:15.417924 13671 raft_consensus.cc:2454] T 7e4fe7762300435995b06413fa84abdf P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 0.
I20260925 09:58:15.419706 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e4fe7762300435995b06413fa84abdf" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
I20260925 09:58:15.420305 13505 raft_consensus.cc:2454] T 7e4fe7762300435995b06413fa84abdf P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 0.
I20260925 09:58:15.421558 13297 leader_election.cc:302] T 7e4fe7762300435995b06413fa84abdf P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, b3082cf15dc74c0480e5d33ecf592843; no voters:
I20260925 09:58:15.423177 13782 raft_consensus.cc:2781] T 7e4fe7762300435995b06413fa84abdf P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:15.423566 13782 raft_consensus.cc:482] T 7e4fe7762300435995b06413fa84abdf P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:15.423949 13782 raft_consensus.cc:3037] T 7e4fe7762300435995b06413fa84abdf P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:15.427053 13770 tablet_bootstrap.cc:654] T 676f113a83ee4931b7652562f59034a8 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:15.430207 13079 catalog_manager.cc:3411] renamed table default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477] in HMS: new name default.table_832367e542164f49b92f562d7a77699f
I20260925 09:58:15.430147 13782 raft_consensus.cc:504] T 7e4fe7762300435995b06413fa84abdf P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:15.438730 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e4fe7762300435995b06413fa84abdf" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9"
I20260925 09:58:15.439293 13505 raft_consensus.cc:3037] T 7e4fe7762300435995b06413fa84abdf P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:15.445746 13505 raft_consensus.cc:2454] T 7e4fe7762300435995b06413fa84abdf P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 1.
I20260925 09:58:15.446944 13298 leader_election.cc:302] T 7e4fe7762300435995b06413fa84abdf P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:15.454630 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e4fe7762300435995b06413fa84abdf" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
I20260925 09:58:15.457636 13671 raft_consensus.cc:3037] T 7e4fe7762300435995b06413fa84abdf P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:15.463500 13772 tablet_bootstrap.cc:492] T 22c600a879ed420b931f782012e41162 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:15.472910 13671 raft_consensus.cc:2454] T 7e4fe7762300435995b06413fa84abdf P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 1.
I20260925 09:58:15.474558 13772 tablet_bootstrap.cc:654] T 22c600a879ed420b931f782012e41162 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:15.474468 13788 raft_consensus.cc:482] T da32d2b6fdef415e9b704bc8b1eff532 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:15.475066 13788 raft_consensus.cc:504] T da32d2b6fdef415e9b704bc8b1eff532 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:15.477535 13788 leader_election.cc:288] T da32d2b6fdef415e9b704bc8b1eff532 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:15.479298 13782 leader_election.cc:288] T 7e4fe7762300435995b06413fa84abdf P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Requested vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:15.479943 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da32d2b6fdef415e9b704bc8b1eff532" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
W20260925 09:58:15.481197 13440 leader_election.cc:341] T da32d2b6fdef415e9b704bc8b1eff532 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:15.479743 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da32d2b6fdef415e9b704bc8b1eff532" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:15.482409 13363 raft_consensus.cc:2454] T da32d2b6fdef415e9b704bc8b1eff532 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 0.
I20260925 09:58:15.484216 13438 leader_election.cc:302] T da32d2b6fdef415e9b704bc8b1eff532 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237
I20260925 09:58:15.486213 13788 raft_consensus.cc:2781] T da32d2b6fdef415e9b704bc8b1eff532 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:15.486516 13788 raft_consensus.cc:482] T da32d2b6fdef415e9b704bc8b1eff532 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:15.486790 13788 raft_consensus.cc:3037] T da32d2b6fdef415e9b704bc8b1eff532 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:15.487015 13777 ts_tablet_manager.cc:1918] T 20ccdde9c1e3476485f50e8361e5ae0b P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:15.487381 13773 tablet_bootstrap.cc:492] T 22c600a879ed420b931f782012e41162 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:15.487366 13777 log.cc:1196] T 20ccdde9c1e3476485f50e8361e5ae0b P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/20ccdde9c1e3476485f50e8361e5ae0b
I20260925 09:58:15.487777 13773 ts_tablet_manager.cc:1397] T 22c600a879ed420b931f782012e41162 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.096s user 0.021s sys 0.061s
I20260925 09:58:15.488029 13777 ts_tablet_manager.cc:1939] T 20ccdde9c1e3476485f50e8361e5ae0b P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:15.490352 13777 tablet_replica.cc:330] T 20dd5e4109ff4e7ab838363bd1d0a12b P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:15.490111 13773 raft_consensus.cc:348] T 22c600a879ed420b931f782012e41162 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:15.490569 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet 20ccdde9c1e3476485f50e8361e5ae0b (table default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]) successfully deleted
I20260925 09:58:15.490739 13773 raft_consensus.cc:374] T 22c600a879ed420b931f782012e41162 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:15.490993 13773 raft_consensus.cc:729] T 22c600a879ed420b931f782012e41162 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:15.491117 13777 raft_consensus.cc:2229] T 20dd5e4109ff4e7ab838363bd1d0a12b P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:15.492417 13773 consensus_queue.cc:260] T 22c600a879ed420b931f782012e41162 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:15.494352 13788 raft_consensus.cc:504] T da32d2b6fdef415e9b704bc8b1eff532 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:15.497084 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da32d2b6fdef415e9b704bc8b1eff532" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843"
I20260925 09:58:15.497592 13363 raft_consensus.cc:3037] T da32d2b6fdef415e9b704bc8b1eff532 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:15.501634 13773 ts_tablet_manager.cc:1428] T 22c600a879ed420b931f782012e41162 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.014s user 0.004s sys 0.007s
I20260925 09:58:15.509032 13363 raft_consensus.cc:2454] T da32d2b6fdef415e9b704bc8b1eff532 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:15.510149 13438 leader_election.cc:302] T da32d2b6fdef415e9b704bc8b1eff532 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:15.480281 13772 tablet_bootstrap.cc:492] T 22c600a879ed420b931f782012e41162 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:15.513634 13772 ts_tablet_manager.cc:1397] T 22c600a879ed420b931f782012e41162 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.050s user 0.008s sys 0.002s
I20260925 09:58:15.516126 13772 raft_consensus.cc:348] T 22c600a879ed420b931f782012e41162 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:15.518039 13847 raft_consensus.cc:2781] T da32d2b6fdef415e9b704bc8b1eff532 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:15.519425 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da32d2b6fdef415e9b704bc8b1eff532" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
I20260925 09:58:15.519594 13847 raft_consensus.cc:686] T da32d2b6fdef415e9b704bc8b1eff532 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: Becoming Leader. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Running, Role: LEADER
W20260925 09:58:15.520648 13440 leader_election.cc:341] T da32d2b6fdef415e9b704bc8b1eff532 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:15.520452 13847 consensus_queue.cc:237] T da32d2b6fdef415e9b704bc8b1eff532 P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:15.524370 13788 leader_election.cc:288] T da32d2b6fdef415e9b704bc8b1eff532 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Requested vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:15.525185 13772 raft_consensus.cc:374] T 22c600a879ed420b931f782012e41162 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:15.519722 13777 raft_consensus.cc:2258] T 20dd5e4109ff4e7ab838363bd1d0a12b P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:15.516997 13782 raft_consensus.cc:2781] T 7e4fe7762300435995b06413fa84abdf P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:15.526870 13782 raft_consensus.cc:686] T 7e4fe7762300435995b06413fa84abdf P b3082cf15dc74c0480e5d33ecf592843 [term 1 LEADER]: Becoming Leader. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Running, Role: LEADER
I20260925 09:58:15.527802 13782 consensus_queue.cc:237] T 7e4fe7762300435995b06413fa84abdf P b3082cf15dc74c0480e5d33ecf592843 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:15.528282 13777 ts_tablet_manager.cc:1905] T 20dd5e4109ff4e7ab838363bd1d0a12b P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 09:58:15.533594 13298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.2:44327, user_credentials={real_user=slave}} blocked reactor thread for 86745.8us
I20260925 09:58:15.538928 13770 tablet_bootstrap.cc:492] T 676f113a83ee4931b7652562f59034a8 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:15.539469 13770 ts_tablet_manager.cc:1397] T 676f113a83ee4931b7652562f59034a8 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.260s user 0.014s sys 0.004s
I20260925 09:58:15.541559 13770 raft_consensus.cc:348] T 676f113a83ee4931b7652562f59034a8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:15.542201 13770 raft_consensus.cc:374] T 676f113a83ee4931b7652562f59034a8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:15.542438 13770 raft_consensus.cc:729] T 676f113a83ee4931b7652562f59034a8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:15.528643 13772 raft_consensus.cc:729] T 22c600a879ed420b931f782012e41162 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:15.543004 13770 consensus_queue.cc:260] T 676f113a83ee4931b7652562f59034a8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:15.543906 13772 consensus_queue.cc:260] T 22c600a879ed420b931f782012e41162 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:15.556939 13772 ts_tablet_manager.cc:1428] T 22c600a879ed420b931f782012e41162 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.043s user 0.006s sys 0.000s
I20260925 09:58:15.562983 13077 catalog_manager.cc:5495] T da32d2b6fdef415e9b704bc8b1eff532 P c4d724d4b83e4d1e817a3a505dc24eb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2). New cstate: current_term: 1 leader_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: UNKNOWN } } }
I20260925 09:58:15.565681 13077 catalog_manager.cc:5587] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327) does not have the latest schema for tablet da32d2b6fdef415e9b704bc8b1eff532 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]). Expected version 1 got 0
I20260925 09:58:15.588196 13847 raft_consensus.cc:482] T 22c600a879ed420b931f782012e41162 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:15.588637 13847 raft_consensus.cc:504] T 22c600a879ed420b931f782012e41162 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:15.595500 13847 leader_election.cc:288] T 22c600a879ed420b931f782012e41162 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:15.596019 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22c600a879ed420b931f782012e41162" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
W20260925 09:58:15.597399 13440 leader_election.cc:341] T 22c600a879ed420b931f782012e41162 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:15.607849 13650 tablet_service.cc:1514] Processing DeleteTablet for tablet 676f113a83ee4931b7652562f59034a8 with delete_type TABLET_DATA_DELETED (replaced by tablet 22c600a879ed420b931f782012e41162) from {username='slave'} at 127.0.0.1:54548
W20260925 09:58:15.609627 13068 catalog_manager.cc:4821] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): delete failed for tablet 676f113a83ee4931b7652562f59034a8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 676f113a83ee4931b7652562f59034a8 already in progress: creating tablet
I20260925 09:58:15.615665 13770 ts_tablet_manager.cc:1428] T 676f113a83ee4931b7652562f59034a8 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.076s user 0.005s sys 0.000s
I20260925 09:58:15.621714 13073 master_service.cc:946] ReplaceTablet: received request to replace tablet 7e4fe7762300435995b06413fa84abdf from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:15.623142 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22c600a879ed420b931f782012e41162" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:15.624094 13770 tablet_bootstrap.cc:492] T da32d2b6fdef415e9b704bc8b1eff532 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:15.624127 13363 raft_consensus.cc:2454] T 22c600a879ed420b931f782012e41162 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 0.
I20260925 09:58:15.628350 13438 leader_election.cc:302] T 22c600a879ed420b931f782012e41162 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237
I20260925 09:58:15.629179 13770 tablet_bootstrap.cc:654] T da32d2b6fdef415e9b704bc8b1eff532 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:15.630473 13782 raft_consensus.cc:482] T 22c600a879ed420b931f782012e41162 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:15.631285 13782 raft_consensus.cc:504] T 22c600a879ed420b931f782012e41162 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:15.633030 13847 raft_consensus.cc:2781] T 22c600a879ed420b931f782012e41162 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:15.633358 13847 raft_consensus.cc:482] T 22c600a879ed420b931f782012e41162 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:15.633663 13847 raft_consensus.cc:3037] T 22c600a879ed420b931f782012e41162 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:15.634367 13782 leader_election.cc:288] T 22c600a879ed420b931f782012e41162 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:15.636837 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22c600a879ed420b931f782012e41162" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
I20260925 09:58:15.638993 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22c600a879ed420b931f782012e41162" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
W20260925 09:58:15.640704 13297 leader_election.cc:341] T 22c600a879ed420b931f782012e41162 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:15.641815 13777 ts_tablet_manager.cc:1918] T 20dd5e4109ff4e7ab838363bd1d0a12b P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:15.642584 13777 log.cc:1196] T 20dd5e4109ff4e7ab838363bd1d0a12b P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/20dd5e4109ff4e7ab838363bd1d0a12b
I20260925 09:58:15.643991 13077 catalog_manager.cc:5495] T 7e4fe7762300435995b06413fa84abdf P b3082cf15dc74c0480e5d33ecf592843 reported cstate change: term changed from 0 to 1, leader changed from <none> to b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b3082cf15dc74c0480e5d33ecf592843" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: UNKNOWN } } }
I20260925 09:58:15.647089 13777 ts_tablet_manager.cc:1939] T 20dd5e4109ff4e7ab838363bd1d0a12b P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:15.651108 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet 20dd5e4109ff4e7ab838363bd1d0a12b (table default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477]) successfully deleted
I20260925 09:58:15.651511 13847 raft_consensus.cc:504] T 22c600a879ed420b931f782012e41162 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:15.652704 13505 raft_consensus.cc:2379] T 22c600a879ed420b931f782012e41162 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3082cf15dc74c0480e5d33ecf592843 in current term 1: Already voted for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in this term.
I20260925 09:58:15.654232 13770 tablet_bootstrap.cc:492] T da32d2b6fdef415e9b704bc8b1eff532 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:15.654647 13770 ts_tablet_manager.cc:1397] T da32d2b6fdef415e9b704bc8b1eff532 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.031s user 0.010s sys 0.005s
I20260925 09:58:15.656564 13298 leader_election.cc:302] T 22c600a879ed420b931f782012e41162 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:58:15.657330 13782 raft_consensus.cc:3037] T 22c600a879ed420b931f782012e41162 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:15.657049 13770 raft_consensus.cc:348] T da32d2b6fdef415e9b704bc8b1eff532 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:15.657702 13770 raft_consensus.cc:374] T da32d2b6fdef415e9b704bc8b1eff532 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:15.657948 13770 raft_consensus.cc:729] T da32d2b6fdef415e9b704bc8b1eff532 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:15.658739 13770 consensus_queue.cc:260] T da32d2b6fdef415e9b704bc8b1eff532 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:15.660231 13777 tablet_replica.cc:330] T 39854cd5a1834ade80ed37a7f181eb25 P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:15.662073 13777 raft_consensus.cc:2229] T 39854cd5a1834ade80ed37a7f181eb25 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:15.662791 13777 raft_consensus.cc:2258] T 39854cd5a1834ade80ed37a7f181eb25 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:15.665349 13777 ts_tablet_manager.cc:1905] T 39854cd5a1834ade80ed37a7f181eb25 P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:15.665489 13782 raft_consensus.cc:2726] T 22c600a879ed420b931f782012e41162 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:58:15.666268 13770 ts_tablet_manager.cc:1428] T da32d2b6fdef415e9b704bc8b1eff532 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20260925 09:58:15.671851 13770 tablet_bootstrap.cc:492] T 22c600a879ed420b931f782012e41162 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:15.678440 13770 tablet_bootstrap.cc:654] T 22c600a879ed420b931f782012e41162 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:15.697764 13077 master_service.cc:946] ReplaceTablet: received request to replace tablet 39854cd5a1834ade80ed37a7f181eb25 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:15.701500 13770 tablet_bootstrap.cc:492] T 22c600a879ed420b931f782012e41162 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:15.701889 13770 ts_tablet_manager.cc:1397] T 22c600a879ed420b931f782012e41162 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.030s user 0.010s sys 0.004s
I20260925 09:58:15.704619 13650 tablet_service.cc:1514] Processing DeleteTablet for tablet 676f113a83ee4931b7652562f59034a8 with delete_type TABLET_DATA_DELETED (replaced by tablet 22c600a879ed420b931f782012e41162) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:15.706763 13780 tablet_replica.cc:330] T 676f113a83ee4931b7652562f59034a8 P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:15.707423 13780 raft_consensus.cc:2229] T 676f113a83ee4931b7652562f59034a8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:15.707181 13770 raft_consensus.cc:348] T 22c600a879ed420b931f782012e41162 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:15.707813 13780 raft_consensus.cc:2258] T 676f113a83ee4931b7652562f59034a8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:15.708289 13770 raft_consensus.cc:374] T 22c600a879ed420b931f782012e41162 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:15.708622 13770 raft_consensus.cc:729] T 22c600a879ed420b931f782012e41162 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:15.709291 13770 consensus_queue.cc:260] T 22c600a879ed420b931f782012e41162 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:15.713006 13780 ts_tablet_manager.cc:1905] T 676f113a83ee4931b7652562f59034a8 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:15.717769 13770 ts_tablet_manager.cc:1428] T 22c600a879ed420b931f782012e41162 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.016s user 0.006s sys 0.000s
I20260925 09:58:15.723151 13073 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet 7e4fe7762300435995b06413fa84abdf
I20260925 09:58:15.724953 13073 catalog_manager.cc:6565] ReplaceTablet: tablet 7e4fe7762300435995b06413fa84abdf deleted and replaced by tablet e8cdf5212ed64c0cbb4243cf18079714
I20260925 09:58:15.726725 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e4fe7762300435995b06413fa84abdf with delete_type TABLET_DATA_DELETED (replaced by tablet e8cdf5212ed64c0cbb4243cf18079714) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:15.731662 13650 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e4fe7762300435995b06413fa84abdf with delete_type TABLET_DATA_DELETED (replaced by tablet e8cdf5212ed64c0cbb4243cf18079714) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:15.733584 13847 leader_election.cc:288] T 22c600a879ed420b931f782012e41162 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Requested vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:15.734413 13485 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e4fe7762300435995b06413fa84abdf with delete_type TABLET_DATA_DELETED (replaced by tablet e8cdf5212ed64c0cbb4243cf18079714) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:15.735162 13781 tablet_replica.cc:330] T 7e4fe7762300435995b06413fa84abdf P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:15.735805 13781 raft_consensus.cc:2229] T 7e4fe7762300435995b06413fa84abdf P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:15.736238 13781 raft_consensus.cc:2258] T 7e4fe7762300435995b06413fa84abdf P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:15.736349 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22c600a879ed420b931f782012e41162" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843"
I20260925 09:58:15.737849 13781 ts_tablet_manager.cc:1905] T 7e4fe7762300435995b06413fa84abdf P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:15.737931 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22c600a879ed420b931f782012e41162" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
I20260925 09:58:15.738531 13671 raft_consensus.cc:3037] T 22c600a879ed420b931f782012e41162 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:15.742344 13073 catalog_manager.cc:5587] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731) does not have the latest schema for tablet da32d2b6fdef415e9b704bc8b1eff532 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]). Expected version 1 got 0
I20260925 09:58:15.744999 13671 raft_consensus.cc:2454] T 22c600a879ed420b931f782012e41162 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:15.746035 13440 leader_election.cc:302] T 22c600a879ed420b931f782012e41162 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:15.749507 13788 raft_consensus.cc:2781] T 22c600a879ed420b931f782012e41162 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:15.749958 13788 raft_consensus.cc:686] T 22c600a879ed420b931f782012e41162 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: Becoming Leader. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Running, Role: LEADER
I20260925 09:58:15.750794 13788 consensus_queue.cc:237] T 22c600a879ed420b931f782012e41162 P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:15.753075 13362 raft_consensus.cc:1261] T da32d2b6fdef415e9b704bc8b1eff532 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Refusing update from remote peer c4d724d4b83e4d1e817a3a505dc24eb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:58:15.760200 13848 consensus_queue.cc:1035] T da32d2b6fdef415e9b704bc8b1eff532 P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:58:15.764844 13671 raft_consensus.cc:3037] T da32d2b6fdef415e9b704bc8b1eff532 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:15.792291 13671 raft_consensus.cc:1261] T da32d2b6fdef415e9b704bc8b1eff532 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Refusing update from remote peer c4d724d4b83e4d1e817a3a505dc24eb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:58:15.793901 13788 consensus_queue.cc:1035] T da32d2b6fdef415e9b704bc8b1eff532 P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:15.813233 13781 ts_tablet_manager.cc:1918] T 7e4fe7762300435995b06413fa84abdf P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:15.813721 13781 log.cc:1196] T 7e4fe7762300435995b06413fa84abdf P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/7e4fe7762300435995b06413fa84abdf
I20260925 09:58:15.814786 13782 raft_consensus.cc:482] T 676f113a83ee4931b7652562f59034a8 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:15.820096 13650 tablet_service.cc:1467] Processing CreateTablet for tablet e8cdf5212ed64c0cbb4243cf18079714 (DEFAULT_TABLE table=default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:15.821226 13650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8cdf5212ed64c0cbb4243cf18079714. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:15.824719 13342 tablet_service.cc:1467] Processing CreateTablet for tablet e8cdf5212ed64c0cbb4243cf18079714 (DEFAULT_TABLE table=default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:15.826036 13342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8cdf5212ed64c0cbb4243cf18079714. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:15.827807 13781 ts_tablet_manager.cc:1939] T 7e4fe7762300435995b06413fa84abdf P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:15.839895 13483 tablet_service.cc:1467] Processing CreateTablet for tablet e8cdf5212ed64c0cbb4243cf18079714 (DEFAULT_TABLE table=default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:15.841075 13483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8cdf5212ed64c0cbb4243cf18079714. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:15.844647 13077 catalog_manager.cc:5495] T 22c600a879ed420b931f782012e41162 P c4d724d4b83e4d1e817a3a505dc24eb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2). New cstate: current_term: 1 leader_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: UNKNOWN } } }
I20260925 09:58:15.837846 13777 ts_tablet_manager.cc:1918] T 39854cd5a1834ade80ed37a7f181eb25 P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:15.846578 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet 7e4fe7762300435995b06413fa84abdf (table default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]) successfully deleted
I20260925 09:58:15.846765 13777 log.cc:1196] T 39854cd5a1834ade80ed37a7f181eb25 P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/39854cd5a1834ade80ed37a7f181eb25
I20260925 09:58:15.815294 13782 raft_consensus.cc:504] T 676f113a83ee4931b7652562f59034a8 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:15.849293 13777 ts_tablet_manager.cc:1939] T 39854cd5a1834ade80ed37a7f181eb25 P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
W20260925 09:58:15.850725 13298 leader_election.cc:341] T 676f113a83ee4931b7652562f59034a8 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: 676f113a83ee4931b7652562f59034a8
I20260925 09:58:15.852839 13670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "676f113a83ee4931b7652562f59034a8" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
W20260925 09:58:15.854766 13297 leader_election.cc:341] T 676f113a83ee4931b7652562f59034a8 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Illegal state: Tablet not RUNNING: STOPPED
I20260925 09:58:15.855410 13297 leader_election.cc:302] T 676f113a83ee4931b7652562f59034a8 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:58:15.856777 13843 raft_consensus.cc:2726] T 676f113a83ee4931b7652562f59034a8 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:58:15.857026 13777 tablet_replica.cc:330] T 676f113a83ee4931b7652562f59034a8 P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:15.857919 13777 raft_consensus.cc:2229] T 676f113a83ee4931b7652562f59034a8 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:15.857820 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet 39854cd5a1834ade80ed37a7f181eb25 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]) successfully deleted
I20260925 09:58:15.858310 13777 raft_consensus.cc:2258] T 676f113a83ee4931b7652562f59034a8 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:15.858400 13363 raft_consensus.cc:2454] T 22c600a879ed420b931f782012e41162 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:15.897475 13773 tablet_bootstrap.cc:492] T e8cdf5212ed64c0cbb4243cf18079714 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:15.904747 13773 tablet_bootstrap.cc:654] T e8cdf5212ed64c0cbb4243cf18079714 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:15.911171 13782 leader_election.cc:288] T 676f113a83ee4931b7652562f59034a8 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:15.917939 13777 ts_tablet_manager.cc:1905] T 676f113a83ee4931b7652562f59034a8 P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:15.922447 13853 mvcc.cc:205] Tried to move back new op lower bound from 7333192891360358400 to 7333192890466922496. Current Snapshot: MvccSnapshot[applied={T|T < 7333192891360358400}]
I20260925 09:58:15.930202 13773 tablet_bootstrap.cc:492] T e8cdf5212ed64c0cbb4243cf18079714 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:15.930678 13773 ts_tablet_manager.cc:1397] T e8cdf5212ed64c0cbb4243cf18079714 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.033s user 0.011s sys 0.000s
I20260925 09:58:15.933022 13773 raft_consensus.cc:348] T e8cdf5212ed64c0cbb4243cf18079714 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:15.933679 13773 raft_consensus.cc:374] T e8cdf5212ed64c0cbb4243cf18079714 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:15.933930 13773 raft_consensus.cc:729] T e8cdf5212ed64c0cbb4243cf18079714 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:15.934512 13773 consensus_queue.cc:260] T e8cdf5212ed64c0cbb4243cf18079714 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:15.936323 13773 ts_tablet_manager.cc:1428] T e8cdf5212ed64c0cbb4243cf18079714 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260925 09:58:15.950452 13863 tablet.cc:1722] T da32d2b6fdef415e9b704bc8b1eff532 P a06c3ef7a6a94f7cbcc03e7cd232f237: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:58:15.960145 13777 ts_tablet_manager.cc:1918] T 676f113a83ee4931b7652562f59034a8 P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:15.960511 13777 log.cc:1196] T 676f113a83ee4931b7652562f59034a8 P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/676f113a83ee4931b7652562f59034a8
I20260925 09:58:15.961138 13777 ts_tablet_manager.cc:1939] T 676f113a83ee4931b7652562f59034a8 P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:15.962220 13777 tablet_replica.cc:330] T 7e4fe7762300435995b06413fa84abdf P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:15.962273 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet 676f113a83ee4931b7652562f59034a8 (table default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477]) successfully deleted
I20260925 09:58:15.961789 13073 catalog_manager.cc:5587] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731) does not have the latest schema for tablet da32d2b6fdef415e9b704bc8b1eff532 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]). Expected version 1 got 0
I20260925 09:58:15.962872 13777 raft_consensus.cc:2229] T 7e4fe7762300435995b06413fa84abdf P b3082cf15dc74c0480e5d33ecf592843 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:58:15.963603 13777 pending_rounds.cc:62] T 7e4fe7762300435995b06413fa84abdf P b3082cf15dc74c0480e5d33ecf592843: Trying to abort 1 pending ops.
I20260925 09:58:15.966761 13077 catalog_manager.cc:5587] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327) does not have the latest schema for tablet da32d2b6fdef415e9b704bc8b1eff532 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]). Expected version 1 got 0
I20260925 09:58:15.968789 13862 tablet.cc:1722] T da32d2b6fdef415e9b704bc8b1eff532 P c4d724d4b83e4d1e817a3a505dc24eb9: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:58:15.970809 13780 ts_tablet_manager.cc:1918] T 676f113a83ee4931b7652562f59034a8 P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:15.963824 13777 pending_rounds.cc:68] T 7e4fe7762300435995b06413fa84abdf P b3082cf15dc74c0480e5d33ecf592843: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192890534768640 op_type: NO_OP noop_request { }
I20260925 09:58:15.971751 13777 raft_consensus.cc:2866] T 7e4fe7762300435995b06413fa84abdf P b3082cf15dc74c0480e5d33ecf592843 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:58:15.973430 13777 raft_consensus.cc:2258] T 7e4fe7762300435995b06413fa84abdf P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:15.976715 13777 ts_tablet_manager.cc:1905] T 7e4fe7762300435995b06413fa84abdf P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:15.990221 13073 master_service.cc:946] ReplaceTablet: received request to replace tablet 676f113a83ee4931b7652562f59034a8 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:15.991983 13073 master_service.cc:946] ReplaceTablet: received request to replace tablet 22c600a879ed420b931f782012e41162 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:16.005136 13780 log.cc:1196] T 676f113a83ee4931b7652562f59034a8 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/676f113a83ee4931b7652562f59034a8
I20260925 09:58:16.005980 13780 ts_tablet_manager.cc:1939] T 676f113a83ee4931b7652562f59034a8 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:16.007637 13077 catalog_manager.cc:6040] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e] alter complete (version 1)
I20260925 09:58:16.007869 13772 tablet_bootstrap.cc:492] T e8cdf5212ed64c0cbb4243cf18079714 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:16.013089 13077 catalog_manager.cc:5587] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161) does not have the latest schema for tablet da32d2b6fdef415e9b704bc8b1eff532 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]). Expected version 1 got 0
I20260925 09:58:16.018299 13772 tablet_bootstrap.cc:654] T e8cdf5212ed64c0cbb4243cf18079714 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:16.049216 13780 tablet_replica.cc:330] T 7e4fe7762300435995b06413fa84abdf P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:16.049710 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet 676f113a83ee4931b7652562f59034a8 (table default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477]) successfully deleted
I20260925 09:58:16.049888 13780 raft_consensus.cc:2229] T 7e4fe7762300435995b06413fa84abdf P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:16.050287 13780 raft_consensus.cc:2258] T 7e4fe7762300435995b06413fa84abdf P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:16.050554 13829 consensus_queue.cc:1035] T 7c004b586c60494fae35ebdbe2308f4c P a06c3ef7a6a94f7cbcc03e7cd232f237 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:16.071545 13777 ts_tablet_manager.cc:1918] T 7e4fe7762300435995b06413fa84abdf P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:16.075016 13073 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet 22c600a879ed420b931f782012e41162
I20260925 09:58:16.075582 13777 log.cc:1196] T 7e4fe7762300435995b06413fa84abdf P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/7e4fe7762300435995b06413fa84abdf
I20260925 09:58:16.076850 13777 ts_tablet_manager.cc:1939] T 7e4fe7762300435995b06413fa84abdf P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:16.076997 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet 22c600a879ed420b931f782012e41162 with delete_type TABLET_DATA_DELETED (replaced by tablet 9b3b332e2c7c4af4a3ce9764d314217f) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:16.078459 13073 catalog_manager.cc:6565] ReplaceTablet: tablet 22c600a879ed420b931f782012e41162 deleted and replaced by tablet 9b3b332e2c7c4af4a3ce9764d314217f
I20260925 09:58:16.078543 13649 tablet_service.cc:1514] Processing DeleteTablet for tablet 22c600a879ed420b931f782012e41162 with delete_type TABLET_DATA_DELETED (replaced by tablet 9b3b332e2c7c4af4a3ce9764d314217f) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:16.079329 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet 7e4fe7762300435995b06413fa84abdf (table default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]) successfully deleted
I20260925 09:58:16.080116 13777 tablet_replica.cc:330] T 22c600a879ed420b931f782012e41162 P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:16.080747 13777 raft_consensus.cc:2229] T 22c600a879ed420b931f782012e41162 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:16.081231 13777 raft_consensus.cc:2258] T 22c600a879ed420b931f782012e41162 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:16.083069 13777 ts_tablet_manager.cc:1905] T 22c600a879ed420b931f782012e41162 P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:16.086344 13483 tablet_service.cc:1514] Processing DeleteTablet for tablet 22c600a879ed420b931f782012e41162 with delete_type TABLET_DATA_DELETED (replaced by tablet 9b3b332e2c7c4af4a3ce9764d314217f) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:16.087105 13781 tablet_replica.cc:330] T 22c600a879ed420b931f782012e41162 P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:16.087844 13781 raft_consensus.cc:2229] T 22c600a879ed420b931f782012e41162 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:58:16.087940 13780 ts_tablet_manager.cc:1905] T 7e4fe7762300435995b06413fa84abdf P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:16.088472 13781 pending_rounds.cc:62] T 22c600a879ed420b931f782012e41162 P c4d724d4b83e4d1e817a3a505dc24eb9: Trying to abort 1 pending ops.
I20260925 09:58:16.088693 13781 pending_rounds.cc:68] T 22c600a879ed420b931f782012e41162 P c4d724d4b83e4d1e817a3a505dc24eb9: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192891403206656 op_type: NO_OP noop_request { }
I20260925 09:58:16.089037 13781 raft_consensus.cc:2866] T 22c600a879ed420b931f782012e41162 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:58:16.089361 13781 raft_consensus.cc:2258] T 22c600a879ed420b931f782012e41162 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:16.091576 13781 ts_tablet_manager.cc:1905] T 22c600a879ed420b931f782012e41162 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:16.092504 13770 tablet_bootstrap.cc:492] T e8cdf5212ed64c0cbb4243cf18079714 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:16.082823 13865 tablet.cc:1722] T da32d2b6fdef415e9b704bc8b1eff532 P b3082cf15dc74c0480e5d33ecf592843: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:58:16.116115 13770 tablet_bootstrap.cc:654] T e8cdf5212ed64c0cbb4243cf18079714 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:16.137980 13781 ts_tablet_manager.cc:1918] T 22c600a879ed420b931f782012e41162 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:16.146127 13777 ts_tablet_manager.cc:1918] T 22c600a879ed420b931f782012e41162 P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:16.150209 13770 tablet_bootstrap.cc:492] T e8cdf5212ed64c0cbb4243cf18079714 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:16.150637 13770 ts_tablet_manager.cc:1397] T e8cdf5212ed64c0cbb4243cf18079714 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.058s user 0.003s sys 0.011s
I20260925 09:58:16.150808 13781 log.cc:1196] T 22c600a879ed420b931f782012e41162 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/22c600a879ed420b931f782012e41162
I20260925 09:58:16.151831 13781 ts_tablet_manager.cc:1939] T 22c600a879ed420b931f782012e41162 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:16.153391 13777 log.cc:1196] T 22c600a879ed420b931f782012e41162 P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/22c600a879ed420b931f782012e41162
I20260925 09:58:16.154145 13777 ts_tablet_manager.cc:1939] T 22c600a879ed420b931f782012e41162 P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:16.156033 13073 catalog_manager.cc:5587] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161) does not have the latest schema for tablet da32d2b6fdef415e9b704bc8b1eff532 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]). Expected version 1 got 0
I20260925 09:58:16.160329 13770 raft_consensus.cc:348] T e8cdf5212ed64c0cbb4243cf18079714 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:16.161070 13770 raft_consensus.cc:374] T e8cdf5212ed64c0cbb4243cf18079714 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:16.161329 13770 raft_consensus.cc:729] T e8cdf5212ed64c0cbb4243cf18079714 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:16.163977 13770 consensus_queue.cc:260] T e8cdf5212ed64c0cbb4243cf18079714 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:16.170207 13829 consensus_queue.cc:1035] T 7c004b586c60494fae35ebdbe2308f4c P a06c3ef7a6a94f7cbcc03e7cd232f237 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:16.172703 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet 22c600a879ed420b931f782012e41162 (table default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477]) successfully deleted
I20260925 09:58:16.173497 13770 ts_tablet_manager.cc:1428] T e8cdf5212ed64c0cbb4243cf18079714 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.023s user 0.006s sys 0.000s
I20260925 09:58:16.197351 13772 tablet_bootstrap.cc:492] T e8cdf5212ed64c0cbb4243cf18079714 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:16.197899 13772 ts_tablet_manager.cc:1397] T e8cdf5212ed64c0cbb4243cf18079714 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.190s user 0.028s sys 0.031s
I20260925 09:58:16.200310 13772 raft_consensus.cc:348] T e8cdf5212ed64c0cbb4243cf18079714 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:16.201064 13772 raft_consensus.cc:374] T e8cdf5212ed64c0cbb4243cf18079714 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:16.201325 13772 raft_consensus.cc:729] T e8cdf5212ed64c0cbb4243cf18079714 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:16.201962 13772 consensus_queue.cc:260] T e8cdf5212ed64c0cbb4243cf18079714 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:16.203331 13780 ts_tablet_manager.cc:1918] T 7e4fe7762300435995b06413fa84abdf P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:16.203728 13780 log.cc:1196] T 7e4fe7762300435995b06413fa84abdf P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/7e4fe7762300435995b06413fa84abdf
I20260925 09:58:16.204336 13780 ts_tablet_manager.cc:1939] T 7e4fe7762300435995b06413fa84abdf P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:16.206733 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet 7e4fe7762300435995b06413fa84abdf (table default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]) successfully deleted
I20260925 09:58:16.208243 13772 ts_tablet_manager.cc:1428] T e8cdf5212ed64c0cbb4243cf18079714 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20260925 09:58:16.211710 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet 22c600a879ed420b931f782012e41162 (table default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477]) successfully deleted
I20260925 09:58:16.216159 13342 tablet_service.cc:1467] Processing CreateTablet for tablet 9b3b332e2c7c4af4a3ce9764d314217f (DEFAULT_TABLE table=default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:16.217257 13342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b3b332e2c7c4af4a3ce9764d314217f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:16.221041 13483 tablet_service.cc:1467] Processing CreateTablet for tablet 9b3b332e2c7c4af4a3ce9764d314217f (DEFAULT_TABLE table=default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:16.222168 13483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b3b332e2c7c4af4a3ce9764d314217f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:16.225622 13650 tablet_service.cc:1467] Processing CreateTablet for tablet 9b3b332e2c7c4af4a3ce9764d314217f (DEFAULT_TABLE table=default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:16.226603 13650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b3b332e2c7c4af4a3ce9764d314217f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:16.231624 13077 master_service.cc:946] ReplaceTablet: received request to replace tablet da32d2b6fdef415e9b704bc8b1eff532 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:16.233806 13780 tablet_replica.cc:330] T 22c600a879ed420b931f782012e41162 P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:16.234525 13780 raft_consensus.cc:2229] T 22c600a879ed420b931f782012e41162 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:16.234977 13780 raft_consensus.cc:2258] T 22c600a879ed420b931f782012e41162 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:16.243319 13780 ts_tablet_manager.cc:1905] T 22c600a879ed420b931f782012e41162 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:16.266185 13077 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet da32d2b6fdef415e9b704bc8b1eff532
I20260925 09:58:16.267580 13077 catalog_manager.cc:6565] ReplaceTablet: tablet da32d2b6fdef415e9b704bc8b1eff532 deleted and replaced by tablet b6445667835142acb08a24889415284d
I20260925 09:58:16.268090 13780 ts_tablet_manager.cc:1918] T 22c600a879ed420b931f782012e41162 P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:16.268244 13485 tablet_service.cc:1514] Processing DeleteTablet for tablet da32d2b6fdef415e9b704bc8b1eff532 with delete_type TABLET_DATA_DELETED (replaced by tablet b6445667835142acb08a24889415284d) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:16.268556 13780 log.cc:1196] T 22c600a879ed420b931f782012e41162 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/22c600a879ed420b931f782012e41162
I20260925 09:58:16.269003 13781 tablet_replica.cc:330] T da32d2b6fdef415e9b704bc8b1eff532 P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:16.269302 13780 ts_tablet_manager.cc:1939] T 22c600a879ed420b931f782012e41162 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:16.269619 13781 raft_consensus.cc:2229] T da32d2b6fdef415e9b704bc8b1eff532 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:58:16.270426 13781 raft_consensus.cc:2258] T da32d2b6fdef415e9b704bc8b1eff532 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:16.272436 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet 22c600a879ed420b931f782012e41162 (table default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477]) successfully deleted
I20260925 09:58:16.274138 13781 ts_tablet_manager.cc:1905] T da32d2b6fdef415e9b704bc8b1eff532 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:16.274269 13343 tablet_service.cc:1514] Processing DeleteTablet for tablet da32d2b6fdef415e9b704bc8b1eff532 with delete_type TABLET_DATA_DELETED (replaced by tablet b6445667835142acb08a24889415284d) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:16.275154 13777 tablet_replica.cc:330] T da32d2b6fdef415e9b704bc8b1eff532 P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:16.275774 13777 raft_consensus.cc:2229] T da32d2b6fdef415e9b704bc8b1eff532 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:16.276279 13777 raft_consensus.cc:2258] T da32d2b6fdef415e9b704bc8b1eff532 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:16.277889 13649 tablet_service.cc:1514] Processing DeleteTablet for tablet da32d2b6fdef415e9b704bc8b1eff532 with delete_type TABLET_DATA_DELETED (replaced by tablet b6445667835142acb08a24889415284d) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:16.279021 13773 tablet_bootstrap.cc:492] T 9b3b332e2c7c4af4a3ce9764d314217f P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:16.280725 13777 ts_tablet_manager.cc:1905] T da32d2b6fdef415e9b704bc8b1eff532 P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:16.282740 13780 tablet_replica.cc:330] T da32d2b6fdef415e9b704bc8b1eff532 P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:16.284333 13780 raft_consensus.cc:2229] T da32d2b6fdef415e9b704bc8b1eff532 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:16.284927 13780 raft_consensus.cc:2258] T da32d2b6fdef415e9b704bc8b1eff532 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:16.287603 13773 tablet_bootstrap.cc:654] T 9b3b332e2c7c4af4a3ce9764d314217f P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:16.296159 13781 ts_tablet_manager.cc:1918] T da32d2b6fdef415e9b704bc8b1eff532 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:58:16.296581 13781 log.cc:1196] T da32d2b6fdef415e9b704bc8b1eff532 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/da32d2b6fdef415e9b704bc8b1eff532
I20260925 09:58:16.297292 13781 ts_tablet_manager.cc:1939] T da32d2b6fdef415e9b704bc8b1eff532 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:16.299808 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet da32d2b6fdef415e9b704bc8b1eff532 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]) successfully deleted
I20260925 09:58:16.302745 13780 ts_tablet_manager.cc:1905] T da32d2b6fdef415e9b704bc8b1eff532 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:16.317006 13777 ts_tablet_manager.cc:1918] T da32d2b6fdef415e9b704bc8b1eff532 P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:58:16.317392 13777 log.cc:1196] T da32d2b6fdef415e9b704bc8b1eff532 P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/da32d2b6fdef415e9b704bc8b1eff532
I20260925 09:58:16.318089 13777 ts_tablet_manager.cc:1939] T da32d2b6fdef415e9b704bc8b1eff532 P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:16.320590 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet da32d2b6fdef415e9b704bc8b1eff532 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]) successfully deleted
I20260925 09:58:16.325242 13773 tablet_bootstrap.cc:492] T 9b3b332e2c7c4af4a3ce9764d314217f P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:16.325671 13773 ts_tablet_manager.cc:1397] T 9b3b332e2c7c4af4a3ce9764d314217f P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.047s user 0.010s sys 0.002s
I20260925 09:58:16.326367 13343 tablet_service.cc:1467] Processing CreateTablet for tablet b6445667835142acb08a24889415284d (DEFAULT_TABLE table=default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:16.327497 13343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6445667835142acb08a24889415284d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:16.327893 13773 raft_consensus.cc:348] T 9b3b332e2c7c4af4a3ce9764d314217f P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:16.328519 13773 raft_consensus.cc:374] T 9b3b332e2c7c4af4a3ce9764d314217f P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:16.328758 13773 raft_consensus.cc:729] T 9b3b332e2c7c4af4a3ce9764d314217f P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:16.329377 13073 master_service.cc:946] ReplaceTablet: received request to replace tablet e8cdf5212ed64c0cbb4243cf18079714 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:16.331868 13773 consensus_queue.cc:260] T 9b3b332e2c7c4af4a3ce9764d314217f P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:16.348173 13770 tablet_bootstrap.cc:492] T 9b3b332e2c7c4af4a3ce9764d314217f P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:16.349143 13773 ts_tablet_manager.cc:1428] T 9b3b332e2c7c4af4a3ce9764d314217f P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.023s user 0.005s sys 0.001s
I20260925 09:58:16.350718 13772 tablet_bootstrap.cc:492] T 9b3b332e2c7c4af4a3ce9764d314217f P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:16.356361 13772 tablet_bootstrap.cc:654] T 9b3b332e2c7c4af4a3ce9764d314217f P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:16.365767 13780 ts_tablet_manager.cc:1918] T da32d2b6fdef415e9b704bc8b1eff532 P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:58:16.366338 13780 log.cc:1196] T da32d2b6fdef415e9b704bc8b1eff532 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/da32d2b6fdef415e9b704bc8b1eff532
I20260925 09:58:16.367367 13780 ts_tablet_manager.cc:1939] T da32d2b6fdef415e9b704bc8b1eff532 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:16.368223 13770 tablet_bootstrap.cc:654] T 9b3b332e2c7c4af4a3ce9764d314217f P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:16.372735 13772 tablet_bootstrap.cc:492] T 9b3b332e2c7c4af4a3ce9764d314217f P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:16.373143 13772 ts_tablet_manager.cc:1397] T 9b3b332e2c7c4af4a3ce9764d314217f P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.023s user 0.014s sys 0.003s
I20260925 09:58:16.377414 13483 tablet_service.cc:1467] Processing CreateTablet for tablet b6445667835142acb08a24889415284d (DEFAULT_TABLE table=default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:16.379078 13483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6445667835142acb08a24889415284d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:16.380741 13650 tablet_service.cc:1467] Processing CreateTablet for tablet b6445667835142acb08a24889415284d (DEFAULT_TABLE table=default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:16.386852 13073 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet e8cdf5212ed64c0cbb4243cf18079714
I20260925 09:58:16.387531 13772 raft_consensus.cc:348] T 9b3b332e2c7c4af4a3ce9764d314217f P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:16.388273 13772 raft_consensus.cc:374] T 9b3b332e2c7c4af4a3ce9764d314217f P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:16.388437 13073 catalog_manager.cc:6565] ReplaceTablet: tablet e8cdf5212ed64c0cbb4243cf18079714 deleted and replaced by tablet 7098bdf9a9fc4234811ffdb29d43b1d6
I20260925 09:58:16.388521 13772 raft_consensus.cc:729] T 9b3b332e2c7c4af4a3ce9764d314217f P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:16.388821 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet e8cdf5212ed64c0cbb4243cf18079714 with delete_type TABLET_DATA_DELETED (replaced by tablet 7098bdf9a9fc4234811ffdb29d43b1d6) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:16.389603 13777 tablet_replica.cc:330] T e8cdf5212ed64c0cbb4243cf18079714 P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:16.389164 13772 consensus_queue.cc:260] T 9b3b332e2c7c4af4a3ce9764d314217f P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:16.390257 13777 raft_consensus.cc:2229] T e8cdf5212ed64c0cbb4243cf18079714 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:16.390717 13777 raft_consensus.cc:2258] T e8cdf5212ed64c0cbb4243cf18079714 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:16.391265 13485 tablet_service.cc:1514] Processing DeleteTablet for tablet e8cdf5212ed64c0cbb4243cf18079714 with delete_type TABLET_DATA_DELETED (replaced by tablet 7098bdf9a9fc4234811ffdb29d43b1d6) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:16.394164 13781 tablet_replica.cc:330] T e8cdf5212ed64c0cbb4243cf18079714 P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:16.394893 13781 raft_consensus.cc:2229] T e8cdf5212ed64c0cbb4243cf18079714 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:16.394987 13772 ts_tablet_manager.cc:1428] T 9b3b332e2c7c4af4a3ce9764d314217f P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.022s user 0.006s sys 0.001s
I20260925 09:58:16.395354 13781 raft_consensus.cc:2258] T e8cdf5212ed64c0cbb4243cf18079714 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:16.401460 13649 tablet_service.cc:1514] Processing DeleteTablet for tablet e8cdf5212ed64c0cbb4243cf18079714 with delete_type TABLET_DATA_DELETED (replaced by tablet 7098bdf9a9fc4234811ffdb29d43b1d6) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:16.404027 13777 ts_tablet_manager.cc:1905] T e8cdf5212ed64c0cbb4243cf18079714 P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:16.409369 13781 ts_tablet_manager.cc:1905] T e8cdf5212ed64c0cbb4243cf18079714 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:16.420722 13650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6445667835142acb08a24889415284d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:16.436115 13342 tablet_service.cc:1467] Processing CreateTablet for tablet 7098bdf9a9fc4234811ffdb29d43b1d6 (DEFAULT_TABLE table=default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:16.437377 13342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7098bdf9a9fc4234811ffdb29d43b1d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:16.449334 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet da32d2b6fdef415e9b704bc8b1eff532 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]) successfully deleted
I20260925 09:58:16.452301 13485 tablet_service.cc:1467] Processing CreateTablet for tablet 7098bdf9a9fc4234811ffdb29d43b1d6 (DEFAULT_TABLE table=default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:16.453562 13485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7098bdf9a9fc4234811ffdb29d43b1d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:16.453409 13649 tablet_service.cc:1467] Processing CreateTablet for tablet 7098bdf9a9fc4234811ffdb29d43b1d6 (DEFAULT_TABLE table=default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:16.454649 13649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7098bdf9a9fc4234811ffdb29d43b1d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:16.455005 13773 tablet_bootstrap.cc:492] T b6445667835142acb08a24889415284d P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:16.460992 13773 tablet_bootstrap.cc:654] T b6445667835142acb08a24889415284d P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:16.462137 13772 tablet_bootstrap.cc:492] T 7098bdf9a9fc4234811ffdb29d43b1d6 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:16.463346 13780 tablet_replica.cc:330] T e8cdf5212ed64c0cbb4243cf18079714 P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:16.464077 13780 raft_consensus.cc:2229] T e8cdf5212ed64c0cbb4243cf18079714 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:16.464535 13780 raft_consensus.cc:2258] T e8cdf5212ed64c0cbb4243cf18079714 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:16.470865 13772 tablet_bootstrap.cc:654] T 7098bdf9a9fc4234811ffdb29d43b1d6 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:16.482313 13073 master_service.cc:946] ReplaceTablet: received request to replace tablet 7098bdf9a9fc4234811ffdb29d43b1d6 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:16.486598 13770 tablet_bootstrap.cc:492] T 9b3b332e2c7c4af4a3ce9764d314217f P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:16.487054 13770 ts_tablet_manager.cc:1397] T 9b3b332e2c7c4af4a3ce9764d314217f P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.139s user 0.009s sys 0.000s
I20260925 09:58:16.489555 13770 raft_consensus.cc:348] T 9b3b332e2c7c4af4a3ce9764d314217f P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:16.490285 13770 raft_consensus.cc:374] T 9b3b332e2c7c4af4a3ce9764d314217f P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:16.490581 13770 raft_consensus.cc:729] T 9b3b332e2c7c4af4a3ce9764d314217f P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:16.508993 13780 ts_tablet_manager.cc:1905] T e8cdf5212ed64c0cbb4243cf18079714 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:16.510104 13782 raft_consensus.cc:482] T 9b3b332e2c7c4af4a3ce9764d314217f P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:16.510552 13782 raft_consensus.cc:504] T 9b3b332e2c7c4af4a3ce9764d314217f P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:16.512382 13782 leader_election.cc:288] T 9b3b332e2c7c4af4a3ce9764d314217f P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:16.514910 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b3b332e2c7c4af4a3ce9764d314217f" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
I20260925 09:58:16.515553 13505 raft_consensus.cc:2454] T 9b3b332e2c7c4af4a3ce9764d314217f P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 0.
I20260925 09:58:16.516543 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b3b332e2c7c4af4a3ce9764d314217f" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:16.518342 13298 leader_election.cc:302] T 9b3b332e2c7c4af4a3ce9764d314217f P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:16.519960 13073 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet 7098bdf9a9fc4234811ffdb29d43b1d6
I20260925 09:58:16.521636 13343 tablet_service.cc:1514] Processing DeleteTablet for tablet 7098bdf9a9fc4234811ffdb29d43b1d6 with delete_type TABLET_DATA_DELETED (replaced by tablet ffd52766626d4d0691b4093659f3918e) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:16.522199 13770 consensus_queue.cc:260] T 9b3b332e2c7c4af4a3ce9764d314217f P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:16.522742 13073 catalog_manager.cc:6565] ReplaceTablet: tablet 7098bdf9a9fc4234811ffdb29d43b1d6 deleted and replaced by tablet ffd52766626d4d0691b4093659f3918e
I20260925 09:58:16.523136 13671 raft_consensus.cc:2454] T 9b3b332e2c7c4af4a3ce9764d314217f P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 0.
I20260925 09:58:16.523241 13483 tablet_service.cc:1514] Processing DeleteTablet for tablet 7098bdf9a9fc4234811ffdb29d43b1d6 with delete_type TABLET_DATA_DELETED (replaced by tablet ffd52766626d4d0691b4093659f3918e) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:16.524191 13782 raft_consensus.cc:2781] T 9b3b332e2c7c4af4a3ce9764d314217f P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:16.524559 13782 raft_consensus.cc:482] T 9b3b332e2c7c4af4a3ce9764d314217f P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:16.524884 13782 raft_consensus.cc:3037] T 9b3b332e2c7c4af4a3ce9764d314217f P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:16.539917 13770 ts_tablet_manager.cc:1428] T 9b3b332e2c7c4af4a3ce9764d314217f P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.053s user 0.008s sys 0.000s
I20260925 09:58:16.542438 13649 tablet_service.cc:1514] Processing DeleteTablet for tablet 7098bdf9a9fc4234811ffdb29d43b1d6 with delete_type TABLET_DATA_DELETED (replaced by tablet ffd52766626d4d0691b4093659f3918e) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:16.550367 13770 tablet_bootstrap.cc:492] T 7098bdf9a9fc4234811ffdb29d43b1d6 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:16.551898 13777 ts_tablet_manager.cc:1918] T e8cdf5212ed64c0cbb4243cf18079714 P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:16.552296 13777 log.cc:1196] T e8cdf5212ed64c0cbb4243cf18079714 P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/e8cdf5212ed64c0cbb4243cf18079714
I20260925 09:58:16.552953 13777 ts_tablet_manager.cc:1939] T e8cdf5212ed64c0cbb4243cf18079714 P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:16.553628 13773 tablet_bootstrap.cc:492] T b6445667835142acb08a24889415284d P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:16.553697 13781 ts_tablet_manager.cc:1918] T e8cdf5212ed64c0cbb4243cf18079714 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:16.554750 13781 log.cc:1196] T e8cdf5212ed64c0cbb4243cf18079714 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/e8cdf5212ed64c0cbb4243cf18079714
I20260925 09:58:16.554351 13773 ts_tablet_manager.cc:1397] T b6445667835142acb08a24889415284d P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.100s user 0.012s sys 0.006s
I20260925 09:58:16.557902 13773 raft_consensus.cc:348] T b6445667835142acb08a24889415284d P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:16.558519 13773 raft_consensus.cc:374] T b6445667835142acb08a24889415284d P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:16.558758 13773 raft_consensus.cc:729] T b6445667835142acb08a24889415284d P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:16.562775 13781 ts_tablet_manager.cc:1939] T e8cdf5212ed64c0cbb4243cf18079714 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:16.563586 13770 tablet_bootstrap.cc:654] T 7098bdf9a9fc4234811ffdb29d43b1d6 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:16.565090 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet e8cdf5212ed64c0cbb4243cf18079714 (table default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]) successfully deleted
I20260925 09:58:16.566470 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet e8cdf5212ed64c0cbb4243cf18079714 (table default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]) successfully deleted
W20260925 09:58:16.568650 13069 catalog_manager.cc:4821] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): delete failed for tablet 7098bdf9a9fc4234811ffdb29d43b1d6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7098bdf9a9fc4234811ffdb29d43b1d6 already in progress: creating tablet
W20260925 09:58:16.569075 13066 catalog_manager.cc:4821] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): delete failed for tablet 7098bdf9a9fc4234811ffdb29d43b1d6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7098bdf9a9fc4234811ffdb29d43b1d6 already in progress: creating tablet
I20260925 09:58:16.571934 13782 raft_consensus.cc:504] T 9b3b332e2c7c4af4a3ce9764d314217f P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:16.579300 13780 ts_tablet_manager.cc:1918] T e8cdf5212ed64c0cbb4243cf18079714 P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:16.573851 13773 consensus_queue.cc:260] T b6445667835142acb08a24889415284d P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:16.579753 13780 log.cc:1196] T e8cdf5212ed64c0cbb4243cf18079714 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/e8cdf5212ed64c0cbb4243cf18079714
I20260925 09:58:16.582350 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b3b332e2c7c4af4a3ce9764d314217f" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9"
I20260925 09:58:16.582664 13780 ts_tablet_manager.cc:1939] T e8cdf5212ed64c0cbb4243cf18079714 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:16.582950 13505 raft_consensus.cc:3037] T 9b3b332e2c7c4af4a3ce9764d314217f P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:16.581740 13770 tablet_bootstrap.cc:492] T 7098bdf9a9fc4234811ffdb29d43b1d6 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:16.584524 13770 ts_tablet_manager.cc:1397] T 7098bdf9a9fc4234811ffdb29d43b1d6 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.034s user 0.007s sys 0.003s
I20260925 09:58:16.586735 13770 raft_consensus.cc:348] T 7098bdf9a9fc4234811ffdb29d43b1d6 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:16.587426 13770 raft_consensus.cc:374] T 7098bdf9a9fc4234811ffdb29d43b1d6 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:16.587688 13770 raft_consensus.cc:729] T 7098bdf9a9fc4234811ffdb29d43b1d6 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:16.590710 13505 raft_consensus.cc:2454] T 9b3b332e2c7c4af4a3ce9764d314217f P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 1.
I20260925 09:58:16.591830 13298 leader_election.cc:302] T 9b3b332e2c7c4af4a3ce9764d314217f P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:16.593992 13773 ts_tablet_manager.cc:1428] T b6445667835142acb08a24889415284d P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.038s user 0.003s sys 0.002s
I20260925 09:58:16.598309 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b3b332e2c7c4af4a3ce9764d314217f" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
I20260925 09:58:16.599059 13671 raft_consensus.cc:3037] T 9b3b332e2c7c4af4a3ce9764d314217f P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:16.599613 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet e8cdf5212ed64c0cbb4243cf18079714 (table default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]) successfully deleted
W20260925 09:58:16.602281 13068 catalog_manager.cc:4821] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): delete failed for tablet 7098bdf9a9fc4234811ffdb29d43b1d6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7098bdf9a9fc4234811ffdb29d43b1d6 already in progress: creating tablet
I20260925 09:58:16.589665 13770 consensus_queue.cc:260] T 7098bdf9a9fc4234811ffdb29d43b1d6 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:16.603672 13773 tablet_bootstrap.cc:492] T 7098bdf9a9fc4234811ffdb29d43b1d6 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:16.608659 13671 raft_consensus.cc:2454] T 9b3b332e2c7c4af4a3ce9764d314217f P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 1.
I20260925 09:58:16.610433 13782 leader_election.cc:288] T 9b3b332e2c7c4af4a3ce9764d314217f P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Requested vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:16.612231 13772 tablet_bootstrap.cc:492] T 7098bdf9a9fc4234811ffdb29d43b1d6 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:16.613662 13772 ts_tablet_manager.cc:1397] T 7098bdf9a9fc4234811ffdb29d43b1d6 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.152s user 0.016s sys 0.007s
I20260925 09:58:16.616114 13772 raft_consensus.cc:348] T 7098bdf9a9fc4234811ffdb29d43b1d6 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:16.617025 13772 raft_consensus.cc:374] T 7098bdf9a9fc4234811ffdb29d43b1d6 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:16.617635 13649 tablet_service.cc:1514] Processing DeleteTablet for tablet 7098bdf9a9fc4234811ffdb29d43b1d6 with delete_type TABLET_DATA_DELETED (replaced by tablet ffd52766626d4d0691b4093659f3918e) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:16.617326 13772 raft_consensus.cc:729] T 7098bdf9a9fc4234811ffdb29d43b1d6 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:16.618566 13770 ts_tablet_manager.cc:1428] T 7098bdf9a9fc4234811ffdb29d43b1d6 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.034s user 0.005s sys 0.000s
I20260925 09:58:16.619074 13782 raft_consensus.cc:2781] T 9b3b332e2c7c4af4a3ce9764d314217f P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:16.619374 13780 tablet_replica.cc:330] T 7098bdf9a9fc4234811ffdb29d43b1d6 P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:16.620350 13780 raft_consensus.cc:2229] T 7098bdf9a9fc4234811ffdb29d43b1d6 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:16.620764 13780 raft_consensus.cc:2258] T 7098bdf9a9fc4234811ffdb29d43b1d6 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:16.630950 13773 tablet_bootstrap.cc:654] T 7098bdf9a9fc4234811ffdb29d43b1d6 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:16.640010 13770 tablet_bootstrap.cc:492] T b6445667835142acb08a24889415284d P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:16.642628 13073 catalog_manager.cc:5587] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327) does not have the latest schema for tablet b6445667835142acb08a24889415284d (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]). Expected version 1 got 0
W20260925 09:58:16.644323 13073 catalog_manager.cc:5687] Failed to send AlterTable RPC for tablet b6445667835142acb08a24889415284d (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet b6445667835142acb08a24889415284d consensus state has no leader
I20260925 09:58:16.650674 13077 catalog_manager.cc:5587] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327) does not have the latest schema for tablet b6445667835142acb08a24889415284d (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]). Expected version 1 got 0
I20260925 09:58:16.652227 13485 tablet_service.cc:1467] Processing CreateTablet for tablet ffd52766626d4d0691b4093659f3918e (DEFAULT_TABLE table=default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:16.653327 13485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffd52766626d4d0691b4093659f3918e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:16.654945 13343 tablet_service.cc:1467] Processing CreateTablet for tablet ffd52766626d4d0691b4093659f3918e (DEFAULT_TABLE table=default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:16.656328 13343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffd52766626d4d0691b4093659f3918e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:16.656657 13780 ts_tablet_manager.cc:1905] T 7098bdf9a9fc4234811ffdb29d43b1d6 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:16.662070 13770 tablet_bootstrap.cc:654] T b6445667835142acb08a24889415284d P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:16.668977 13773 tablet_bootstrap.cc:492] T 7098bdf9a9fc4234811ffdb29d43b1d6 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:16.669554 13773 ts_tablet_manager.cc:1397] T 7098bdf9a9fc4234811ffdb29d43b1d6 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.066s user 0.003s sys 0.010s
I20260925 09:58:16.671625 13782 raft_consensus.cc:686] T 9b3b332e2c7c4af4a3ce9764d314217f P b3082cf15dc74c0480e5d33ecf592843 [term 1 LEADER]: Becoming Leader. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Running, Role: LEADER
I20260925 09:58:16.672727 13782 consensus_queue.cc:237] T 9b3b332e2c7c4af4a3ce9764d314217f P b3082cf15dc74c0480e5d33ecf592843 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:16.673193 13773 raft_consensus.cc:348] T 7098bdf9a9fc4234811ffdb29d43b1d6 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:16.674185 13773 raft_consensus.cc:374] T 7098bdf9a9fc4234811ffdb29d43b1d6 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260925 09:58:16.677943 13067 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet b6445667835142acb08a24889415284d (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet b6445667835142acb08a24889415284d consensus state has no leader
I20260925 09:58:16.680032 13780 ts_tablet_manager.cc:1918] T 7098bdf9a9fc4234811ffdb29d43b1d6 P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:16.680387 13780 log.cc:1196] T 7098bdf9a9fc4234811ffdb29d43b1d6 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/7098bdf9a9fc4234811ffdb29d43b1d6
I20260925 09:58:16.680974 13780 ts_tablet_manager.cc:1939] T 7098bdf9a9fc4234811ffdb29d43b1d6 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:16.618685 13772 consensus_queue.cc:260] T 7098bdf9a9fc4234811ffdb29d43b1d6 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
W20260925 09:58:16.646893 13298 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.2:44327, user_credentials={real_user=slave}} blocked reactor thread for 55152.9us
I20260925 09:58:16.683249 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet 7098bdf9a9fc4234811ffdb29d43b1d6 (table default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]) successfully deleted
I20260925 09:58:16.674525 13773 raft_consensus.cc:729] T 7098bdf9a9fc4234811ffdb29d43b1d6 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:16.684800 13773 consensus_queue.cc:260] T 7098bdf9a9fc4234811ffdb29d43b1d6 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:16.687636 13077 master_service.cc:946] ReplaceTablet: received request to replace tablet b6445667835142acb08a24889415284d from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:16.688235 13772 ts_tablet_manager.cc:1428] T 7098bdf9a9fc4234811ffdb29d43b1d6 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.074s user 0.003s sys 0.001s
I20260925 09:58:16.693768 13772 tablet_bootstrap.cc:492] T b6445667835142acb08a24889415284d P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:16.700474 13773 ts_tablet_manager.cc:1428] T 7098bdf9a9fc4234811ffdb29d43b1d6 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.031s user 0.006s sys 0.001s
I20260925 09:58:16.700328 13649 tablet_service.cc:1467] Processing CreateTablet for tablet ffd52766626d4d0691b4093659f3918e (DEFAULT_TABLE table=default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:16.701642 13649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffd52766626d4d0691b4093659f3918e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:16.704254 13773 tablet_bootstrap.cc:492] T ffd52766626d4d0691b4093659f3918e P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:16.722148 13077 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet b6445667835142acb08a24889415284d
I20260925 09:58:16.717054 13773 tablet_bootstrap.cc:654] T ffd52766626d4d0691b4093659f3918e P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:16.724946 13650 tablet_service.cc:1514] Processing DeleteTablet for tablet b6445667835142acb08a24889415284d with delete_type TABLET_DATA_DELETED (replaced by tablet 56559fc1c66e41fd8d6e1e767d4dd7cc) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:16.725513 13077 catalog_manager.cc:6565] ReplaceTablet: tablet b6445667835142acb08a24889415284d deleted and replaced by tablet 56559fc1c66e41fd8d6e1e767d4dd7cc
I20260925 09:58:16.727149 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet b6445667835142acb08a24889415284d with delete_type TABLET_DATA_DELETED (replaced by tablet 56559fc1c66e41fd8d6e1e767d4dd7cc) from {username='slave'} at 127.0.0.1:49260
W20260925 09:58:16.729030 13066 catalog_manager.cc:4821] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): delete failed for tablet b6445667835142acb08a24889415284d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b6445667835142acb08a24889415284d already in progress: creating tablet
I20260925 09:58:16.731149 13485 tablet_service.cc:1514] Processing DeleteTablet for tablet b6445667835142acb08a24889415284d with delete_type TABLET_DATA_DELETED (replaced by tablet 56559fc1c66e41fd8d6e1e767d4dd7cc) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:16.731909 13781 tablet_replica.cc:330] T b6445667835142acb08a24889415284d P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:16.732545 13781 raft_consensus.cc:2229] T b6445667835142acb08a24889415284d P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:16.732980 13781 raft_consensus.cc:2258] T b6445667835142acb08a24889415284d P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:16.734946 13781 ts_tablet_manager.cc:1905] T b6445667835142acb08a24889415284d P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:16.736984 13485 tablet_service.cc:1514] Processing DeleteTablet for tablet 7098bdf9a9fc4234811ffdb29d43b1d6 with delete_type TABLET_DATA_DELETED (replaced by tablet ffd52766626d4d0691b4093659f3918e) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:16.749054 13773 tablet_bootstrap.cc:492] T ffd52766626d4d0691b4093659f3918e P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:16.749606 13773 ts_tablet_manager.cc:1397] T ffd52766626d4d0691b4093659f3918e P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.046s user 0.007s sys 0.002s
I20260925 09:58:16.751880 13770 tablet_bootstrap.cc:492] T b6445667835142acb08a24889415284d P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:16.752391 13770 ts_tablet_manager.cc:1397] T b6445667835142acb08a24889415284d P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.113s user 0.007s sys 0.005s
I20260925 09:58:16.755300 13772 tablet_bootstrap.cc:654] T b6445667835142acb08a24889415284d P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:16.754998 13770 raft_consensus.cc:348] T b6445667835142acb08a24889415284d P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:16.755797 13770 raft_consensus.cc:374] T b6445667835142acb08a24889415284d P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:16.756201 13770 raft_consensus.cc:729] T b6445667835142acb08a24889415284d P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:16.758314 13770 consensus_queue.cc:260] T b6445667835142acb08a24889415284d P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:16.761439 13770 ts_tablet_manager.cc:1428] T b6445667835142acb08a24889415284d P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.009s user 0.004s sys 0.000s
I20260925 09:58:16.767755 13773 raft_consensus.cc:348] T ffd52766626d4d0691b4093659f3918e P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:16.771659 13773 raft_consensus.cc:374] T ffd52766626d4d0691b4093659f3918e P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:16.774478 13773 raft_consensus.cc:729] T ffd52766626d4d0691b4093659f3918e P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:16.775280 13773 consensus_queue.cc:260] T ffd52766626d4d0691b4093659f3918e P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:16.792040 13773 ts_tablet_manager.cc:1428] T ffd52766626d4d0691b4093659f3918e P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.042s user 0.005s sys 0.000s
I20260925 09:58:16.792367 13073 master_service.cc:946] ReplaceTablet: received request to replace tablet b6445667835142acb08a24889415284d from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:16.774163 13781 ts_tablet_manager.cc:1918] T b6445667835142acb08a24889415284d P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:16.796610 13781 log.cc:1196] T b6445667835142acb08a24889415284d P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/b6445667835142acb08a24889415284d
I20260925 09:58:16.797340 13781 ts_tablet_manager.cc:1939] T b6445667835142acb08a24889415284d P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:16.800071 13077 catalog_manager.cc:5495] T 9b3b332e2c7c4af4a3ce9764d314217f P b3082cf15dc74c0480e5d33ecf592843 reported cstate change: term changed from 0 to 1, leader changed from <none> to b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b3082cf15dc74c0480e5d33ecf592843" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: UNKNOWN } } }
I20260925 09:58:16.805918 13781 tablet_replica.cc:330] T 7098bdf9a9fc4234811ffdb29d43b1d6 P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:16.805979 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet b6445667835142acb08a24889415284d (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]) successfully deleted
I20260925 09:58:16.811568 13781 raft_consensus.cc:2229] T 7098bdf9a9fc4234811ffdb29d43b1d6 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:16.812196 13781 raft_consensus.cc:2258] T 7098bdf9a9fc4234811ffdb29d43b1d6 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:16.814214 13781 ts_tablet_manager.cc:1905] T 7098bdf9a9fc4234811ffdb29d43b1d6 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:16.836771 13343 tablet_service.cc:1467] Processing CreateTablet for tablet 56559fc1c66e41fd8d6e1e767d4dd7cc (DEFAULT_TABLE table=default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:16.838218 13343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56559fc1c66e41fd8d6e1e767d4dd7cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:16.846844 13770 tablet_bootstrap.cc:492] T ffd52766626d4d0691b4093659f3918e P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:16.852666 13485 tablet_service.cc:1467] Processing CreateTablet for tablet 56559fc1c66e41fd8d6e1e767d4dd7cc (DEFAULT_TABLE table=default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:16.853818 13485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56559fc1c66e41fd8d6e1e767d4dd7cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:16.857934 13649 tablet_service.cc:1467] Processing CreateTablet for tablet 56559fc1c66e41fd8d6e1e767d4dd7cc (DEFAULT_TABLE table=default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:16.859187 13770 tablet_bootstrap.cc:654] T ffd52766626d4d0691b4093659f3918e P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:16.859287 13649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56559fc1c66e41fd8d6e1e767d4dd7cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:16.877308 13770 tablet_bootstrap.cc:492] T ffd52766626d4d0691b4093659f3918e P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:16.877806 13770 ts_tablet_manager.cc:1397] T ffd52766626d4d0691b4093659f3918e P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.031s user 0.008s sys 0.000s
I20260925 09:58:16.881116 13770 raft_consensus.cc:348] T ffd52766626d4d0691b4093659f3918e P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:16.882109 13770 raft_consensus.cc:374] T ffd52766626d4d0691b4093659f3918e P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:16.882357 13770 raft_consensus.cc:729] T ffd52766626d4d0691b4093659f3918e P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:16.882965 13770 consensus_queue.cc:260] T ffd52766626d4d0691b4093659f3918e P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:16.884869 13770 ts_tablet_manager.cc:1428] T ffd52766626d4d0691b4093659f3918e P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260925 09:58:16.895094 13772 tablet_bootstrap.cc:492] T b6445667835142acb08a24889415284d P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:16.895572 13772 ts_tablet_manager.cc:1397] T b6445667835142acb08a24889415284d P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.202s user 0.010s sys 0.000s
I20260925 09:58:16.859871 13780 tablet_replica.cc:330] T b6445667835142acb08a24889415284d P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:16.898413 13773 tablet_bootstrap.cc:492] T 56559fc1c66e41fd8d6e1e767d4dd7cc P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:16.900277 13780 raft_consensus.cc:2229] T b6445667835142acb08a24889415284d P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:16.900732 13780 raft_consensus.cc:2258] T b6445667835142acb08a24889415284d P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:16.903232 13773 tablet_bootstrap.cc:654] T 56559fc1c66e41fd8d6e1e767d4dd7cc P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:16.912257 13770 tablet_bootstrap.cc:492] T 56559fc1c66e41fd8d6e1e767d4dd7cc P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:16.915895 13781 ts_tablet_manager.cc:1918] T 7098bdf9a9fc4234811ffdb29d43b1d6 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:16.916149 13780 ts_tablet_manager.cc:1905] T b6445667835142acb08a24889415284d P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:16.916368 13781 log.cc:1196] T 7098bdf9a9fc4234811ffdb29d43b1d6 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/7098bdf9a9fc4234811ffdb29d43b1d6
I20260925 09:58:16.917518 13770 tablet_bootstrap.cc:654] T 56559fc1c66e41fd8d6e1e767d4dd7cc P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:16.918128 13781 ts_tablet_manager.cc:1939] T 7098bdf9a9fc4234811ffdb29d43b1d6 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:16.925755 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet 7098bdf9a9fc4234811ffdb29d43b1d6 (table default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]) successfully deleted
I20260925 09:58:16.931712 13773 tablet_bootstrap.cc:492] T 56559fc1c66e41fd8d6e1e767d4dd7cc P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:16.932485 13773 ts_tablet_manager.cc:1397] T 56559fc1c66e41fd8d6e1e767d4dd7cc P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.034s user 0.011s sys 0.000s
I20260925 09:58:16.940972 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet 7098bdf9a9fc4234811ffdb29d43b1d6 with delete_type TABLET_DATA_DELETED (replaced by tablet ffd52766626d4d0691b4093659f3918e) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:16.943365 13777 tablet_replica.cc:330] T 7098bdf9a9fc4234811ffdb29d43b1d6 P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:16.935555 13773 raft_consensus.cc:348] T 56559fc1c66e41fd8d6e1e767d4dd7cc P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:16.945197 13773 raft_consensus.cc:374] T 56559fc1c66e41fd8d6e1e767d4dd7cc P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:16.945681 13773 raft_consensus.cc:729] T 56559fc1c66e41fd8d6e1e767d4dd7cc P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:16.946110 13777 raft_consensus.cc:2229] T 7098bdf9a9fc4234811ffdb29d43b1d6 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:16.947031 13777 raft_consensus.cc:2258] T 7098bdf9a9fc4234811ffdb29d43b1d6 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:16.946914 13773 consensus_queue.cc:260] T 56559fc1c66e41fd8d6e1e767d4dd7cc P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:16.954288 13777 ts_tablet_manager.cc:1905] T 7098bdf9a9fc4234811ffdb29d43b1d6 P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:16.958753 13773 ts_tablet_manager.cc:1428] T 56559fc1c66e41fd8d6e1e767d4dd7cc P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.026s user 0.006s sys 0.000s
I20260925 09:58:16.988565 13848 raft_consensus.cc:482] T 56559fc1c66e41fd8d6e1e767d4dd7cc P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:16.989018 13848 raft_consensus.cc:504] T 56559fc1c66e41fd8d6e1e767d4dd7cc P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:16.991178 13848 leader_election.cc:288] T 56559fc1c66e41fd8d6e1e767d4dd7cc P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:16.994264 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56559fc1c66e41fd8d6e1e767d4dd7cc" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
W20260925 09:58:16.996515 13440 leader_election.cc:341] T 56559fc1c66e41fd8d6e1e767d4dd7cc P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:58:16.997648 13438 leader_election.cc:341] T 56559fc1c66e41fd8d6e1e767d4dd7cc P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Not found: Tablet not found: 56559fc1c66e41fd8d6e1e767d4dd7cc
I20260925 09:58:16.998095 13438 leader_election.cc:302] T 56559fc1c66e41fd8d6e1e767d4dd7cc P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4d724d4b83e4d1e817a3a505dc24eb9; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, b3082cf15dc74c0480e5d33ecf592843
I20260925 09:58:16.999365 13848 raft_consensus.cc:2726] T 56559fc1c66e41fd8d6e1e767d4dd7cc P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:58:17.012408 13077 master_service.cc:946] ReplaceTablet: received request to replace tablet 9b3b332e2c7c4af4a3ce9764d314217f from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:16.980630 13772 raft_consensus.cc:348] T b6445667835142acb08a24889415284d P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:17.023197 13772 raft_consensus.cc:374] T b6445667835142acb08a24889415284d P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:17.030336 13772 raft_consensus.cc:729] T b6445667835142acb08a24889415284d P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:17.045044 13073 catalog_manager.cc:5587] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327) does not have the latest schema for tablet 56559fc1c66e41fd8d6e1e767d4dd7cc (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]). Expected version 1 got 0
W20260925 09:58:17.047766 13073 catalog_manager.cc:5687] Failed to send AlterTable RPC for tablet 56559fc1c66e41fd8d6e1e767d4dd7cc (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 56559fc1c66e41fd8d6e1e767d4dd7cc consensus state has no leader
I20260925 09:58:17.038461 13772 consensus_queue.cc:260] T b6445667835142acb08a24889415284d P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:17.053354 13770 tablet_bootstrap.cc:492] T 56559fc1c66e41fd8d6e1e767d4dd7cc P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:17.053854 13770 ts_tablet_manager.cc:1397] T 56559fc1c66e41fd8d6e1e767d4dd7cc P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.142s user 0.023s sys 0.036s
I20260925 09:58:17.057189 13770 raft_consensus.cc:348] T 56559fc1c66e41fd8d6e1e767d4dd7cc P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:17.058166 13770 raft_consensus.cc:374] T 56559fc1c66e41fd8d6e1e767d4dd7cc P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:17.058488 13770 raft_consensus.cc:729] T 56559fc1c66e41fd8d6e1e767d4dd7cc P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:17.072084 13772 ts_tablet_manager.cc:1428] T b6445667835142acb08a24889415284d P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.176s user 0.007s sys 0.000s
I20260925 09:58:17.069587 13770 consensus_queue.cc:260] T 56559fc1c66e41fd8d6e1e767d4dd7cc P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:17.077893 13770 ts_tablet_manager.cc:1428] T 56559fc1c66e41fd8d6e1e767d4dd7cc P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.024s user 0.007s sys 0.001s
I20260925 09:58:17.105010 13772 tablet_bootstrap.cc:492] T ffd52766626d4d0691b4093659f3918e P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:17.138715 13077 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet 9b3b332e2c7c4af4a3ce9764d314217f
I20260925 09:58:17.141207 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b3b332e2c7c4af4a3ce9764d314217f with delete_type TABLET_DATA_DELETED (replaced by tablet 761a36153d004cef86126c719cfe7946) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:17.143361 13485 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b3b332e2c7c4af4a3ce9764d314217f with delete_type TABLET_DATA_DELETED (replaced by tablet 761a36153d004cef86126c719cfe7946) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:17.144143 13781 tablet_replica.cc:330] T 9b3b332e2c7c4af4a3ce9764d314217f P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:17.144863 13781 raft_consensus.cc:2229] T 9b3b332e2c7c4af4a3ce9764d314217f P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:17.145349 13781 raft_consensus.cc:2258] T 9b3b332e2c7c4af4a3ce9764d314217f P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:17.145742 13649 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b3b332e2c7c4af4a3ce9764d314217f with delete_type TABLET_DATA_DELETED (replaced by tablet 761a36153d004cef86126c719cfe7946) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:17.146761 13077 catalog_manager.cc:6565] ReplaceTablet: tablet 9b3b332e2c7c4af4a3ce9764d314217f deleted and replaced by tablet 761a36153d004cef86126c719cfe7946
I20260925 09:58:17.150298 13077 catalog_manager.cc:5587] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731) does not have the latest schema for tablet 56559fc1c66e41fd8d6e1e767d4dd7cc (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]). Expected version 1 got 0
I20260925 09:58:17.152987 13781 ts_tablet_manager.cc:1905] T 9b3b332e2c7c4af4a3ce9764d314217f P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:17.201618 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet b6445667835142acb08a24889415284d with delete_type TABLET_DATA_DELETED (replaced by tablet 56559fc1c66e41fd8d6e1e767d4dd7cc) from {username='slave'} at 127.0.0.1:49260
W20260925 09:58:17.218889 13077 catalog_manager.cc:5328] Ignoring report from unknown tablet b6445667835142acb08a24889415284d
I20260925 09:58:17.233325 13777 ts_tablet_manager.cc:1918] T 7098bdf9a9fc4234811ffdb29d43b1d6 P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:17.233863 13777 log.cc:1196] T 7098bdf9a9fc4234811ffdb29d43b1d6 P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/7098bdf9a9fc4234811ffdb29d43b1d6
I20260925 09:58:17.240136 13772 tablet_bootstrap.cc:654] T ffd52766626d4d0691b4093659f3918e P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:17.250936 13780 ts_tablet_manager.cc:1918] T b6445667835142acb08a24889415284d P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:17.251502 13780 log.cc:1196] T b6445667835142acb08a24889415284d P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/b6445667835142acb08a24889415284d
I20260925 09:58:17.253283 13780 ts_tablet_manager.cc:1939] T b6445667835142acb08a24889415284d P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:17.279749 13077 master_service.cc:946] ReplaceTablet: received request to replace tablet 7c004b586c60494fae35ebdbe2308f4c from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:17.283588 13777 ts_tablet_manager.cc:1939] T 7098bdf9a9fc4234811ffdb29d43b1d6 P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:17.286284 13772 tablet_bootstrap.cc:492] T ffd52766626d4d0691b4093659f3918e P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:17.286888 13772 ts_tablet_manager.cc:1397] T ffd52766626d4d0691b4093659f3918e P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.182s user 0.008s sys 0.001s
I20260925 09:58:17.288434 13781 ts_tablet_manager.cc:1918] T 9b3b332e2c7c4af4a3ce9764d314217f P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:17.288928 13781 log.cc:1196] T 9b3b332e2c7c4af4a3ce9764d314217f P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/9b3b332e2c7c4af4a3ce9764d314217f
I20260925 09:58:17.289762 13781 ts_tablet_manager.cc:1939] T 9b3b332e2c7c4af4a3ce9764d314217f P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:17.290519 13777 tablet_replica.cc:330] T 9b3b332e2c7c4af4a3ce9764d314217f P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:17.290647 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet 7098bdf9a9fc4234811ffdb29d43b1d6 (table default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]) successfully deleted
I20260925 09:58:17.291203 13777 raft_consensus.cc:2229] T 9b3b332e2c7c4af4a3ce9764d314217f P b3082cf15dc74c0480e5d33ecf592843 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:58:17.291877 13777 pending_rounds.cc:62] T 9b3b332e2c7c4af4a3ce9764d314217f P b3082cf15dc74c0480e5d33ecf592843: Trying to abort 1 pending ops.
I20260925 09:58:17.292109 13777 pending_rounds.cc:68] T 9b3b332e2c7c4af4a3ce9764d314217f P b3082cf15dc74c0480e5d33ecf592843: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192895397478400 op_type: NO_OP noop_request { }
I20260925 09:58:17.292523 13777 raft_consensus.cc:2866] T 9b3b332e2c7c4af4a3ce9764d314217f P b3082cf15dc74c0480e5d33ecf592843 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:58:17.292922 13777 raft_consensus.cc:2258] T 9b3b332e2c7c4af4a3ce9764d314217f P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:17.300738 13777 ts_tablet_manager.cc:1905] T 9b3b332e2c7c4af4a3ce9764d314217f P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:17.301554 13649 tablet_service.cc:1467] Processing CreateTablet for tablet 761a36153d004cef86126c719cfe7946 (DEFAULT_TABLE table=default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:17.302814 13649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 761a36153d004cef86126c719cfe7946. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:17.303243 13485 tablet_service.cc:1467] Processing CreateTablet for tablet 761a36153d004cef86126c719cfe7946 (DEFAULT_TABLE table=default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:17.304523 13485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 761a36153d004cef86126c719cfe7946. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:17.306133 13780 tablet_replica.cc:330] T 9b3b332e2c7c4af4a3ce9764d314217f P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:17.306419 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet b6445667835142acb08a24889415284d (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]) successfully deleted
I20260925 09:58:17.306778 13780 raft_consensus.cc:2229] T 9b3b332e2c7c4af4a3ce9764d314217f P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:17.307174 13780 raft_consensus.cc:2258] T 9b3b332e2c7c4af4a3ce9764d314217f P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:17.309378 13780 ts_tablet_manager.cc:1905] T 9b3b332e2c7c4af4a3ce9764d314217f P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:17.313194 13772 raft_consensus.cc:348] T ffd52766626d4d0691b4093659f3918e P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:17.313876 13772 raft_consensus.cc:374] T ffd52766626d4d0691b4093659f3918e P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:17.314116 13772 raft_consensus.cc:729] T ffd52766626d4d0691b4093659f3918e P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:17.314702 13772 consensus_queue.cc:260] T ffd52766626d4d0691b4093659f3918e P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:17.315301 13073 catalog_manager.cc:5587] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327) does not have the latest schema for tablet 56559fc1c66e41fd8d6e1e767d4dd7cc (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]). Expected version 1 got 0
I20260925 09:58:17.314731 13343 tablet_service.cc:1467] Processing CreateTablet for tablet 761a36153d004cef86126c719cfe7946 (DEFAULT_TABLE table=default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:17.317831 13343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 761a36153d004cef86126c719cfe7946. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:17.319182 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet 9b3b332e2c7c4af4a3ce9764d314217f (table default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477]) successfully deleted
I20260925 09:58:17.326193 13777 ts_tablet_manager.cc:1918] T 9b3b332e2c7c4af4a3ce9764d314217f P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:17.326543 13777 log.cc:1196] T 9b3b332e2c7c4af4a3ce9764d314217f P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/9b3b332e2c7c4af4a3ce9764d314217f
I20260925 09:58:17.327193 13777 ts_tablet_manager.cc:1939] T 9b3b332e2c7c4af4a3ce9764d314217f P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:17.328040 13077 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet 7c004b586c60494fae35ebdbe2308f4c
I20260925 09:58:17.329332 13777 tablet_replica.cc:330] T b6445667835142acb08a24889415284d P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:17.329438 13077 catalog_manager.cc:6565] ReplaceTablet: tablet 7c004b586c60494fae35ebdbe2308f4c deleted and replaced by tablet bb7bd9e45a6049518cff38bf28e2cbc2
I20260925 09:58:17.329910 13777 raft_consensus.cc:2229] T b6445667835142acb08a24889415284d P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:17.330308 13777 raft_consensus.cc:2258] T b6445667835142acb08a24889415284d P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:17.331434 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet 9b3b332e2c7c4af4a3ce9764d314217f (table default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477]) successfully deleted
I20260925 09:58:17.334121 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c004b586c60494fae35ebdbe2308f4c with delete_type TABLET_DATA_DELETED (replaced by tablet bb7bd9e45a6049518cff38bf28e2cbc2) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:17.336912 13772 ts_tablet_manager.cc:1428] T ffd52766626d4d0691b4093659f3918e P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.050s user 0.005s sys 0.003s
I20260925 09:58:17.338164 13772 tablet_bootstrap.cc:492] T 56559fc1c66e41fd8d6e1e767d4dd7cc P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:17.342630 13777 ts_tablet_manager.cc:1905] T b6445667835142acb08a24889415284d P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:17.346544 13483 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c004b586c60494fae35ebdbe2308f4c with delete_type TABLET_DATA_DELETED (replaced by tablet bb7bd9e45a6049518cff38bf28e2cbc2) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:17.347265 13781 tablet_replica.cc:330] T 7c004b586c60494fae35ebdbe2308f4c P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:17.347970 13781 raft_consensus.cc:2229] T 7c004b586c60494fae35ebdbe2308f4c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:17.348657 13781 raft_consensus.cc:2258] T 7c004b586c60494fae35ebdbe2308f4c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:17.350980 13781 ts_tablet_manager.cc:1905] T 7c004b586c60494fae35ebdbe2308f4c P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:17.356880 13073 master_service.cc:946] ReplaceTablet: received request to replace tablet ffd52766626d4d0691b4093659f3918e from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:17.359968 13773 tablet_bootstrap.cc:492] T 761a36153d004cef86126c719cfe7946 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:17.365365 13773 tablet_bootstrap.cc:654] T 761a36153d004cef86126c719cfe7946 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:17.375605 13777 ts_tablet_manager.cc:1918] T b6445667835142acb08a24889415284d P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:17.376313 13777 log.cc:1196] T b6445667835142acb08a24889415284d P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/b6445667835142acb08a24889415284d
I20260925 09:58:17.377070 13777 ts_tablet_manager.cc:1939] T b6445667835142acb08a24889415284d P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:17.379447 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet b6445667835142acb08a24889415284d (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]) successfully deleted
I20260925 09:58:17.383957 13777 tablet_replica.cc:330] T 7c004b586c60494fae35ebdbe2308f4c P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:17.384543 13777 raft_consensus.cc:2229] T 7c004b586c60494fae35ebdbe2308f4c P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:17.384997 13777 raft_consensus.cc:2258] T 7c004b586c60494fae35ebdbe2308f4c P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:17.386584 13777 ts_tablet_manager.cc:1905] T 7c004b586c60494fae35ebdbe2308f4c P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:17.397845 13773 tablet_bootstrap.cc:492] T 761a36153d004cef86126c719cfe7946 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:17.398433 13773 ts_tablet_manager.cc:1397] T 761a36153d004cef86126c719cfe7946 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.039s user 0.005s sys 0.003s
I20260925 09:58:17.401889 13773 raft_consensus.cc:348] T 761a36153d004cef86126c719cfe7946 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:17.402568 13773 raft_consensus.cc:374] T 761a36153d004cef86126c719cfe7946 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:17.402802 13773 raft_consensus.cc:729] T 761a36153d004cef86126c719cfe7946 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:17.403507 13773 consensus_queue.cc:260] T 761a36153d004cef86126c719cfe7946 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:17.410441 13777 ts_tablet_manager.cc:1918] T 7c004b586c60494fae35ebdbe2308f4c P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:17.410784 13777 log.cc:1196] T 7c004b586c60494fae35ebdbe2308f4c P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/7c004b586c60494fae35ebdbe2308f4c
I20260925 09:58:17.411466 13777 ts_tablet_manager.cc:1939] T 7c004b586c60494fae35ebdbe2308f4c P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:17.411545 13342 tablet_service.cc:1467] Processing CreateTablet for tablet bb7bd9e45a6049518cff38bf28e2cbc2 (DEFAULT_TABLE table=default.table_7be114188e08442db683500d68eafdfd [id=07bcde2a154b4eb2b4ee526d680b5ceb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:17.412766 13342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb7bd9e45a6049518cff38bf28e2cbc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:17.417363 13848 raft_consensus.cc:482] T ffd52766626d4d0691b4093659f3918e P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:17.417938 13848 raft_consensus.cc:504] T ffd52766626d4d0691b4093659f3918e P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
W20260925 09:58:17.421499 13603 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.1:42161, user_credentials={real_user=slave}} blocked reactor thread for 104605us
I20260925 09:58:17.422130 13362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffd52766626d4d0691b4093659f3918e" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:17.422828 13362 raft_consensus.cc:2454] T ffd52766626d4d0691b4093659f3918e P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 0.
I20260925 09:58:17.423138 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffd52766626d4d0691b4093659f3918e" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:17.423816 13671 raft_consensus.cc:2454] T ffd52766626d4d0691b4093659f3918e P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 0.
I20260925 09:58:17.424089 13438 leader_election.cc:302] T ffd52766626d4d0691b4093659f3918e P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:17.425230 13650 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c004b586c60494fae35ebdbe2308f4c with delete_type TABLET_DATA_DELETED (replaced by tablet bb7bd9e45a6049518cff38bf28e2cbc2) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:17.425247 13848 leader_election.cc:288] T ffd52766626d4d0691b4093659f3918e P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:17.425753 13848 raft_consensus.cc:2781] T ffd52766626d4d0691b4093659f3918e P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:17.426069 13848 raft_consensus.cc:482] T ffd52766626d4d0691b4093659f3918e P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:17.426517 13848 raft_consensus.cc:3037] T ffd52766626d4d0691b4093659f3918e P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:17.433852 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet 7c004b586c60494fae35ebdbe2308f4c (table default.table_7be114188e08442db683500d68eafdfd [id=07bcde2a154b4eb2b4ee526d680b5ceb]) successfully deleted
I20260925 09:58:17.442268 13485 tablet_service.cc:1467] Processing CreateTablet for tablet bb7bd9e45a6049518cff38bf28e2cbc2 (DEFAULT_TABLE table=default.table_7be114188e08442db683500d68eafdfd [id=07bcde2a154b4eb2b4ee526d680b5ceb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:17.443449 13485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb7bd9e45a6049518cff38bf28e2cbc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:17.443861 13650 tablet_service.cc:1467] Processing CreateTablet for tablet bb7bd9e45a6049518cff38bf28e2cbc2 (DEFAULT_TABLE table=default.table_7be114188e08442db683500d68eafdfd [id=07bcde2a154b4eb2b4ee526d680b5ceb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:17.444886 13650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb7bd9e45a6049518cff38bf28e2cbc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:17.448153 13772 tablet_bootstrap.cc:654] T 56559fc1c66e41fd8d6e1e767d4dd7cc P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:17.454797 13848 raft_consensus.cc:504] T ffd52766626d4d0691b4093659f3918e P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:17.457818 13362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffd52766626d4d0691b4093659f3918e" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843"
I20260925 09:58:17.458455 13362 raft_consensus.cc:3037] T ffd52766626d4d0691b4093659f3918e P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:17.463299 13073 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet ffd52766626d4d0691b4093659f3918e
I20260925 09:58:17.464980 13073 catalog_manager.cc:6565] ReplaceTablet: tablet ffd52766626d4d0691b4093659f3918e deleted and replaced by tablet af6ace3011c84603b32d0a409d5b8c1c
I20260925 09:58:17.465306 13343 tablet_service.cc:1514] Processing DeleteTablet for tablet ffd52766626d4d0691b4093659f3918e with delete_type TABLET_DATA_DELETED (replaced by tablet af6ace3011c84603b32d0a409d5b8c1c) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:17.465960 13777 tablet_replica.cc:330] T ffd52766626d4d0691b4093659f3918e P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:17.466027 13362 raft_consensus.cc:2454] T ffd52766626d4d0691b4093659f3918e P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:17.466512 13777 raft_consensus.cc:2229] T ffd52766626d4d0691b4093659f3918e P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:17.466889 13777 raft_consensus.cc:2258] T ffd52766626d4d0691b4093659f3918e P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:17.471926 13777 ts_tablet_manager.cc:1905] T ffd52766626d4d0691b4093659f3918e P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:17.475723 13651 tablet_service.cc:1514] Processing DeleteTablet for tablet ffd52766626d4d0691b4093659f3918e with delete_type TABLET_DATA_DELETED (replaced by tablet af6ace3011c84603b32d0a409d5b8c1c) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:17.476547 13483 tablet_service.cc:1514] Processing DeleteTablet for tablet ffd52766626d4d0691b4093659f3918e with delete_type TABLET_DATA_DELETED (replaced by tablet af6ace3011c84603b32d0a409d5b8c1c) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:17.477433 13438 leader_election.cc:302] T ffd52766626d4d0691b4093659f3918e P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:17.482391 13892 raft_consensus.cc:2781] T ffd52766626d4d0691b4093659f3918e P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:17.484793 13770 tablet_bootstrap.cc:492] T 761a36153d004cef86126c719cfe7946 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:17.489714 13770 tablet_bootstrap.cc:654] T 761a36153d004cef86126c719cfe7946 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:17.495229 13894 raft_consensus.cc:482] T 761a36153d004cef86126c719cfe7946 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:17.497550 13773 ts_tablet_manager.cc:1428] T 761a36153d004cef86126c719cfe7946 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.099s user 0.006s sys 0.001s
I20260925 09:58:17.497900 13892 raft_consensus.cc:686] T ffd52766626d4d0691b4093659f3918e P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: Becoming Leader. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Running, Role: LEADER
I20260925 09:58:17.498926 13777 ts_tablet_manager.cc:1918] T ffd52766626d4d0691b4093659f3918e P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:17.499317 13892 consensus_queue.cc:237] T ffd52766626d4d0691b4093659f3918e P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:17.496290 13894 raft_consensus.cc:504] T 761a36153d004cef86126c719cfe7946 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:17.503648 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffd52766626d4d0691b4093659f3918e" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
I20260925 09:58:17.504324 13671 raft_consensus.cc:3037] T ffd52766626d4d0691b4093659f3918e P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:17.506816 13670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "761a36153d004cef86126c719cfe7946" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
W20260925 09:58:17.508500 13440 leader_election.cc:341] T 761a36153d004cef86126c719cfe7946 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:17.510087 13894 leader_election.cc:288] T 761a36153d004cef86126c719cfe7946 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:17.512719 13770 tablet_bootstrap.cc:492] T 761a36153d004cef86126c719cfe7946 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:17.513120 13770 ts_tablet_manager.cc:1397] T 761a36153d004cef86126c719cfe7946 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.028s user 0.007s sys 0.004s
I20260925 09:58:17.499285 13777 log.cc:1196] T ffd52766626d4d0691b4093659f3918e P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/ffd52766626d4d0691b4093659f3918e
I20260925 09:58:17.513618 13362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "761a36153d004cef86126c719cfe7946" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:17.514271 13777 ts_tablet_manager.cc:1939] T ffd52766626d4d0691b4093659f3918e P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
W20260925 09:58:17.515627 13438 leader_election.cc:341] T 761a36153d004cef86126c719cfe7946 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:17.516353 13438 leader_election.cc:302] T 761a36153d004cef86126c719cfe7946 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4d724d4b83e4d1e817a3a505dc24eb9; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, b3082cf15dc74c0480e5d33ecf592843
I20260925 09:58:17.517429 13894 raft_consensus.cc:2726] T 761a36153d004cef86126c719cfe7946 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:58:17.517527 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet ffd52766626d4d0691b4093659f3918e (table default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]) successfully deleted
I20260925 09:58:17.520749 13770 raft_consensus.cc:348] T 761a36153d004cef86126c719cfe7946 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:17.521863 13770 raft_consensus.cc:374] T 761a36153d004cef86126c719cfe7946 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:17.522315 13770 raft_consensus.cc:729] T 761a36153d004cef86126c719cfe7946 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:17.523471 13770 consensus_queue.cc:260] T 761a36153d004cef86126c719cfe7946 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:17.525833 13770 ts_tablet_manager.cc:1428] T 761a36153d004cef86126c719cfe7946 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
I20260925 09:58:17.533613 13772 tablet_bootstrap.cc:492] T 56559fc1c66e41fd8d6e1e767d4dd7cc P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:17.534235 13772 ts_tablet_manager.cc:1397] T 56559fc1c66e41fd8d6e1e767d4dd7cc P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.196s user 0.010s sys 0.000s
I20260925 09:58:17.540311 13770 tablet_bootstrap.cc:492] T bb7bd9e45a6049518cff38bf28e2cbc2 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:17.546243 13770 tablet_bootstrap.cc:654] T bb7bd9e45a6049518cff38bf28e2cbc2 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:17.551524 13848 leader_election.cc:288] T ffd52766626d4d0691b4093659f3918e P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Requested vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:17.566478 13073 master_service.cc:946] ReplaceTablet: received request to replace tablet 56559fc1c66e41fd8d6e1e767d4dd7cc from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:17.569103 13773 tablet_bootstrap.cc:492] T bb7bd9e45a6049518cff38bf28e2cbc2 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:17.527516 13671 raft_consensus.cc:2454] T ffd52766626d4d0691b4093659f3918e P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:17.577346 13772 raft_consensus.cc:348] T 56559fc1c66e41fd8d6e1e767d4dd7cc P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:17.578369 13772 raft_consensus.cc:374] T 56559fc1c66e41fd8d6e1e767d4dd7cc P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:17.578706 13772 raft_consensus.cc:729] T 56559fc1c66e41fd8d6e1e767d4dd7cc P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:17.580544 13780 ts_tablet_manager.cc:1918] T 9b3b332e2c7c4af4a3ce9764d314217f P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:17.581075 13780 log.cc:1196] T 9b3b332e2c7c4af4a3ce9764d314217f P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/9b3b332e2c7c4af4a3ce9764d314217f
I20260925 09:58:17.581976 13780 ts_tablet_manager.cc:1939] T 9b3b332e2c7c4af4a3ce9764d314217f P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:17.596870 13773 tablet_bootstrap.cc:654] T bb7bd9e45a6049518cff38bf28e2cbc2 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:17.601967 13781 ts_tablet_manager.cc:1918] T 7c004b586c60494fae35ebdbe2308f4c P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:17.602542 13781 log.cc:1196] T 7c004b586c60494fae35ebdbe2308f4c P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/7c004b586c60494fae35ebdbe2308f4c
I20260925 09:58:17.603556 13781 ts_tablet_manager.cc:1939] T 7c004b586c60494fae35ebdbe2308f4c P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:17.611738 13867 raft_consensus.cc:482] T 761a36153d004cef86126c719cfe7946 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:17.612198 13867 raft_consensus.cc:504] T 761a36153d004cef86126c719cfe7946 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:17.614163 13867 leader_election.cc:288] T 761a36153d004cef86126c719cfe7946 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
I20260925 09:58:17.616259 13362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "761a36153d004cef86126c719cfe7946" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:17.616685 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "761a36153d004cef86126c719cfe7946" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
I20260925 09:58:17.617286 13505 raft_consensus.cc:2454] T 761a36153d004cef86126c719cfe7946 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 0.
W20260925 09:58:17.618232 13603 leader_election.cc:341] T 761a36153d004cef86126c719cfe7946 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:17.621897 13342 tablet_service.cc:1467] Processing CreateTablet for tablet af6ace3011c84603b32d0a409d5b8c1c (DEFAULT_TABLE table=default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:17.622318 13606 leader_election.cc:302] T 761a36153d004cef86126c719cfe7946 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9; no voters: b3082cf15dc74c0480e5d33ecf592843
I20260925 09:58:17.622931 13342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af6ace3011c84603b32d0a409d5b8c1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:17.623486 13650 tablet_service.cc:1467] Processing CreateTablet for tablet af6ace3011c84603b32d0a409d5b8c1c (DEFAULT_TABLE table=default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:17.624380 13650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af6ace3011c84603b32d0a409d5b8c1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:17.625789 13485 tablet_service.cc:1467] Processing CreateTablet for tablet af6ace3011c84603b32d0a409d5b8c1c (DEFAULT_TABLE table=default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:17.626433 13867 raft_consensus.cc:2781] T 761a36153d004cef86126c719cfe7946 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:17.626761 13867 raft_consensus.cc:482] T 761a36153d004cef86126c719cfe7946 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:17.627051 13485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af6ace3011c84603b32d0a409d5b8c1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:17.627061 13867 raft_consensus.cc:3037] T 761a36153d004cef86126c719cfe7946 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:17.625608 13772 consensus_queue.cc:260] T 56559fc1c66e41fd8d6e1e767d4dd7cc P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:17.647763 13782 raft_consensus.cc:482] T 56559fc1c66e41fd8d6e1e767d4dd7cc P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:17.648288 13782 raft_consensus.cc:504] T 56559fc1c66e41fd8d6e1e767d4dd7cc P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:17.650344 13782 leader_election.cc:288] T 56559fc1c66e41fd8d6e1e767d4dd7cc P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:17.655763 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56559fc1c66e41fd8d6e1e767d4dd7cc" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
I20260925 09:58:17.656122 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56559fc1c66e41fd8d6e1e767d4dd7cc" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:17.656395 13505 raft_consensus.cc:2454] T 56559fc1c66e41fd8d6e1e767d4dd7cc P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 0.
I20260925 09:58:17.656778 13671 raft_consensus.cc:2454] T 56559fc1c66e41fd8d6e1e767d4dd7cc P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 0.
I20260925 09:58:17.657920 13298 leader_election.cc:302] T 56559fc1c66e41fd8d6e1e767d4dd7cc P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:17.659773 13782 raft_consensus.cc:2781] T 56559fc1c66e41fd8d6e1e767d4dd7cc P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:17.660208 13782 raft_consensus.cc:482] T 56559fc1c66e41fd8d6e1e767d4dd7cc P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:17.660604 13782 raft_consensus.cc:3037] T 56559fc1c66e41fd8d6e1e767d4dd7cc P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:17.665565 13867 raft_consensus.cc:504] T 761a36153d004cef86126c719cfe7946 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:17.666571 13073 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet 56559fc1c66e41fd8d6e1e767d4dd7cc
I20260925 09:58:17.668546 13362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "761a36153d004cef86126c719cfe7946" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843"
I20260925 09:58:17.668915 13343 tablet_service.cc:1514] Processing DeleteTablet for tablet 56559fc1c66e41fd8d6e1e767d4dd7cc with delete_type TABLET_DATA_DELETED (replaced by tablet 64cfc93464ae4dc1bf6788105513fe8d) from {username='slave'} at 127.0.0.1:49260
W20260925 09:58:17.670672 13066 catalog_manager.cc:4821] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): delete failed for tablet 56559fc1c66e41fd8d6e1e767d4dd7cc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 56559fc1c66e41fd8d6e1e767d4dd7cc already in progress: creating tablet
I20260925 09:58:17.670796 13649 tablet_service.cc:1514] Processing DeleteTablet for tablet 56559fc1c66e41fd8d6e1e767d4dd7cc with delete_type TABLET_DATA_DELETED (replaced by tablet 64cfc93464ae4dc1bf6788105513fe8d) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:17.671550 13073 catalog_manager.cc:6565] ReplaceTablet: tablet 56559fc1c66e41fd8d6e1e767d4dd7cc deleted and replaced by tablet 64cfc93464ae4dc1bf6788105513fe8d
W20260925 09:58:17.672461 13603 leader_election.cc:341] T 761a36153d004cef86126c719cfe7946 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:17.681995 13485 tablet_service.cc:1514] Processing DeleteTablet for tablet 56559fc1c66e41fd8d6e1e767d4dd7cc with delete_type TABLET_DATA_DELETED (replaced by tablet 64cfc93464ae4dc1bf6788105513fe8d) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:17.684670 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "761a36153d004cef86126c719cfe7946" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9"
I20260925 09:58:17.685257 13505 raft_consensus.cc:3037] T 761a36153d004cef86126c719cfe7946 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:17.692162 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet 9b3b332e2c7c4af4a3ce9764d314217f (table default.table_d170c346194441ee8bff277382a7aea3 [id=ac75711cba134e34919b4106c864f477]) successfully deleted
I20260925 09:58:17.692544 13505 raft_consensus.cc:2454] T 761a36153d004cef86126c719cfe7946 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 1.
I20260925 09:58:17.693188 13780 tablet_replica.cc:330] T 7c004b586c60494fae35ebdbe2308f4c P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:17.693871 13780 raft_consensus.cc:2229] T 7c004b586c60494fae35ebdbe2308f4c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:58:17.693710 13606 leader_election.cc:302] T 761a36153d004cef86126c719cfe7946 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9; no voters: b3082cf15dc74c0480e5d33ecf592843
I20260925 09:58:17.694583 13890 raft_consensus.cc:2781] T 761a36153d004cef86126c719cfe7946 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:17.694852 13780 raft_consensus.cc:2258] T 7c004b586c60494fae35ebdbe2308f4c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:17.695021 13890 raft_consensus.cc:686] T 761a36153d004cef86126c719cfe7946 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 LEADER]: Becoming Leader. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Running, Role: LEADER
I20260925 09:58:17.695819 13890 consensus_queue.cc:237] T 761a36153d004cef86126c719cfe7946 P a06c3ef7a6a94f7cbcc03e7cd232f237 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:17.696727 13780 ts_tablet_manager.cc:1905] T 7c004b586c60494fae35ebdbe2308f4c P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:17.704581 13772 ts_tablet_manager.cc:1428] T 56559fc1c66e41fd8d6e1e767d4dd7cc P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.170s user 0.010s sys 0.000s
I20260925 09:58:17.705946 13772 tablet_bootstrap.cc:492] T 761a36153d004cef86126c719cfe7946 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:17.706514 13867 leader_election.cc:288] T 761a36153d004cef86126c719cfe7946 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 election: Requested vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
I20260925 09:58:17.709815 13770 tablet_bootstrap.cc:492] T bb7bd9e45a6049518cff38bf28e2cbc2 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:17.710289 13770 ts_tablet_manager.cc:1397] T bb7bd9e45a6049518cff38bf28e2cbc2 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.170s user 0.013s sys 0.007s
I20260925 09:58:17.712554 13770 raft_consensus.cc:348] T bb7bd9e45a6049518cff38bf28e2cbc2 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:17.713234 13770 raft_consensus.cc:374] T bb7bd9e45a6049518cff38bf28e2cbc2 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:17.713354 13077 master_service.cc:946] ReplaceTablet: received request to replace tablet 56559fc1c66e41fd8d6e1e767d4dd7cc from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:17.713370 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet 56559fc1c66e41fd8d6e1e767d4dd7cc with delete_type TABLET_DATA_DELETED (replaced by tablet 64cfc93464ae4dc1bf6788105513fe8d) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:17.713485 13770 raft_consensus.cc:729] T bb7bd9e45a6049518cff38bf28e2cbc2 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:17.714041 13777 tablet_replica.cc:330] T 56559fc1c66e41fd8d6e1e767d4dd7cc P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:17.714242 13770 consensus_queue.cc:260] T bb7bd9e45a6049518cff38bf28e2cbc2 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:17.716326 13770 ts_tablet_manager.cc:1428] T bb7bd9e45a6049518cff38bf28e2cbc2 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260925 09:58:17.717095 13770 tablet_bootstrap.cc:492] T af6ace3011c84603b32d0a409d5b8c1c P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:17.722139 13781 tablet_replica.cc:330] T ffd52766626d4d0691b4093659f3918e P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:17.722922 13781 raft_consensus.cc:2229] T ffd52766626d4d0691b4093659f3918e P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:58:17.723634 13781 pending_rounds.cc:62] T ffd52766626d4d0691b4093659f3918e P c4d724d4b83e4d1e817a3a505dc24eb9: Trying to abort 1 pending ops.
I20260925 09:58:17.723891 13781 pending_rounds.cc:68] T ffd52766626d4d0691b4093659f3918e P c4d724d4b83e4d1e817a3a505dc24eb9: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192898567061504 op_type: NO_OP noop_request { }
I20260925 09:58:17.724275 13781 raft_consensus.cc:2866] T ffd52766626d4d0691b4093659f3918e P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:58:17.724670 13781 raft_consensus.cc:2258] T ffd52766626d4d0691b4093659f3918e P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:17.725530 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet 7c004b586c60494fae35ebdbe2308f4c (table default.table_7be114188e08442db683500d68eafdfd [id=07bcde2a154b4eb2b4ee526d680b5ceb]) successfully deleted
I20260925 09:58:17.681715 13782 raft_consensus.cc:504] T 56559fc1c66e41fd8d6e1e767d4dd7cc P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:17.726399 13777 raft_consensus.cc:2229] T 56559fc1c66e41fd8d6e1e767d4dd7cc P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:17.728811 13777 raft_consensus.cc:2258] T 56559fc1c66e41fd8d6e1e767d4dd7cc P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:17.731309 13782 leader_election.cc:288] T 56559fc1c66e41fd8d6e1e767d4dd7cc P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Requested vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:17.732728 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56559fc1c66e41fd8d6e1e767d4dd7cc" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9"
I20260925 09:58:17.733390 13505 raft_consensus.cc:3037] T 56559fc1c66e41fd8d6e1e767d4dd7cc P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:17.741151 13777 ts_tablet_manager.cc:1905] T 56559fc1c66e41fd8d6e1e767d4dd7cc P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:17.741703 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56559fc1c66e41fd8d6e1e767d4dd7cc" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
I20260925 09:58:17.742276 13671 raft_consensus.cc:3037] T 56559fc1c66e41fd8d6e1e767d4dd7cc P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:17.747892 13770 tablet_bootstrap.cc:654] T af6ace3011c84603b32d0a409d5b8c1c P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:17.754871 13505 raft_consensus.cc:2454] T 56559fc1c66e41fd8d6e1e767d4dd7cc P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 1.
I20260925 09:58:17.755677 13671 raft_consensus.cc:2454] T 56559fc1c66e41fd8d6e1e767d4dd7cc P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 1.
I20260925 09:58:17.756390 13298 leader_election.cc:302] T 56559fc1c66e41fd8d6e1e767d4dd7cc P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
W20260925 09:58:17.757032 13298 raft_consensus.cc:2670] T 56559fc1c66e41fd8d6e1e767d4dd7cc P b3082cf15dc74c0480e5d33ecf592843: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260925 09:58:17.761018 13770 tablet_bootstrap.cc:492] T af6ace3011c84603b32d0a409d5b8c1c P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:17.761409 13770 ts_tablet_manager.cc:1397] T af6ace3011c84603b32d0a409d5b8c1c P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.044s user 0.009s sys 0.000s
I20260925 09:58:17.763788 13770 raft_consensus.cc:348] T af6ace3011c84603b32d0a409d5b8c1c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:17.764415 13770 raft_consensus.cc:374] T af6ace3011c84603b32d0a409d5b8c1c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:17.764681 13770 raft_consensus.cc:729] T af6ace3011c84603b32d0a409d5b8c1c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:17.765302 13773 tablet_bootstrap.cc:492] T bb7bd9e45a6049518cff38bf28e2cbc2 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:17.765758 13773 ts_tablet_manager.cc:1397] T bb7bd9e45a6049518cff38bf28e2cbc2 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.197s user 0.012s sys 0.002s
I20260925 09:58:17.765434 13770 consensus_queue.cc:260] T af6ace3011c84603b32d0a409d5b8c1c P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:17.771795 13770 ts_tablet_manager.cc:1428] T af6ace3011c84603b32d0a409d5b8c1c P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.010s user 0.006s sys 0.001s
I20260925 09:58:17.779588 13772 tablet_bootstrap.cc:654] T 761a36153d004cef86126c719cfe7946 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:17.777545 13781 ts_tablet_manager.cc:1905] T ffd52766626d4d0691b4093659f3918e P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:17.771040 13773 raft_consensus.cc:348] T bb7bd9e45a6049518cff38bf28e2cbc2 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:17.783800 13773 raft_consensus.cc:374] T bb7bd9e45a6049518cff38bf28e2cbc2 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:17.784160 13773 raft_consensus.cc:729] T bb7bd9e45a6049518cff38bf28e2cbc2 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:17.784885 13773 consensus_queue.cc:260] T bb7bd9e45a6049518cff38bf28e2cbc2 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:17.790261 13773 ts_tablet_manager.cc:1428] T bb7bd9e45a6049518cff38bf28e2cbc2 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.024s user 0.005s sys 0.000s
W20260925 09:58:17.811218 13067 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet b6445667835142acb08a24889415284d (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet b6445667835142acb08a24889415284d consensus state has no leader
I20260925 09:58:17.812605 13073 catalog_manager.cc:5495] T 761a36153d004cef86126c719cfe7946 P a06c3ef7a6a94f7cbcc03e7cd232f237 reported cstate change: term changed from 0 to 1, leader changed from <none> to a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3). New cstate: current_term: 1 leader_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: HEALTHY } } }
I20260925 09:58:17.820025 13781 ts_tablet_manager.cc:1918] T ffd52766626d4d0691b4093659f3918e P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:17.820436 13781 log.cc:1196] T ffd52766626d4d0691b4093659f3918e P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/ffd52766626d4d0691b4093659f3918e
I20260925 09:58:17.821128 13781 ts_tablet_manager.cc:1939] T ffd52766626d4d0691b4093659f3918e P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:17.832763 13781 tablet_replica.cc:330] T 56559fc1c66e41fd8d6e1e767d4dd7cc P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:17.833896 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet ffd52766626d4d0691b4093659f3918e (table default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]) successfully deleted
I20260925 09:58:17.836330 13781 raft_consensus.cc:2229] T 56559fc1c66e41fd8d6e1e767d4dd7cc P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:17.837826 13781 raft_consensus.cc:2258] T 56559fc1c66e41fd8d6e1e767d4dd7cc P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:17.845635 13342 tablet_service.cc:1467] Processing CreateTablet for tablet 64cfc93464ae4dc1bf6788105513fe8d (DEFAULT_TABLE table=default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:17.846670 13342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64cfc93464ae4dc1bf6788105513fe8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:17.846602 13781 ts_tablet_manager.cc:1905] T 56559fc1c66e41fd8d6e1e767d4dd7cc P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:17.847023 13485 tablet_service.cc:1467] Processing CreateTablet for tablet 64cfc93464ae4dc1bf6788105513fe8d (DEFAULT_TABLE table=default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:17.847993 13485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64cfc93464ae4dc1bf6788105513fe8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:17.869521 13650 tablet_service.cc:1467] Processing CreateTablet for tablet 64cfc93464ae4dc1bf6788105513fe8d (DEFAULT_TABLE table=default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:17.870568 13650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64cfc93464ae4dc1bf6788105513fe8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:17.879006 13777 ts_tablet_manager.cc:1918] T 56559fc1c66e41fd8d6e1e767d4dd7cc P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:17.879491 13777 log.cc:1196] T 56559fc1c66e41fd8d6e1e767d4dd7cc P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/56559fc1c66e41fd8d6e1e767d4dd7cc
I20260925 09:58:17.880170 13777 ts_tablet_manager.cc:1939] T 56559fc1c66e41fd8d6e1e767d4dd7cc P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:17.883610 13077 master_service.cc:946] ReplaceTablet: received request to replace tablet bb7bd9e45a6049518cff38bf28e2cbc2 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:17.889293 13773 tablet_bootstrap.cc:492] T af6ace3011c84603b32d0a409d5b8c1c P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:17.889765 13781 ts_tablet_manager.cc:1918] T 56559fc1c66e41fd8d6e1e767d4dd7cc P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:17.890125 13781 log.cc:1196] T 56559fc1c66e41fd8d6e1e767d4dd7cc P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/56559fc1c66e41fd8d6e1e767d4dd7cc
I20260925 09:58:17.890727 13781 ts_tablet_manager.cc:1939] T 56559fc1c66e41fd8d6e1e767d4dd7cc P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:17.900602 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet 56559fc1c66e41fd8d6e1e767d4dd7cc (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]) successfully deleted
I20260925 09:58:17.907907 13773 tablet_bootstrap.cc:654] T af6ace3011c84603b32d0a409d5b8c1c P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:17.910585 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet 56559fc1c66e41fd8d6e1e767d4dd7cc (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]) successfully deleted
I20260925 09:58:17.897718 13772 tablet_bootstrap.cc:492] T 761a36153d004cef86126c719cfe7946 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:17.911593 13770 tablet_bootstrap.cc:492] T 64cfc93464ae4dc1bf6788105513fe8d P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:17.911731 13772 ts_tablet_manager.cc:1397] T 761a36153d004cef86126c719cfe7946 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.206s user 0.009s sys 0.000s
I20260925 09:58:17.914222 13772 raft_consensus.cc:348] T 761a36153d004cef86126c719cfe7946 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:17.914964 13772 raft_consensus.cc:374] T 761a36153d004cef86126c719cfe7946 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:17.916990 13770 tablet_bootstrap.cc:654] T 64cfc93464ae4dc1bf6788105513fe8d P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:17.915264 13772 raft_consensus.cc:729] T 761a36153d004cef86126c719cfe7946 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:17.925981 13770 tablet_bootstrap.cc:492] T 64cfc93464ae4dc1bf6788105513fe8d P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:17.926412 13770 ts_tablet_manager.cc:1397] T 64cfc93464ae4dc1bf6788105513fe8d P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.003s
I20260925 09:58:17.928687 13770 raft_consensus.cc:348] T 64cfc93464ae4dc1bf6788105513fe8d P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:17.929503 13770 raft_consensus.cc:374] T 64cfc93464ae4dc1bf6788105513fe8d P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:17.929802 13770 raft_consensus.cc:729] T 64cfc93464ae4dc1bf6788105513fe8d P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:17.930367 13770 consensus_queue.cc:260] T 64cfc93464ae4dc1bf6788105513fe8d P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:17.945312 13077 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet bb7bd9e45a6049518cff38bf28e2cbc2
I20260925 09:58:17.946985 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet bb7bd9e45a6049518cff38bf28e2cbc2 with delete_type TABLET_DATA_DELETED (replaced by tablet 780164f28e2e45e38cfb4b47d7a6584c) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:17.951128 13073 master_service.cc:946] ReplaceTablet: received request to replace tablet af6ace3011c84603b32d0a409d5b8c1c from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:17.954061 13867 raft_consensus.cc:482] T 64cfc93464ae4dc1bf6788105513fe8d P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:17.954515 13867 raft_consensus.cc:504] T 64cfc93464ae4dc1bf6788105513fe8d P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:17.956260 13650 tablet_service.cc:1514] Processing DeleteTablet for tablet bb7bd9e45a6049518cff38bf28e2cbc2 with delete_type TABLET_DATA_DELETED (replaced by tablet 780164f28e2e45e38cfb4b47d7a6584c) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:17.957864 13867 leader_election.cc:288] T 64cfc93464ae4dc1bf6788105513fe8d P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
I20260925 09:58:17.958577 13362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64cfc93464ae4dc1bf6788105513fe8d" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:17.959900 13770 ts_tablet_manager.cc:1428] T 64cfc93464ae4dc1bf6788105513fe8d P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.033s user 0.007s sys 0.001s
W20260925 09:58:17.959956 13603 leader_election.cc:341] T 64cfc93464ae4dc1bf6788105513fe8d P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:58:17.976375 13066 catalog_manager.cc:4821] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): delete failed for tablet bb7bd9e45a6049518cff38bf28e2cbc2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bb7bd9e45a6049518cff38bf28e2cbc2 already in progress: creating tablet
I20260925 09:58:17.922855 13772 consensus_queue.cc:260] T 761a36153d004cef86126c719cfe7946 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:17.987488 13077 catalog_manager.cc:6565] ReplaceTablet: tablet bb7bd9e45a6049518cff38bf28e2cbc2 deleted and replaced by tablet 780164f28e2e45e38cfb4b47d7a6584c
I20260925 09:58:17.950620 13483 tablet_service.cc:1514] Processing DeleteTablet for tablet bb7bd9e45a6049518cff38bf28e2cbc2 with delete_type TABLET_DATA_DELETED (replaced by tablet 780164f28e2e45e38cfb4b47d7a6584c) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:17.959309 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64cfc93464ae4dc1bf6788105513fe8d" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
I20260925 09:58:17.998561 13781 tablet_replica.cc:330] T bb7bd9e45a6049518cff38bf28e2cbc2 P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
W20260925 09:58:17.998898 13606 leader_election.cc:341] T 64cfc93464ae4dc1bf6788105513fe8d P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:17.999218 13781 raft_consensus.cc:2229] T bb7bd9e45a6049518cff38bf28e2cbc2 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:17.999327 13606 leader_election.cc:302] T 64cfc93464ae4dc1bf6788105513fe8d P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237; no voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:58:17.999888 13781 raft_consensus.cc:2258] T bb7bd9e45a6049518cff38bf28e2cbc2 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:18.000432 13867 raft_consensus.cc:2726] T 64cfc93464ae4dc1bf6788105513fe8d P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:58:18.001926 13781 ts_tablet_manager.cc:1905] T bb7bd9e45a6049518cff38bf28e2cbc2 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:18.015988 13772 ts_tablet_manager.cc:1428] T 761a36153d004cef86126c719cfe7946 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.104s user 0.005s sys 0.000s
W20260925 09:58:18.019505 13067 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.9.60:42083: connect: Connection refused (error 111) [suppressed 9 similar messages]
W20260925 09:58:18.055629 13267 hms_client.cc:305] Time spent drop HMS table: real 2.625s user 0.000s sys 0.000s
I20260925 09:58:18.056082 13081 catalog_manager.cc:2685] deleted HMS catalog entry for table default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]
I20260925 09:58:18.063372 13073 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet af6ace3011c84603b32d0a409d5b8c1c
I20260925 09:58:18.065528 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet af6ace3011c84603b32d0a409d5b8c1c with delete_type TABLET_DATA_DELETED (replaced by tablet 1c8dce72450545e0b304ee41e01bff34) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:18.066658 13650 tablet_service.cc:1514] Processing DeleteTablet for tablet af6ace3011c84603b32d0a409d5b8c1c with delete_type TABLET_DATA_DELETED (replaced by tablet 1c8dce72450545e0b304ee41e01bff34) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:18.068120 13483 tablet_service.cc:1514] Processing DeleteTablet for tablet af6ace3011c84603b32d0a409d5b8c1c with delete_type TABLET_DATA_DELETED (replaced by tablet 1c8dce72450545e0b304ee41e01bff34) from {username='slave'} at 127.0.0.1:44532
W20260925 09:58:18.068500 13066 catalog_manager.cc:4821] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): delete failed for tablet af6ace3011c84603b32d0a409d5b8c1c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet af6ace3011c84603b32d0a409d5b8c1c already in progress: creating tablet
I20260925 09:58:18.070266 13073 catalog_manager.cc:6565] ReplaceTablet: tablet af6ace3011c84603b32d0a409d5b8c1c deleted and replaced by tablet 1c8dce72450545e0b304ee41e01bff34
I20260925 09:58:18.071177 13772 tablet_bootstrap.cc:492] T bb7bd9e45a6049518cff38bf28e2cbc2 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:18.078068 13781 ts_tablet_manager.cc:1918] T bb7bd9e45a6049518cff38bf28e2cbc2 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:18.078444 13781 log.cc:1196] T bb7bd9e45a6049518cff38bf28e2cbc2 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/bb7bd9e45a6049518cff38bf28e2cbc2
I20260925 09:58:18.079067 13781 ts_tablet_manager.cc:1939] T bb7bd9e45a6049518cff38bf28e2cbc2 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
W20260925 09:58:18.076474 13067 consensus_peers.cc:487] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 -> Peer 61e29afab0cf46819cbffc91248bf761 (127.10.9.60:42083): Couldn't send request to peer 61e29afab0cf46819cbffc91248bf761. Status: Network error: Client connection negotiation failed: client connection to 127.10.9.60:42083: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260925 09:58:18.080687 13773 tablet_bootstrap.cc:492] T af6ace3011c84603b32d0a409d5b8c1c P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:18.081111 13773 ts_tablet_manager.cc:1397] T af6ace3011c84603b32d0a409d5b8c1c P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.192s user 0.007s sys 0.001s
I20260925 09:58:18.086963 13780 ts_tablet_manager.cc:1918] T 7c004b586c60494fae35ebdbe2308f4c P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:18.087442 13780 log.cc:1196] T 7c004b586c60494fae35ebdbe2308f4c P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/7c004b586c60494fae35ebdbe2308f4c
I20260925 09:58:18.088193 13780 ts_tablet_manager.cc:1939] T 7c004b586c60494fae35ebdbe2308f4c P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:18.090458 13780 tablet_replica.cc:330] T ffd52766626d4d0691b4093659f3918e P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:18.094103 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet 7c004b586c60494fae35ebdbe2308f4c (table default.table_7be114188e08442db683500d68eafdfd [id=07bcde2a154b4eb2b4ee526d680b5ceb]) successfully deleted
I20260925 09:58:18.106982 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet bb7bd9e45a6049518cff38bf28e2cbc2 (table default.table_7be114188e08442db683500d68eafdfd [id=07bcde2a154b4eb2b4ee526d680b5ceb]) successfully deleted
I20260925 09:58:18.107254 13773 raft_consensus.cc:348] T af6ace3011c84603b32d0a409d5b8c1c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:18.108857 13780 raft_consensus.cc:2229] T ffd52766626d4d0691b4093659f3918e P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:18.109350 13780 raft_consensus.cc:2258] T ffd52766626d4d0691b4093659f3918e P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:18.108047 13773 raft_consensus.cc:374] T af6ace3011c84603b32d0a409d5b8c1c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:18.111064 13773 raft_consensus.cc:729] T af6ace3011c84603b32d0a409d5b8c1c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
W20260925 09:58:18.113186 13069 catalog_manager.cc:4821] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): delete failed for tablet af6ace3011c84603b32d0a409d5b8c1c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet af6ace3011c84603b32d0a409d5b8c1c already in progress: creating tablet
I20260925 09:58:18.111753 13773 consensus_queue.cc:260] T af6ace3011c84603b32d0a409d5b8c1c P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:18.115266 13772 tablet_bootstrap.cc:654] T bb7bd9e45a6049518cff38bf28e2cbc2 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:18.123564 13773 ts_tablet_manager.cc:1428] T af6ace3011c84603b32d0a409d5b8c1c P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.042s user 0.008s sys 0.000s
I20260925 09:58:18.124181 13780 ts_tablet_manager.cc:1905] T ffd52766626d4d0691b4093659f3918e P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:18.134191 13773 tablet_bootstrap.cc:492] T 64cfc93464ae4dc1bf6788105513fe8d P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:18.145471 13892 raft_consensus.cc:482] T af6ace3011c84603b32d0a409d5b8c1c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:18.146099 13892 raft_consensus.cc:504] T af6ace3011c84603b32d0a409d5b8c1c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:18.148077 13892 leader_election.cc:288] T af6ace3011c84603b32d0a409d5b8c1c P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:18.149983 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af6ace3011c84603b32d0a409d5b8c1c" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:18.150611 13671 raft_consensus.cc:2454] T af6ace3011c84603b32d0a409d5b8c1c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 0.
I20260925 09:58:18.152120 13440 leader_election.cc:302] T af6ace3011c84603b32d0a409d5b8c1c P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:18.153779 13892 raft_consensus.cc:2781] T af6ace3011c84603b32d0a409d5b8c1c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:18.153916 13362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af6ace3011c84603b32d0a409d5b8c1c" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:18.154170 13892 raft_consensus.cc:482] T af6ace3011c84603b32d0a409d5b8c1c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:18.154613 13892 raft_consensus.cc:3037] T af6ace3011c84603b32d0a409d5b8c1c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:58:18.156615 13438 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:18.157109 13773 tablet_bootstrap.cc:654] T 64cfc93464ae4dc1bf6788105513fe8d P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:18.157732 13077 catalog_manager.cc:5587] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731) does not have the latest schema for tablet 64cfc93464ae4dc1bf6788105513fe8d (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]). Expected version 1 got 0
W20260925 09:58:18.159379 13077 catalog_manager.cc:5687] Failed to send AlterTable RPC for tablet 64cfc93464ae4dc1bf6788105513fe8d (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 64cfc93464ae4dc1bf6788105513fe8d consensus state has no leader
I20260925 09:58:18.163960 13342 tablet_service.cc:1467] Processing CreateTablet for tablet 780164f28e2e45e38cfb4b47d7a6584c (DEFAULT_TABLE table=default.table_7be114188e08442db683500d68eafdfd [id=07bcde2a154b4eb2b4ee526d680b5ceb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:18.164902 13342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 780164f28e2e45e38cfb4b47d7a6584c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:18.166967 13892 raft_consensus.cc:504] T af6ace3011c84603b32d0a409d5b8c1c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:18.170404 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af6ace3011c84603b32d0a409d5b8c1c" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
I20260925 09:58:18.171048 13671 raft_consensus.cc:3037] T af6ace3011c84603b32d0a409d5b8c1c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:18.171581 13362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af6ace3011c84603b32d0a409d5b8c1c" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843"
W20260925 09:58:18.173578 13438 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:18.177738 13773 tablet_bootstrap.cc:492] T 64cfc93464ae4dc1bf6788105513fe8d P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:18.178169 13773 ts_tablet_manager.cc:1397] T 64cfc93464ae4dc1bf6788105513fe8d P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.045s user 0.009s sys 0.000s
I20260925 09:58:18.178371 13671 raft_consensus.cc:2454] T af6ace3011c84603b32d0a409d5b8c1c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:18.179603 13440 leader_election.cc:302] T af6ace3011c84603b32d0a409d5b8c1c P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9; no voters: b3082cf15dc74c0480e5d33ecf592843
I20260925 09:58:18.185871 13505 raft_consensus.cc:1261] T 761a36153d004cef86126c719cfe7946 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Refusing update from remote peer a06c3ef7a6a94f7cbcc03e7cd232f237: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:58:18.186141 13912 raft_consensus.cc:2781] T af6ace3011c84603b32d0a409d5b8c1c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:18.187739 13867 consensus_queue.cc:1035] T 761a36153d004cef86126c719cfe7946 P a06c3ef7a6a94f7cbcc03e7cd232f237 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:18.189527 13892 leader_election.cc:288] T af6ace3011c84603b32d0a409d5b8c1c P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Requested vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:18.192606 13773 raft_consensus.cc:348] T 64cfc93464ae4dc1bf6788105513fe8d P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:18.194981 13773 raft_consensus.cc:374] T 64cfc93464ae4dc1bf6788105513fe8d P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:18.195268 13773 raft_consensus.cc:729] T 64cfc93464ae4dc1bf6788105513fe8d P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:18.213542 13269 catalog_manager.cc:2709] Deleting table default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118] in response to Hive Metastore notification log event 7
I20260925 09:58:18.216789 13074 catalog_manager.cc:2148] created HMS catalog entry for table [id=7e0502ffa5544a5bb252a72020649cd7]
I20260925 09:58:18.221437 13773 consensus_queue.cc:260] T 64cfc93464ae4dc1bf6788105513fe8d P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:18.224193 13780 ts_tablet_manager.cc:1918] T ffd52766626d4d0691b4093659f3918e P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:18.224588 13780 log.cc:1196] T ffd52766626d4d0691b4093659f3918e P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/ffd52766626d4d0691b4093659f3918e
I20260925 09:58:18.225301 13780 ts_tablet_manager.cc:1939] T ffd52766626d4d0691b4093659f3918e P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:18.227905 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet ffd52766626d4d0691b4093659f3918e (table default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]) successfully deleted
I20260925 09:58:18.240454 13916 tablet.cc:1722] T 761a36153d004cef86126c719cfe7946 P a06c3ef7a6a94f7cbcc03e7cd232f237: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:58:18.241698 13917 tablet.cc:1722] T 761a36153d004cef86126c719cfe7946 P c4d724d4b83e4d1e817a3a505dc24eb9: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:58:18.245874 13074 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:50556 (request call id 12) took 4727 ms. Trace:
I20260925 09:58:18.246254 13074 rpcz_store.cc:276] 0925 09:58:13.518617 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:13.518809 (+ 192us) service_pool.cc:227] Handling call
0925 09:58:15.088810 (+1570001us) catalog_manager.cc:2075] Acquired catalog manager lock
0925 09:58:15.090536 (+ 1726us) catalog_manager.cc:2121] Created new table and tablet info
0925 09:58:18.216745 (+3126209us) catalog_manager.cc:2147] Created new table in HMS catalog
0925 09:58:18.245122 (+ 28377us) catalog_manager.cc:2181] Wrote table and tablets to system table
0925 09:58:18.245471 (+ 349us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0925 09:58:18.245795 (+ 324us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:58:18.218625 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:58:18.218974 (+ 349us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:58:18.218997 (+ 23us) write_op.cc:435] Acquired schema lock
0925 09:58:18.219008 (+ 11us) tablet.cc:660] Decoding operations
0925 09:58:18.219268 (+ 260us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:58:18.219306 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0925 09:58:18.219327 (+ 21us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:58:18.219540 (+ 213us) tablet.cc:699] Row locks acquired
0925 09:58:18.219553 (+ 13us) write_op.cc:260] PREPARE: finished
0925 09:58:18.219658 (+ 105us) write_op.cc:270] Start()
0925 09:58:18.219730 (+ 72us) write_op.cc:276] Timestamp: P: 1790330298219590 usec, L: 0
0925 09:58:18.219744 (+ 14us) op_driver.cc:331] REPLICATION: starting
0925 09:58:18.220199 (+ 455us) log.cc:842] Serialized 432 byte log entry
0925 09:58:18.242273 (+ 22074us) op_driver.cc:444] REPLICATION: finished
0925 09:58:18.243170 (+ 897us) write_op.cc:301] APPLY: starting
0925 09:58:18.243233 (+ 63us) tablet.cc:1365] starting BulkCheckPresence
0925 09:58:18.243338 (+ 105us) tablet.cc:1368] finished BulkCheckPresence
0925 09:58:18.243498 (+ 160us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:58:18.244014 (+ 516us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:58:18.244212 (+ 198us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 09:58:18.244278 (+ 66us) write_op.cc:312] APPLY: finished
0925 09:58:18.244531 (+ 253us) log.cc:842] Serialized 28 byte log entry
0925 09:58:18.244722 (+ 191us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:18.244808 (+ 86us) write_op.cc:454] Released schema lock
0925 09:58:18.244870 (+ 62us) write_op.cc:341] FINISH: Updating metrics
0925 09:58:18.245094 (+ 224us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:18.245129 (+ 35us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":2973136,"Hive Metastore.run_cpu_time_us":595,"Hive Metastore.run_wall_time_us":152140,"child_traces":[["op",{"apply.queue_time_us":717,"apply.run_cpu_time_us":2134,"apply.run_wall_time_us":2134,"num_ops":2,"prepare.queue_time_us":141,"prepare.run_cpu_time_us":1836,"prepare.run_wall_time_us":1836,"raft.queue_time_us":1328,"raft.run_cpu_time_us":21,"raft.run_wall_time_us":22,"replication_time_us":22423}]]}
I20260925 09:58:18.255554 13074 catalog_manager.cc:5587] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731) does not have the latest schema for tablet 64cfc93464ae4dc1bf6788105513fe8d (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]). Expected version 1 got 0
I20260925 09:58:18.244911 13773 ts_tablet_manager.cc:1428] T 64cfc93464ae4dc1bf6788105513fe8d P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.066s user 0.009s sys 0.000s
I20260925 09:58:18.245540 13780 tablet_replica.cc:330] T 56559fc1c66e41fd8d6e1e767d4dd7cc P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:18.263778 13780 raft_consensus.cc:2229] T 56559fc1c66e41fd8d6e1e767d4dd7cc P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:18.264273 13780 raft_consensus.cc:2258] T 56559fc1c66e41fd8d6e1e767d4dd7cc P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:18.268466 13780 ts_tablet_manager.cc:1905] T 56559fc1c66e41fd8d6e1e767d4dd7cc P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:18.270746 13483 tablet_service.cc:1467] Processing CreateTablet for tablet 780164f28e2e45e38cfb4b47d7a6584c (DEFAULT_TABLE table=default.table_7be114188e08442db683500d68eafdfd [id=07bcde2a154b4eb2b4ee526d680b5ceb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:18.272164 13483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 780164f28e2e45e38cfb4b47d7a6584c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:18.283536 13867 raft_consensus.cc:482] T bb7bd9e45a6049518cff38bf28e2cbc2 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:18.283871 13077 catalog_manager.cc:5587] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161) does not have the latest schema for tablet 761a36153d004cef86126c719cfe7946 (table default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]). Expected version 1 got 0
I20260925 09:58:18.284011 13867 raft_consensus.cc:504] T bb7bd9e45a6049518cff38bf28e2cbc2 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:18.285812 13867 leader_election.cc:288] T bb7bd9e45a6049518cff38bf28e2cbc2 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
W20260925 09:58:18.290125 13606 leader_election.cc:341] T bb7bd9e45a6049518cff38bf28e2cbc2 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: bb7bd9e45a6049518cff38bf28e2cbc2
I20260925 09:58:18.295544 13362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb7bd9e45a6049518cff38bf28e2cbc2" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
W20260925 09:58:18.297348 13603 leader_election.cc:341] T bb7bd9e45a6049518cff38bf28e2cbc2 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:18.297752 13603 leader_election.cc:302] T bb7bd9e45a6049518cff38bf28e2cbc2 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237; no voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:58:18.298779 13867 raft_consensus.cc:2726] T bb7bd9e45a6049518cff38bf28e2cbc2 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:58:18.303925 13909 mvcc.cc:205] Tried to move back new op lower bound from 7333192901356040192 to 7333192899370782720. Current Snapshot: MvccSnapshot[applied={T|T < 7333192901356040192}]
I20260925 09:58:18.312736 13772 tablet_bootstrap.cc:492] T bb7bd9e45a6049518cff38bf28e2cbc2 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:18.313135 13772 ts_tablet_manager.cc:1397] T bb7bd9e45a6049518cff38bf28e2cbc2 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.242s user 0.022s sys 0.015s
I20260925 09:58:18.315548 13074 master_service.cc:946] ReplaceTablet: received request to replace tablet bb7bd9e45a6049518cff38bf28e2cbc2 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:18.315531 13772 raft_consensus.cc:348] T bb7bd9e45a6049518cff38bf28e2cbc2 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:18.316207 13772 raft_consensus.cc:374] T bb7bd9e45a6049518cff38bf28e2cbc2 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:18.316444 13772 raft_consensus.cc:729] T bb7bd9e45a6049518cff38bf28e2cbc2 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:18.316973 13772 consensus_queue.cc:260] T bb7bd9e45a6049518cff38bf28e2cbc2 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:18.317884 13362 raft_consensus.cc:3037] T 761a36153d004cef86126c719cfe7946 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:18.317922 13650 tablet_service.cc:1467] Processing CreateTablet for tablet 780164f28e2e45e38cfb4b47d7a6584c (DEFAULT_TABLE table=default.table_7be114188e08442db683500d68eafdfd [id=07bcde2a154b4eb2b4ee526d680b5ceb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:18.319696 13077 master_service.cc:946] ReplaceTablet: received request to replace tablet 64cfc93464ae4dc1bf6788105513fe8d from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:18.315718 13269 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 1c8dce72450545e0b304ee41e01bff34
W20260925 09:58:18.321720 13076 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_eadc20bcf90e4a72b8f26e3a4798e259 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:58:18.323069 13912 raft_consensus.cc:686] T af6ace3011c84603b32d0a409d5b8c1c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: Becoming Leader. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Running, Role: LEADER
I20260925 09:58:18.320485 13650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 780164f28e2e45e38cfb4b47d7a6584c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:18.323927 13912 consensus_queue.cc:237] T af6ace3011c84603b32d0a409d5b8c1c P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:18.329962 13362 raft_consensus.cc:1261] T 761a36153d004cef86126c719cfe7946 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Refusing update from remote peer a06c3ef7a6a94f7cbcc03e7cd232f237: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:58:18.332914 13867 consensus_queue.cc:1035] T 761a36153d004cef86126c719cfe7946 P a06c3ef7a6a94f7cbcc03e7cd232f237 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:18.337867 13773 tablet_bootstrap.cc:492] T 780164f28e2e45e38cfb4b47d7a6584c P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:18.340593 13772 ts_tablet_manager.cc:1428] T bb7bd9e45a6049518cff38bf28e2cbc2 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.027s user 0.006s sys 0.001s
I20260925 09:58:18.348483 13772 tablet_bootstrap.cc:492] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:18.355825 13082 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:50556 (request call id 5) took 5126 ms. Trace:
I20260925 09:58:18.356366 13082 rpcz_store.cc:276] 0925 09:58:13.229729 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:13.229940 (+ 211us) service_pool.cc:227] Handling call
0925 09:58:13.623946 (+394006us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:58:18.320788 (+4696842us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:58:18.354785 (+ 33997us) catalog_manager.cc:3616] Apply alter schema
0925 09:58:18.355726 (+ 941us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":77,"Hive Metastore.run_cpu_time_us":2202,"Hive Metastore.run_wall_time_us":1375341}
I20260925 09:58:18.358841 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet bb7bd9e45a6049518cff38bf28e2cbc2 with delete_type TABLET_DATA_DELETED (replaced by tablet 780164f28e2e45e38cfb4b47d7a6584c) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:18.360215 13777 tablet_replica.cc:330] T bb7bd9e45a6049518cff38bf28e2cbc2 P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:18.360721 13773 tablet_bootstrap.cc:654] T 780164f28e2e45e38cfb4b47d7a6584c P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:18.360845 13777 raft_consensus.cc:2229] T bb7bd9e45a6049518cff38bf28e2cbc2 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:18.361303 13777 raft_consensus.cc:2258] T bb7bd9e45a6049518cff38bf28e2cbc2 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:18.366746 13780 ts_tablet_manager.cc:1918] T 56559fc1c66e41fd8d6e1e767d4dd7cc P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:18.368883 13777 ts_tablet_manager.cc:1905] T bb7bd9e45a6049518cff38bf28e2cbc2 P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:18.370676 13780 log.cc:1196] T 56559fc1c66e41fd8d6e1e767d4dd7cc P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/56559fc1c66e41fd8d6e1e767d4dd7cc
I20260925 09:58:18.374455 13780 ts_tablet_manager.cc:1939] T 56559fc1c66e41fd8d6e1e767d4dd7cc P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:18.383107 13077 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet 64cfc93464ae4dc1bf6788105513fe8d
I20260925 09:58:18.384439 13077 catalog_manager.cc:6565] ReplaceTablet: tablet 64cfc93464ae4dc1bf6788105513fe8d deleted and replaced by tablet 2d0e5127e7f548a68dd73858c77f6f84
I20260925 09:58:18.384972 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet 64cfc93464ae4dc1bf6788105513fe8d with delete_type TABLET_DATA_DELETED (replaced by tablet 2d0e5127e7f548a68dd73858c77f6f84) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:18.386057 13649 tablet_service.cc:1514] Processing DeleteTablet for tablet 64cfc93464ae4dc1bf6788105513fe8d with delete_type TABLET_DATA_DELETED (replaced by tablet 2d0e5127e7f548a68dd73858c77f6f84) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:18.388378 13483 tablet_service.cc:1514] Processing DeleteTablet for tablet 64cfc93464ae4dc1bf6788105513fe8d with delete_type TABLET_DATA_DELETED (replaced by tablet 2d0e5127e7f548a68dd73858c77f6f84) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:18.389032 13781 tablet_replica.cc:330] T 64cfc93464ae4dc1bf6788105513fe8d P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:18.389657 13781 raft_consensus.cc:2229] T 64cfc93464ae4dc1bf6788105513fe8d P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:18.390138 13781 raft_consensus.cc:2258] T 64cfc93464ae4dc1bf6788105513fe8d P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:18.392582 13781 ts_tablet_manager.cc:1905] T 64cfc93464ae4dc1bf6788105513fe8d P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:18.412832 13483 tablet_service.cc:1514] Processing DeleteTablet for tablet af6ace3011c84603b32d0a409d5b8c1c with delete_type TABLET_DATA_DELETED (replaced by tablet 1c8dce72450545e0b304ee41e01bff34) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:18.421229 13772 tablet_bootstrap.cc:654] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
W20260925 09:58:15.024207 13826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 09:58:15.037889 13830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 09:58:15.020227 13786 server_base.cc:1034] running on GCE node
I20260925 09:58:18.429875 13786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 09:58:18.388646 13773 tablet_bootstrap.cc:492] T 780164f28e2e45e38cfb4b47d7a6584c P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:18.432101 13773 ts_tablet_manager.cc:1397] T 780164f28e2e45e38cfb4b47d7a6584c P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.094s user 0.010s sys 0.002s
I20260925 09:58:18.434414 13773 raft_consensus.cc:348] T 780164f28e2e45e38cfb4b47d7a6584c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:18.436724 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet 56559fc1c66e41fd8d6e1e767d4dd7cc (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]) successfully deleted
I20260925 09:58:18.436816 13786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 09:58:18.438963 13786 hybrid_clock.cc:648] HybridClock initialized: now 1790330298438875 us; error 88 us; skew 500 ppm
I20260925 09:58:18.440222 13786 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 09:58:18.448062 13780 tablet_replica.cc:330] T bb7bd9e45a6049518cff38bf28e2cbc2 P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:18.448865 13780 raft_consensus.cc:2229] T bb7bd9e45a6049518cff38bf28e2cbc2 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:18.449322 13780 raft_consensus.cc:2258] T bb7bd9e45a6049518cff38bf28e2cbc2 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:18.454396 13777 ts_tablet_manager.cc:1918] T bb7bd9e45a6049518cff38bf28e2cbc2 P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:18.455062 13077 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50556:
name: "default.Table_8f71132b5a004ad89115f7a2278b95d6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:58:18.435127 13773 raft_consensus.cc:374] T 780164f28e2e45e38cfb4b47d7a6584c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:18.459664 13773 raft_consensus.cc:729] T 780164f28e2e45e38cfb4b47d7a6584c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:18.461016 13082 catalog_manager.cc:6040] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477] alter complete (version 1)
I20260925 09:58:18.461931 13770 tablet_bootstrap.cc:492] T 780164f28e2e45e38cfb4b47d7a6584c P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:18.465390 13777 log.cc:1196] T bb7bd9e45a6049518cff38bf28e2cbc2 P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/bb7bd9e45a6049518cff38bf28e2cbc2
I20260925 09:58:18.466192 13777 ts_tablet_manager.cc:1939] T bb7bd9e45a6049518cff38bf28e2cbc2 P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:18.455345 13928 tablet.cc:1722] T 761a36153d004cef86126c719cfe7946 P b3082cf15dc74c0480e5d33ecf592843: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:58:18.459820 13772 tablet_bootstrap.cc:492] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:18.467882 13772 ts_tablet_manager.cc:1397] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.120s user 0.006s sys 0.005s
I20260925 09:58:18.468564 13770 tablet_bootstrap.cc:654] T 780164f28e2e45e38cfb4b47d7a6584c P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:18.470412 13772 raft_consensus.cc:348] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:18.471172 13772 raft_consensus.cc:374] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:18.471414 13772 raft_consensus.cc:729] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:18.472194 13772 consensus_queue.cc:260] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:18.475375 13770 tablet_bootstrap.cc:492] T 780164f28e2e45e38cfb4b47d7a6584c P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:18.475909 13770 ts_tablet_manager.cc:1397] T 780164f28e2e45e38cfb4b47d7a6584c P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.014s user 0.010s sys 0.000s
I20260925 09:58:18.476814 13772 ts_tablet_manager.cc:1428] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20260925 09:58:18.483301 13772 tablet_bootstrap.cc:492] T 64cfc93464ae4dc1bf6788105513fe8d P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:18.484467 13770 raft_consensus.cc:348] T 780164f28e2e45e38cfb4b47d7a6584c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:18.485165 13770 raft_consensus.cc:374] T 780164f28e2e45e38cfb4b47d7a6584c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:18.485419 13770 raft_consensus.cc:729] T 780164f28e2e45e38cfb4b47d7a6584c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:18.486183 13770 consensus_queue.cc:260] T 780164f28e2e45e38cfb4b47d7a6584c P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:18.492027 13773 consensus_queue.cc:260] T 780164f28e2e45e38cfb4b47d7a6584c P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:18.502295 13773 ts_tablet_manager.cc:1428] T 780164f28e2e45e38cfb4b47d7a6584c P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.070s user 0.004s sys 0.002s
I20260925 09:58:18.451197 13780 ts_tablet_manager.cc:1905] T bb7bd9e45a6049518cff38bf28e2cbc2 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:18.467550 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet bb7bd9e45a6049518cff38bf28e2cbc2 (table default.table_7be114188e08442db683500d68eafdfd [id=07bcde2a154b4eb2b4ee526d680b5ceb]) successfully deleted
W20260925 09:58:18.511752 13066 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.10.9.1:42161, user_credentials={real_user=slave}} blocked reactor thread for 44220us
W20260925 09:58:18.512573 13066 catalog_manager.cc:4821] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): delete failed for tablet 64cfc93464ae4dc1bf6788105513fe8d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 64cfc93464ae4dc1bf6788105513fe8d already in progress: creating tablet
W20260925 09:58:18.513686 13073 catalog_manager.cc:5328] Ignoring report from unknown tablet bb7bd9e45a6049518cff38bf28e2cbc2
I20260925 09:58:18.514509 13073 catalog_manager.cc:5587] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161) does not have the latest schema for tablet 761a36153d004cef86126c719cfe7946 (table default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]). Expected version 1 got 0
I20260925 09:58:18.519510 13770 ts_tablet_manager.cc:1428] T 780164f28e2e45e38cfb4b47d7a6584c P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.040s user 0.006s sys 0.000s
W20260925 09:58:18.522871 13074 catalog_manager.cc:5328] Ignoring report from unknown tablet af6ace3011c84603b32d0a409d5b8c1c
I20260925 09:58:18.527976 13772 tablet_bootstrap.cc:654] T 64cfc93464ae4dc1bf6788105513fe8d P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:18.529066 13786 webserver.cc:460] Webserver started at http://127.10.9.60:36655/ using document root <none> and password file <none>
I20260925 09:58:18.530555 13786 fs_manager.cc:359] Metadata directory not provided
I20260925 09:58:18.530910 13786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 09:58:18.542608 13781 ts_tablet_manager.cc:1918] T 64cfc93464ae4dc1bf6788105513fe8d P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:18.543188 13781 log.cc:1196] T 64cfc93464ae4dc1bf6788105513fe8d P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/64cfc93464ae4dc1bf6788105513fe8d
I20260925 09:58:18.543871 13920 raft_consensus.cc:482] T 780164f28e2e45e38cfb4b47d7a6584c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:18.544440 13920 raft_consensus.cc:504] T 780164f28e2e45e38cfb4b47d7a6584c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:18.556519 13650 tablet_service.cc:1467] Processing CreateTablet for tablet b67365d10ba2430594067740623783c9 (DEFAULT_TABLE table=default.table_4c3437905d9e4b05ac1fb4c6e3d0a417 [id=7e0502ffa5544a5bb252a72020649cd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:18.557536 13650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b67365d10ba2430594067740623783c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:18.559813 13362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "780164f28e2e45e38cfb4b47d7a6584c" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
W20260925 09:58:18.561637 13603 leader_election.cc:341] T 780164f28e2e45e38cfb4b47d7a6584c P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:18.561872 13342 tablet_service.cc:1467] Processing CreateTablet for tablet b67365d10ba2430594067740623783c9 (DEFAULT_TABLE table=default.table_4c3437905d9e4b05ac1fb4c6e3d0a417 [id=7e0502ffa5544a5bb252a72020649cd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:18.562446 13080 catalog_manager.cc:2685] deleted HMS catalog entry for table default.table_7be114188e08442db683500d68eafdfd [id=07bcde2a154b4eb2b4ee526d680b5ceb]
I20260925 09:58:18.562820 13342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b67365d10ba2430594067740623783c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:18.564199 13649 tablet_service.cc:1467] Processing CreateTablet for tablet 2d0e5127e7f548a68dd73858c77f6f84 (DEFAULT_TABLE table=default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:18.565220 13649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d0e5127e7f548a68dd73858c77f6f84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:18.565326 13343 tablet_service.cc:1467] Processing CreateTablet for tablet 2d0e5127e7f548a68dd73858c77f6f84 (DEFAULT_TABLE table=default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:18.566350 13343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d0e5127e7f548a68dd73858c77f6f84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:18.568883 13073 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:50556:
table { table_name: "default.Table_4c3437905d9e4b05ac1fb4c6e3d0a417" } new_table_name: "default.Table_e59da12e7b95405787a2f9f73efdc76b" modify_external_catalogs: true
I20260925 09:58:18.570923 13920 leader_election.cc:288] T 780164f28e2e45e38cfb4b47d7a6584c P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
I20260925 09:58:18.576088 13082 master_service.cc:946] ReplaceTablet: received request to replace tablet 761a36153d004cef86126c719cfe7946 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:18.612449 13082 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet 761a36153d004cef86126c719cfe7946
I20260925 09:58:18.613988 13082 catalog_manager.cc:6565] ReplaceTablet: tablet 761a36153d004cef86126c719cfe7946 deleted and replaced by tablet e75bfd761e314d60b6638eb5174b0227
I20260925 09:58:18.615842 13341 tablet_service.cc:1514] Processing DeleteTablet for tablet 761a36153d004cef86126c719cfe7946 with delete_type TABLET_DATA_DELETED (replaced by tablet e75bfd761e314d60b6638eb5174b0227) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:18.616091 13650 tablet_service.cc:1514] Processing DeleteTablet for tablet 761a36153d004cef86126c719cfe7946 with delete_type TABLET_DATA_DELETED (replaced by tablet e75bfd761e314d60b6638eb5174b0227) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:18.618544 13777 tablet_replica.cc:330] T 761a36153d004cef86126c719cfe7946 P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:18.619221 13777 raft_consensus.cc:2229] T 761a36153d004cef86126c719cfe7946 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:18.621592 13777 raft_consensus.cc:2258] T 761a36153d004cef86126c719cfe7946 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:18.624876 13777 ts_tablet_manager.cc:1905] T 761a36153d004cef86126c719cfe7946 P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:18.630303 13074 master_service.cc:946] ReplaceTablet: received request to replace tablet 64cfc93464ae4dc1bf6788105513fe8d from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:18.630949 13075 catalog_manager.cc:2148] created HMS catalog entry for table [id=8fa5a3419e394be2a52d1b13713a04d9]
I20260925 09:58:18.556209 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "780164f28e2e45e38cfb4b47d7a6584c" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
I20260925 09:58:18.631803 13505 raft_consensus.cc:2454] T 780164f28e2e45e38cfb4b47d7a6584c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 0.
I20260925 09:58:18.633055 13606 leader_election.cc:302] T 780164f28e2e45e38cfb4b47d7a6584c P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9; no voters: b3082cf15dc74c0480e5d33ecf592843
I20260925 09:58:18.634192 13920 raft_consensus.cc:2781] T 780164f28e2e45e38cfb4b47d7a6584c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:18.634526 13920 raft_consensus.cc:482] T 780164f28e2e45e38cfb4b47d7a6584c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:18.634820 13920 raft_consensus.cc:3037] T 780164f28e2e45e38cfb4b47d7a6584c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:18.572880 13483 tablet_service.cc:1467] Processing CreateTablet for tablet b67365d10ba2430594067740623783c9 (DEFAULT_TABLE table=default.table_4c3437905d9e4b05ac1fb4c6e3d0a417 [id=7e0502ffa5544a5bb252a72020649cd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:18.580113 13485 tablet_service.cc:1467] Processing CreateTablet for tablet 2d0e5127e7f548a68dd73858c77f6f84 (DEFAULT_TABLE table=default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:18.640277 13483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b67365d10ba2430594067740623783c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:18.618039 13484 tablet_service.cc:1514] Processing DeleteTablet for tablet 761a36153d004cef86126c719cfe7946 with delete_type TABLET_DATA_DELETED (replaced by tablet e75bfd761e314d60b6638eb5174b0227) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:18.641248 13485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d0e5127e7f548a68dd73858c77f6f84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:18.652309 13920 raft_consensus.cc:504] T 780164f28e2e45e38cfb4b47d7a6584c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:18.655521 13362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "780164f28e2e45e38cfb4b47d7a6584c" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843"
W20260925 09:58:18.656962 13603 leader_election.cc:341] T 780164f28e2e45e38cfb4b47d7a6584c P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:18.662014 13770 tablet_bootstrap.cc:492] T b67365d10ba2430594067740623783c9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:18.664265 13781 ts_tablet_manager.cc:1939] T 64cfc93464ae4dc1bf6788105513fe8d P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:18.667896 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet 64cfc93464ae4dc1bf6788105513fe8d (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]) successfully deleted
I20260925 09:58:18.672681 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "780164f28e2e45e38cfb4b47d7a6584c" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9"
I20260925 09:58:18.673246 13505 raft_consensus.cc:3037] T 780164f28e2e45e38cfb4b47d7a6584c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:18.673743 13075 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:50556 (request call id 11) took 5175 ms. Trace:
I20260925 09:58:18.680528 13505 raft_consensus.cc:2454] T 780164f28e2e45e38cfb4b47d7a6584c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 1.
I20260925 09:58:18.681757 13606 leader_election.cc:302] T 780164f28e2e45e38cfb4b47d7a6584c P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9; no voters: b3082cf15dc74c0480e5d33ecf592843
I20260925 09:58:18.682642 13890 raft_consensus.cc:2781] T 780164f28e2e45e38cfb4b47d7a6584c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:18.674161 13075 rpcz_store.cc:276] 0925 09:58:13.498438 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:13.503537 (+ 5099us) service_pool.cc:227] Handling call
0925 09:58:13.513950 (+ 10413us) spinlock_profiling.cc:243] Waited 9.23 ms on lock 0x7b5400021554. stack: 00007fc37b38f033 00007fc37b38ef30 00007fc37af10534 00000000004db2b4 00000000004db249 00007fc38229baf0 00007fc38231ef2a 00007fc3841ead6c 00007fc3842e6156 00007fc381e23034 00007fc381e22fc2 00007fc381e22ef1 00007fc381e22e6c 00007fc381e22122 00007fc3800c594c 00007fc3800c4d86
0925 09:58:15.092596 (+1578646us) catalog_manager.cc:2075] Acquired catalog manager lock
0925 09:58:15.094650 (+ 2054us) catalog_manager.cc:2121] Created new table and tablet info
0925 09:58:18.630896 (+3536246us) catalog_manager.cc:2147] Created new table in HMS catalog
0925 09:58:18.672861 (+ 41965us) catalog_manager.cc:2181] Wrote table and tablets to system table
0925 09:58:18.673289 (+ 428us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0925 09:58:18.673666 (+ 377us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:58:18.645211 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:58:18.645577 (+ 366us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:58:18.645602 (+ 25us) write_op.cc:435] Acquired schema lock
0925 09:58:18.645615 (+ 13us) tablet.cc:660] Decoding operations
0925 09:58:18.645894 (+ 279us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:58:18.645934 (+ 40us) write_op.cc:641] Partition lock acquired for write op
0925 09:58:18.645956 (+ 22us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:58:18.646138 (+ 182us) tablet.cc:699] Row locks acquired
0925 09:58:18.646150 (+ 12us) write_op.cc:260] PREPARE: finished
0925 09:58:18.646269 (+ 119us) write_op.cc:270] Start()
0925 09:58:18.646352 (+ 83us) write_op.cc:276] Timestamp: P: 1790330298646203 usec, L: 0
0925 09:58:18.646365 (+ 13us) op_driver.cc:331] REPLICATION: starting
0925 09:58:18.646882 (+ 517us) log.cc:842] Serialized 432 byte log entry
0925 09:58:18.671127 (+ 24245us) op_driver.cc:444] REPLICATION: finished
0925 09:58:18.671367 (+ 240us) write_op.cc:301] APPLY: starting
0925 09:58:18.671443 (+ 76us) tablet.cc:1365] starting BulkCheckPresence
0925 09:58:18.671543 (+ 100us) tablet.cc:1368] finished BulkCheckPresence
0925 09:58:18.671627 (+ 84us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:58:18.671998 (+ 371us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:58:18.672129 (+ 131us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 09:58:18.672169 (+ 40us) write_op.cc:312] APPLY: finished
0925 09:58:18.672349 (+ 180us) log.cc:842] Serialized 28 byte log entry
0925 09:58:18.672496 (+ 147us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:18.672568 (+ 72us) write_op.cc:454] Released schema lock
0925 09:58:18.672602 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0925 09:58:18.672769 (+ 167us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:18.672791 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":3464852,"Hive Metastore.run_cpu_time_us":607,"Hive Metastore.run_wall_time_us":70585,"spinlock_wait_cycles":21314944,"child_traces":[["op",{"apply.queue_time_us":72,"apply.run_cpu_time_us":1583,"apply.run_wall_time_us":1583,"num_ops":2,"prepare.queue_time_us":12072,"prepare.run_cpu_time_us":1973,"prepare.run_wall_time_us":3132,"raft.queue_time_us":12589,"raft.run_cpu_time_us":684,"raft.run_wall_time_us":685,"replication_time_us":24683}]]}
I20260925 09:58:18.683058 13890 raft_consensus.cc:686] T 780164f28e2e45e38cfb4b47d7a6584c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 LEADER]: Becoming Leader. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Running, Role: LEADER
I20260925 09:58:18.683851 13890 consensus_queue.cc:237] T 780164f28e2e45e38cfb4b47d7a6584c P a06c3ef7a6a94f7cbcc03e7cd232f237 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:18.685832 13777 ts_tablet_manager.cc:1918] T 761a36153d004cef86126c719cfe7946 P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:58:18.686168 13777 log.cc:1196] T 761a36153d004cef86126c719cfe7946 P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/761a36153d004cef86126c719cfe7946
I20260925 09:58:18.686813 13777 ts_tablet_manager.cc:1939] T 761a36153d004cef86126c719cfe7946 P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:18.687963 13773 tablet_bootstrap.cc:492] T 2d0e5127e7f548a68dd73858c77f6f84 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:18.694027 13773 tablet_bootstrap.cc:654] T 2d0e5127e7f548a68dd73858c77f6f84 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:18.708084 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet 761a36153d004cef86126c719cfe7946 (table default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]) successfully deleted
I20260925 09:58:18.722039 13343 tablet_service.cc:1467] Processing CreateTablet for tablet e75bfd761e314d60b6638eb5174b0227 (DEFAULT_TABLE table=default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:18.722694 13650 tablet_service.cc:1467] Processing CreateTablet for tablet e75bfd761e314d60b6638eb5174b0227 (DEFAULT_TABLE table=default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:18.723073 13343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e75bfd761e314d60b6638eb5174b0227. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:18.723637 13650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e75bfd761e314d60b6638eb5174b0227. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:18.719628 13075 catalog_manager.cc:5495] T 780164f28e2e45e38cfb4b47d7a6584c P a06c3ef7a6a94f7cbcc03e7cd232f237 reported cstate change: term changed from 0 to 1, leader changed from <none> to a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3). New cstate: current_term: 1 leader_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: HEALTHY } } }
I20260925 09:58:18.730422 13773 tablet_bootstrap.cc:492] T 2d0e5127e7f548a68dd73858c77f6f84 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:18.730908 13773 ts_tablet_manager.cc:1397] T 2d0e5127e7f548a68dd73858c77f6f84 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.043s user 0.007s sys 0.006s
I20260925 09:58:18.732014 13485 tablet_service.cc:1467] Processing CreateTablet for tablet e75bfd761e314d60b6638eb5174b0227 (DEFAULT_TABLE table=default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:18.736617 13485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e75bfd761e314d60b6638eb5174b0227. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:18.733438 13773 raft_consensus.cc:348] T 2d0e5127e7f548a68dd73858c77f6f84 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:18.742231 13773 raft_consensus.cc:374] T 2d0e5127e7f548a68dd73858c77f6f84 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:18.742516 13773 raft_consensus.cc:729] T 2d0e5127e7f548a68dd73858c77f6f84 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:18.743105 13773 consensus_queue.cc:260] T 2d0e5127e7f548a68dd73858c77f6f84 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:18.746973 13773 ts_tablet_manager.cc:1428] T 2d0e5127e7f548a68dd73858c77f6f84 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.015s user 0.002s sys 0.001s
I20260925 09:58:18.758162 13920 leader_election.cc:288] T 780164f28e2e45e38cfb4b47d7a6584c P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 election: Requested vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
I20260925 09:58:18.763684 13075 catalog_manager.cc:5587] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327) does not have the latest schema for tablet 2d0e5127e7f548a68dd73858c77f6f84 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]). Expected version 1 got 0
W20260925 09:58:18.767221 13075 catalog_manager.cc:5687] Failed to send AlterTable RPC for tablet 2d0e5127e7f548a68dd73858c77f6f84 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 2d0e5127e7f548a68dd73858c77f6f84 consensus state has no leader
I20260925 09:58:18.777561 13773 tablet_bootstrap.cc:492] T e75bfd761e314d60b6638eb5174b0227 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:18.781126 13781 tablet_replica.cc:330] T af6ace3011c84603b32d0a409d5b8c1c P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:18.781769 13781 raft_consensus.cc:2229] T af6ace3011c84603b32d0a409d5b8c1c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:58:18.782393 13781 pending_rounds.cc:62] T af6ace3011c84603b32d0a409d5b8c1c P c4d724d4b83e4d1e817a3a505dc24eb9: Trying to abort 1 pending ops.
I20260925 09:58:18.782600 13781 pending_rounds.cc:68] T af6ace3011c84603b32d0a409d5b8c1c P c4d724d4b83e4d1e817a3a505dc24eb9: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192901943500800 op_type: NO_OP noop_request { }
I20260925 09:58:18.782917 13781 raft_consensus.cc:2866] T af6ace3011c84603b32d0a409d5b8c1c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:58:18.783196 13781 raft_consensus.cc:2258] T af6ace3011c84603b32d0a409d5b8c1c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:18.786264 13773 tablet_bootstrap.cc:654] T e75bfd761e314d60b6638eb5174b0227 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:18.791531 13770 tablet_bootstrap.cc:654] T b67365d10ba2430594067740623783c9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:18.796527 13078 catalog_manager.cc:2148] created HMS catalog entry for table [id=f6ae78b7a67640f5a6e71e04249b4262]
I20260925 09:58:18.796900 13773 tablet_bootstrap.cc:492] T e75bfd761e314d60b6638eb5174b0227 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:18.799300 13773 ts_tablet_manager.cc:1397] T e75bfd761e314d60b6638eb5174b0227 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.022s user 0.010s sys 0.004s
I20260925 09:58:18.803802 13773 raft_consensus.cc:348] T e75bfd761e314d60b6638eb5174b0227 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:18.805415 13773 raft_consensus.cc:374] T e75bfd761e314d60b6638eb5174b0227 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:18.807008 13773 raft_consensus.cc:729] T e75bfd761e314d60b6638eb5174b0227 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:18.809346 13773 consensus_queue.cc:260] T e75bfd761e314d60b6638eb5174b0227 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:18.817834 13078 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:50556 (request call id 8) took 5387 ms. Trace:
I20260925 09:58:18.822908 13074 master_service.cc:946] ReplaceTablet: received request to replace tablet 64cfc93464ae4dc1bf6788105513fe8d from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:18.836359 13269 catalog_manager.cc:2709] Deleting table default.table_7be114188e08442db683500d68eafdfd [id=07bcde2a154b4eb2b4ee526d680b5ceb] in response to Hive Metastore notification log event 9
I20260925 09:58:18.818300 13078 rpcz_store.cc:276] 0925 09:58:13.430514 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:13.430672 (+ 158us) service_pool.cc:227] Handling call
0925 09:58:15.087748 (+1657076us) catalog_manager.cc:2075] Acquired catalog manager lock
0925 09:58:15.095898 (+ 8150us) catalog_manager.cc:2121] Created new table and tablet info
0925 09:58:18.796410 (+3700512us) catalog_manager.cc:2147] Created new table in HMS catalog
0925 09:58:18.816880 (+ 20470us) catalog_manager.cc:2181] Wrote table and tablets to system table
0925 09:58:18.817360 (+ 480us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0925 09:58:18.817752 (+ 392us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:58:18.799317 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:58:18.799699 (+ 382us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:58:18.799722 (+ 23us) write_op.cc:435] Acquired schema lock
0925 09:58:18.799734 (+ 12us) tablet.cc:660] Decoding operations
0925 09:58:18.799991 (+ 257us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:58:18.800027 (+ 36us) write_op.cc:641] Partition lock acquired for write op
0925 09:58:18.800046 (+ 19us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:58:18.800236 (+ 190us) tablet.cc:699] Row locks acquired
0925 09:58:18.800248 (+ 12us) write_op.cc:260] PREPARE: finished
0925 09:58:18.800365 (+ 117us) write_op.cc:270] Start()
0925 09:58:18.800450 (+ 85us) write_op.cc:276] Timestamp: P: 1790330298800288 usec, L: 0
0925 09:58:18.800465 (+ 15us) op_driver.cc:331] REPLICATION: starting
0925 09:58:18.800931 (+ 466us) log.cc:842] Serialized 432 byte log entry
0925 09:58:18.810135 (+ 9204us) op_driver.cc:444] REPLICATION: finished
0925 09:58:18.810562 (+ 427us) write_op.cc:301] APPLY: starting
0925 09:58:18.810621 (+ 59us) tablet.cc:1365] starting BulkCheckPresence
0925 09:58:18.810724 (+ 103us) tablet.cc:1368] finished BulkCheckPresence
0925 09:58:18.810833 (+ 109us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:58:18.811264 (+ 431us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:58:18.812621 (+ 1357us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 09:58:18.812699 (+ 78us) write_op.cc:312] APPLY: finished
0925 09:58:18.812973 (+ 274us) log.cc:842] Serialized 28 byte log entry
0925 09:58:18.813278 (+ 305us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:18.813345 (+ 67us) write_op.cc:454] Released schema lock
0925 09:58:18.813382 (+ 37us) write_op.cc:341] FINISH: Updating metrics
0925 09:58:18.813569 (+ 187us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:18.813591 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":3534345,"Hive Metastore.run_cpu_time_us":691,"Hive Metastore.run_wall_time_us":164010,"child_traces":[["op",{"apply.queue_time_us":246,"apply.run_cpu_time_us":2018,"apply.run_wall_time_us":3188,"num_ops":2,"prepare.queue_time_us":644,"prepare.run_cpu_time_us":1953,"prepare.run_wall_time_us":19602,"raft.queue_time_us":82,"raft.run_cpu_time_us":645,"raft.run_wall_time_us":645,"replication_time_us":9590}]]}
W20260925 09:58:18.840878 13067 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 2d0e5127e7f548a68dd73858c77f6f84 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 2d0e5127e7f548a68dd73858c77f6f84 consensus state has no leader
I20260925 09:58:18.842777 13781 ts_tablet_manager.cc:1905] T af6ace3011c84603b32d0a409d5b8c1c P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:18.844213 13773 ts_tablet_manager.cc:1428] T e75bfd761e314d60b6638eb5174b0227 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.044s user 0.007s sys 0.000s
I20260925 09:58:18.851085 13772 tablet_bootstrap.cc:492] T 64cfc93464ae4dc1bf6788105513fe8d P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:18.851537 13772 ts_tablet_manager.cc:1397] T 64cfc93464ae4dc1bf6788105513fe8d P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.368s user 0.041s sys 0.094s
I20260925 09:58:18.857507 13269 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet 780164f28e2e45e38cfb4b47d7a6584c
I20260925 09:58:18.859678 13483 tablet_service.cc:1514] Processing DeleteTablet for tablet 780164f28e2e45e38cfb4b47d7a6584c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:58:18 UTC) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:18.863663 13650 tablet_service.cc:1514] Processing DeleteTablet for tablet 780164f28e2e45e38cfb4b47d7a6584c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:58:18 UTC) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:18.864351 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet 780164f28e2e45e38cfb4b47d7a6584c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:58:18 UTC) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:18.866353 13772 raft_consensus.cc:348] T 64cfc93464ae4dc1bf6788105513fe8d P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:18.867102 13772 raft_consensus.cc:374] T 64cfc93464ae4dc1bf6788105513fe8d P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:18.867375 13772 raft_consensus.cc:729] T 64cfc93464ae4dc1bf6788105513fe8d P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
W20260925 09:58:18.868994 13066 catalog_manager.cc:4821] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): delete failed for tablet 780164f28e2e45e38cfb4b47d7a6584c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 780164f28e2e45e38cfb4b47d7a6584c already in progress: creating tablet
I20260925 09:58:18.874109 13079 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:50556 (request call id 6) took 5577 ms. Trace:
I20260925 09:58:18.874454 13079 rpcz_store.cc:276] 0925 09:58:13.296996 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:13.298454 (+ 1458us) service_pool.cc:227] Handling call
0925 09:58:15.060345 (+1761891us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:58:18.872210 (+3811865us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:58:18.873170 (+ 960us) catalog_manager.cc:3616] Apply alter schema
0925 09:58:18.874047 (+ 877us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":80,"Hive Metastore.run_cpu_time_us":1795,"Hive Metastore.run_wall_time_us":368406}
I20260925 09:58:18.875627 13081 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:50556 (request call id 7) took 5558 ms. Trace:
I20260925 09:58:18.875945 13081 rpcz_store.cc:276] 0925 09:58:13.317316 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:13.317476 (+ 160us) service_pool.cc:227] Handling call
0925 09:58:15.084558 (+1767082us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:58:18.875545 (+3790987us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":344572,"Hive Metastore.run_cpu_time_us":902,"Hive Metastore.run_wall_time_us":2626088}
I20260925 09:58:18.876770 13773 tablet_bootstrap.cc:492] T b67365d10ba2430594067740623783c9 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:18.882998 13780 ts_tablet_manager.cc:1918] T bb7bd9e45a6049518cff38bf28e2cbc2 P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:18.883569 13780 log.cc:1196] T bb7bd9e45a6049518cff38bf28e2cbc2 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/bb7bd9e45a6049518cff38bf28e2cbc2
I20260925 09:58:18.886394 13773 tablet_bootstrap.cc:654] T b67365d10ba2430594067740623783c9 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:18.886796 13081 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50556:
name: "default.Table_1238f9df428247dc9b9ca7c59823bfe4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:58:18.884305 13780 ts_tablet_manager.cc:1939] T bb7bd9e45a6049518cff38bf28e2cbc2 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:18.891755 13912 raft_consensus.cc:482] T 2d0e5127e7f548a68dd73858c77f6f84 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:18.892230 13912 raft_consensus.cc:504] T 2d0e5127e7f548a68dd73858c77f6f84 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:18.894011 13780 tablet_replica.cc:330] T af6ace3011c84603b32d0a409d5b8c1c P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:18.894057 13912 leader_election.cc:288] T 2d0e5127e7f548a68dd73858c77f6f84 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:18.894657 13780 raft_consensus.cc:2229] T af6ace3011c84603b32d0a409d5b8c1c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:18.895044 13780 raft_consensus.cc:2258] T af6ace3011c84603b32d0a409d5b8c1c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:18.895989 13772 consensus_queue.cc:260] T 64cfc93464ae4dc1bf6788105513fe8d P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:18.898025 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d0e5127e7f548a68dd73858c77f6f84" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:18.900791 13362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d0e5127e7f548a68dd73858c77f6f84" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
W20260925 09:58:18.901582 13440 leader_election.cc:341] T 2d0e5127e7f548a68dd73858c77f6f84 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:18.905280 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet 64cfc93464ae4dc1bf6788105513fe8d with delete_type TABLET_DATA_DELETED (replaced by tablet 2d0e5127e7f548a68dd73858c77f6f84) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:18.905788 13772 ts_tablet_manager.cc:1428] T 64cfc93464ae4dc1bf6788105513fe8d P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.054s user 0.005s sys 0.001s
W20260925 09:58:18.906292 13438 leader_election.cc:341] T 2d0e5127e7f548a68dd73858c77f6f84 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:18.906687 13438 leader_election.cc:302] T 2d0e5127e7f548a68dd73858c77f6f84 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4d724d4b83e4d1e817a3a505dc24eb9; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, b3082cf15dc74c0480e5d33ecf592843
W20260925 09:58:18.907330 13066 catalog_manager.cc:4821] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): delete failed for tablet 64cfc93464ae4dc1bf6788105513fe8d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 64cfc93464ae4dc1bf6788105513fe8d already in progress: creating tablet
I20260925 09:58:18.912178 13773 tablet_bootstrap.cc:492] T b67365d10ba2430594067740623783c9 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:18.912611 13773 ts_tablet_manager.cc:1397] T b67365d10ba2430594067740623783c9 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.036s user 0.006s sys 0.003s
I20260925 09:58:18.914227 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet 64cfc93464ae4dc1bf6788105513fe8d with delete_type TABLET_DATA_DELETED (replaced by tablet 2d0e5127e7f548a68dd73858c77f6f84) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:18.914529 13772 tablet_bootstrap.cc:492] T 780164f28e2e45e38cfb4b47d7a6584c P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:18.915124 13777 tablet_replica.cc:330] T 64cfc93464ae4dc1bf6788105513fe8d P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:18.915753 13777 raft_consensus.cc:2229] T 64cfc93464ae4dc1bf6788105513fe8d P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:18.916148 13777 raft_consensus.cc:2258] T 64cfc93464ae4dc1bf6788105513fe8d P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:18.916255 13912 raft_consensus.cc:2726] T 2d0e5127e7f548a68dd73858c77f6f84 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:58:18.918200 13777 ts_tablet_manager.cc:1905] T 64cfc93464ae4dc1bf6788105513fe8d P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:18.919112 13780 ts_tablet_manager.cc:1905] T af6ace3011c84603b32d0a409d5b8c1c P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:18.919368 13786 fs_manager.cc:709] Time spent opening directory manager: real 0.154s user 0.004s sys 0.004s
I20260925 09:58:18.927804 13079 master_service.cc:946] ReplaceTablet: received request to replace tablet af6ace3011c84603b32d0a409d5b8c1c from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:18.919171 13773 raft_consensus.cc:348] T b67365d10ba2430594067740623783c9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:18.928894 13773 raft_consensus.cc:374] T b67365d10ba2430594067740623783c9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:18.929162 13773 raft_consensus.cc:729] T b67365d10ba2430594067740623783c9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:18.930212 13773 consensus_queue.cc:260] T b67365d10ba2430594067740623783c9 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:18.934391 13773 ts_tablet_manager.cc:1428] T b67365d10ba2430594067740623783c9 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.022s user 0.005s sys 0.001s
I20260925 09:58:18.944028 13075 catalog_manager.cc:5587] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327) does not have the latest schema for tablet e75bfd761e314d60b6638eb5174b0227 (table default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]). Expected version 1 got 0
W20260925 09:58:18.945847 13075 catalog_manager.cc:5687] Failed to send AlterTable RPC for tablet e75bfd761e314d60b6638eb5174b0227 (table default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet e75bfd761e314d60b6638eb5174b0227 consensus state has no leader
I20260925 09:58:18.948258 13650 tablet_service.cc:1467] Processing CreateTablet for tablet dea7e34e5932444daa0ba1a11af3d2ea (DEFAULT_TABLE table=default.table_a29aa5b33fd644a2ab80242e289ed25d [id=8fa5a3419e394be2a52d1b13713a04d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:18.948771 13342 tablet_service.cc:1467] Processing CreateTablet for tablet dea7e34e5932444daa0ba1a11af3d2ea (DEFAULT_TABLE table=default.table_a29aa5b33fd644a2ab80242e289ed25d [id=8fa5a3419e394be2a52d1b13713a04d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:18.949321 13650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dea7e34e5932444daa0ba1a11af3d2ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:18.949756 13342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dea7e34e5932444daa0ba1a11af3d2ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:18.958024 13772 tablet_bootstrap.cc:654] T 780164f28e2e45e38cfb4b47d7a6584c P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:18.974993 13770 tablet_bootstrap.cc:492] T b67365d10ba2430594067740623783c9 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:18.975529 13770 ts_tablet_manager.cc:1397] T b67365d10ba2430594067740623783c9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.314s user 0.009s sys 0.002s
I20260925 09:58:18.977114 13777 ts_tablet_manager.cc:1918] T 64cfc93464ae4dc1bf6788105513fe8d P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:18.977474 13777 log.cc:1196] T 64cfc93464ae4dc1bf6788105513fe8d P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/64cfc93464ae4dc1bf6788105513fe8d
I20260925 09:58:18.978093 13777 ts_tablet_manager.cc:1939] T 64cfc93464ae4dc1bf6788105513fe8d P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:18.980427 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet 64cfc93464ae4dc1bf6788105513fe8d (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]) successfully deleted
I20260925 09:58:18.990137 13772 tablet_bootstrap.cc:492] T 780164f28e2e45e38cfb4b47d7a6584c P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:18.990568 13772 ts_tablet_manager.cc:1397] T 780164f28e2e45e38cfb4b47d7a6584c P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.083s user 0.008s sys 0.001s
I20260925 09:58:18.992865 13772 raft_consensus.cc:348] T 780164f28e2e45e38cfb4b47d7a6584c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:18.993517 13772 raft_consensus.cc:374] T 780164f28e2e45e38cfb4b47d7a6584c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:18.993757 13772 raft_consensus.cc:729] T 780164f28e2e45e38cfb4b47d7a6584c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:18.998569 13772 consensus_queue.cc:260] T 780164f28e2e45e38cfb4b47d7a6584c P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:19.000933 13483 tablet_service.cc:1467] Processing CreateTablet for tablet dea7e34e5932444daa0ba1a11af3d2ea (DEFAULT_TABLE table=default.table_a29aa5b33fd644a2ab80242e289ed25d [id=8fa5a3419e394be2a52d1b13713a04d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:19.002638 13483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dea7e34e5932444daa0ba1a11af3d2ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:19.004253 13770 raft_consensus.cc:348] T b67365d10ba2430594067740623783c9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:19.004951 13770 raft_consensus.cc:374] T b67365d10ba2430594067740623783c9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:19.005208 13770 raft_consensus.cc:729] T b67365d10ba2430594067740623783c9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:19.012539 13343 tablet_service.cc:1514] Processing DeleteTablet for tablet 780164f28e2e45e38cfb4b47d7a6584c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:58:18 UTC) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:19.012570 13649 tablet_service.cc:1467] Processing CreateTablet for tablet d48c0366f9b8428983f0844437d890a8 (DEFAULT_TABLE table=default.table_3823faf045944cc1a828ce929b8a7275 [id=f6ae78b7a67640f5a6e71e04249b4262]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:19.013600 13649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d48c0366f9b8428983f0844437d890a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:19.011827 13772 ts_tablet_manager.cc:1428] T 780164f28e2e45e38cfb4b47d7a6584c P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.021s user 0.004s sys 0.004s
W20260925 09:58:19.013818 13066 catalog_manager.cc:4821] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): delete failed for tablet 780164f28e2e45e38cfb4b47d7a6584c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 780164f28e2e45e38cfb4b47d7a6584c already in progress: creating tablet
I20260925 09:58:19.014529 13342 tablet_service.cc:1467] Processing CreateTablet for tablet d48c0366f9b8428983f0844437d890a8 (DEFAULT_TABLE table=default.table_3823faf045944cc1a828ce929b8a7275 [id=f6ae78b7a67640f5a6e71e04249b4262]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:19.022218 13343 tablet_service.cc:1514] Processing DeleteTablet for tablet 780164f28e2e45e38cfb4b47d7a6584c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:58:18 UTC) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:19.022655 13342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d48c0366f9b8428983f0844437d890a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:19.024644 13777 tablet_replica.cc:330] T 780164f28e2e45e38cfb4b47d7a6584c P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:19.025609 13772 tablet_bootstrap.cc:492] T 2d0e5127e7f548a68dd73858c77f6f84 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:19.026764 13485 tablet_service.cc:1467] Processing CreateTablet for tablet d48c0366f9b8428983f0844437d890a8 (DEFAULT_TABLE table=default.table_3823faf045944cc1a828ce929b8a7275 [id=f6ae78b7a67640f5a6e71e04249b4262]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:19.027663 13780 ts_tablet_manager.cc:1918] T af6ace3011c84603b32d0a409d5b8c1c P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:19.027925 13485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d48c0366f9b8428983f0844437d890a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:19.028035 13780 log.cc:1196] T af6ace3011c84603b32d0a409d5b8c1c P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/af6ace3011c84603b32d0a409d5b8c1c
I20260925 09:58:19.027855 13770 consensus_queue.cc:260] T b67365d10ba2430594067740623783c9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:19.028605 13780 ts_tablet_manager.cc:1939] T af6ace3011c84603b32d0a409d5b8c1c P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:19.031760 13943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 09:58:19.043136 13770 ts_tablet_manager.cc:1428] T b67365d10ba2430594067740623783c9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.067s user 0.001s sys 0.004s
I20260925 09:58:19.045186 13786 fs_manager.cc:725] Time spent opening block manager: real 0.079s user 0.004s sys 0.000s
I20260925 09:58:19.050230 13075 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50556:
name: "default.Table_4d3f82d91a5749679343f60e3dbbc422"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:58:19.059675 13079 catalog_manager.cc:5587] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327) does not have the latest schema for tablet e75bfd761e314d60b6638eb5174b0227 (table default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]). Expected version 1 got 0
I20260925 09:58:19.066146 13780 tablet_replica.cc:330] T 64cfc93464ae4dc1bf6788105513fe8d P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:19.066839 13780 raft_consensus.cc:2229] T 64cfc93464ae4dc1bf6788105513fe8d P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:19.067269 13780 raft_consensus.cc:2258] T 64cfc93464ae4dc1bf6788105513fe8d P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:19.071123 13777 raft_consensus.cc:2229] T 780164f28e2e45e38cfb4b47d7a6584c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:19.071641 13777 raft_consensus.cc:2258] T 780164f28e2e45e38cfb4b47d7a6584c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:19.076359 13079 master_service.cc:946] ReplaceTablet: received request to replace tablet b67365d10ba2430594067740623783c9 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:19.081230 13082 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50556:
table { table_name: "default.Table_3823faf045944cc1a828ce929b8a7275" } modify_external_catalogs: true
I20260925 09:58:19.081318 13074 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:50556:
table { table_name: "default.Table_a29aa5b33fd644a2ab80242e289ed25d" } new_table_name: "default.Table_81574dad60e9490cb79e268b12969a83" modify_external_catalogs: true
I20260925 09:58:19.082010 13082 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50556:
table { table_name: "default.Table_3823faf045944cc1a828ce929b8a7275" } modify_external_catalogs: true
I20260925 09:58:19.045723 13786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-2/data,/tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-2/wal
uuid: "61e29afab0cf46819cbffc91248bf761"
format_stamp: "Formatted at 2026-09-25 09:58:03 on dist-test-slave-4xvl"
I20260925 09:58:19.080742 13781 ts_tablet_manager.cc:1918] T af6ace3011c84603b32d0a409d5b8c1c P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:19.085995 13786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 09:58:19.087038 13781 log.cc:1196] T af6ace3011c84603b32d0a409d5b8c1c P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/af6ace3011c84603b32d0a409d5b8c1c
I20260925 09:58:19.077811 13770 tablet_bootstrap.cc:492] T 2d0e5127e7f548a68dd73858c77f6f84 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:19.088094 13781 ts_tablet_manager.cc:1939] T af6ace3011c84603b32d0a409d5b8c1c P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:19.089582 13780 ts_tablet_manager.cc:1905] T 64cfc93464ae4dc1bf6788105513fe8d P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:19.093149 13777 ts_tablet_manager.cc:1905] T 780164f28e2e45e38cfb4b47d7a6584c P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:19.094357 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet af6ace3011c84603b32d0a409d5b8c1c (table default.table_4ca061cf765540c096d1d9aeaa2dc63d [id=8c90bf002b144971986f1085cd102118]) successfully deleted
I20260925 09:58:19.095005 13781 tablet_replica.cc:330] T 761a36153d004cef86126c719cfe7946 P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:19.095795 13781 raft_consensus.cc:2229] T 761a36153d004cef86126c719cfe7946 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:19.099814 13781 raft_consensus.cc:2258] T 761a36153d004cef86126c719cfe7946 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:19.102377 13772 tablet_bootstrap.cc:654] T 2d0e5127e7f548a68dd73858c77f6f84 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:19.108119 13781 ts_tablet_manager.cc:1905] T 761a36153d004cef86126c719cfe7946 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:19.113277 13076 catalog_manager.cc:2148] created HMS catalog entry for table [id=f196d9d3750b4857905c1332d3ac763e]
I20260925 09:58:19.114389 13773 tablet_bootstrap.cc:492] T d48c0366f9b8428983f0844437d890a8 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:19.120317 13773 tablet_bootstrap.cc:654] T d48c0366f9b8428983f0844437d890a8 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:19.138200 13773 tablet_bootstrap.cc:492] T d48c0366f9b8428983f0844437d890a8 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:19.138761 13773 ts_tablet_manager.cc:1397] T d48c0366f9b8428983f0844437d890a8 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.025s user 0.010s sys 0.001s
I20260925 09:58:19.139043 13079 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet b67365d10ba2430594067740623783c9
I20260925 09:58:19.141315 13343 tablet_service.cc:1514] Processing DeleteTablet for tablet b67365d10ba2430594067740623783c9 with delete_type TABLET_DATA_DELETED (replaced by tablet b103b022a84040ea97247d6503002f47) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:19.141601 13651 tablet_service.cc:1514] Processing DeleteTablet for tablet b67365d10ba2430594067740623783c9 with delete_type TABLET_DATA_DELETED (replaced by tablet b103b022a84040ea97247d6503002f47) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:19.141886 13772 tablet_bootstrap.cc:492] T 2d0e5127e7f548a68dd73858c77f6f84 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:19.142349 13772 ts_tablet_manager.cc:1397] T 2d0e5127e7f548a68dd73858c77f6f84 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.117s user 0.004s sys 0.006s
I20260925 09:58:19.142432 13078 master_service.cc:946] ReplaceTablet: received request to replace tablet 780164f28e2e45e38cfb4b47d7a6584c from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:19.144543 13772 raft_consensus.cc:348] T 2d0e5127e7f548a68dd73858c77f6f84 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:19.145202 13772 raft_consensus.cc:374] T 2d0e5127e7f548a68dd73858c77f6f84 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:19.142692 13773 raft_consensus.cc:348] T d48c0366f9b8428983f0844437d890a8 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:19.145449 13772 raft_consensus.cc:729] T 2d0e5127e7f548a68dd73858c77f6f84 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:19.145758 13773 raft_consensus.cc:374] T d48c0366f9b8428983f0844437d890a8 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:19.146034 13773 raft_consensus.cc:729] T d48c0366f9b8428983f0844437d890a8 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:19.145416 13777 ts_tablet_manager.cc:1918] T 780164f28e2e45e38cfb4b47d7a6584c P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:19.147818 13777 log.cc:1196] T 780164f28e2e45e38cfb4b47d7a6584c P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/780164f28e2e45e38cfb4b47d7a6584c
I20260925 09:58:19.148502 13777 ts_tablet_manager.cc:1939] T 780164f28e2e45e38cfb4b47d7a6584c P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:19.150180 13483 tablet_service.cc:1514] Processing DeleteTablet for tablet b67365d10ba2430594067740623783c9 with delete_type TABLET_DATA_DELETED (replaced by tablet b103b022a84040ea97247d6503002f47) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:19.150434 13079 catalog_manager.cc:6565] ReplaceTablet: tablet b67365d10ba2430594067740623783c9 deleted and replaced by tablet b103b022a84040ea97247d6503002f47
I20260925 09:58:19.151564 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet 780164f28e2e45e38cfb4b47d7a6584c (table default.table_7be114188e08442db683500d68eafdfd [id=07bcde2a154b4eb2b4ee526d680b5ceb]) successfully deleted
I20260925 09:58:19.153913 13773 consensus_queue.cc:260] T d48c0366f9b8428983f0844437d890a8 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:19.155135 13650 tablet_service.cc:1514] Processing DeleteTablet for tablet b67365d10ba2430594067740623783c9 with delete_type TABLET_DATA_DELETED (replaced by tablet b103b022a84040ea97247d6503002f47) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:19.155755 13772 consensus_queue.cc:260] T 2d0e5127e7f548a68dd73858c77f6f84 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:19.162331 13076 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:50556 (request call id 14) took 5421 ms. Trace:
W20260925 09:58:19.168567 13606 consensus_peers.cc:487] T 761a36153d004cef86126c719cfe7946 P a06c3ef7a6a94f7cbcc03e7cd232f237 -> Peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Couldn't send request to peer c4d724d4b83e4d1e817a3a505dc24eb9. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:58:19.162683 13076 rpcz_store.cc:276] 0925 09:58:13.741464 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:13.749635 (+ 8171us) service_pool.cc:227] Handling call
0925 09:58:13.781124 (+ 31489us) spinlock_profiling.cc:243] Waited 27 ms on lock 0x7b5400021554. stack: 00007fc37b38f033 00007fc37b38ef30 00007fc37af10534 00000000004db2b4 00000000004db249 00007fc38229baf0 00007fc38231ef2a 00007fc3841ead6c 00007fc3842e6156 00007fc381e23034 00007fc381e22fc2 00007fc381e22ef1 00007fc381e22e6c 00007fc381e22122 00007fc3800c594c 00007fc3800c4d86
0925 09:58:18.322157 (+4541033us) catalog_manager.cc:2075] Acquired catalog manager lock
0925 09:58:18.403987 (+ 81830us) catalog_manager.cc:2121] Created new table and tablet info
0925 09:58:19.113232 (+709245us) catalog_manager.cc:2147] Created new table in HMS catalog
0925 09:58:19.161586 (+ 48354us) catalog_manager.cc:2181] Wrote table and tablets to system table
0925 09:58:19.161946 (+ 360us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0925 09:58:19.162270 (+ 324us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:58:19.115111 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:58:19.115498 (+ 387us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:58:19.115519 (+ 21us) write_op.cc:435] Acquired schema lock
0925 09:58:19.115530 (+ 11us) tablet.cc:660] Decoding operations
0925 09:58:19.115774 (+ 244us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:58:19.115809 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0925 09:58:19.115831 (+ 22us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:58:19.116007 (+ 176us) tablet.cc:699] Row locks acquired
0925 09:58:19.116019 (+ 12us) write_op.cc:260] PREPARE: finished
0925 09:58:19.116123 (+ 104us) write_op.cc:270] Start()
0925 09:58:19.116203 (+ 80us) write_op.cc:276] Timestamp: P: 1790330299116067 usec, L: 0
0925 09:58:19.116218 (+ 15us) op_driver.cc:331] REPLICATION: starting
0925 09:58:19.116684 (+ 466us) log.cc:842] Serialized 432 byte log entry
0925 09:58:19.158805 (+ 42121us) op_driver.cc:444] REPLICATION: finished
0925 09:58:19.159052 (+ 247us) write_op.cc:301] APPLY: starting
0925 09:58:19.159114 (+ 62us) tablet.cc:1365] starting BulkCheckPresence
0925 09:58:19.159222 (+ 108us) tablet.cc:1368] finished BulkCheckPresence
0925 09:58:19.159308 (+ 86us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:58:19.159731 (+ 423us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:58:19.159872 (+ 141us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 09:58:19.159917 (+ 45us) write_op.cc:312] APPLY: finished
0925 09:58:19.160123 (+ 206us) log.cc:842] Serialized 28 byte log entry
0925 09:58:19.160269 (+ 146us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:19.160336 (+ 67us) write_op.cc:454] Released schema lock
0925 09:58:19.160369 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0925 09:58:19.160537 (+ 168us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:19.160560 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":427235,"Hive Metastore.run_cpu_time_us":611,"Hive Metastore.run_wall_time_us":281131,"spinlock_wait_cycles":62053248,"child_traces":[["op",{"apply.queue_time_us":78,"apply.run_cpu_time_us":1655,"apply.run_wall_time_us":1654,"mutex_wait_us":4,"num_ops":2,"prepare.queue_time_us":183,"prepare.run_cpu_time_us":7495,"prepare.run_wall_time_us":47220,"raft.queue_time_us":2177,"raft.run_cpu_time_us":24,"raft.run_wall_time_us":25,"replication_time_us":42502,"spinlock_wait_cycles":335104,"thread_start_us":40517,"threads_started":1}]]}
I20260925 09:58:19.172073 13773 ts_tablet_manager.cc:1428] T d48c0366f9b8428983f0844437d890a8 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.033s user 0.006s sys 0.000s
I20260925 09:58:19.173786 13772 ts_tablet_manager.cc:1428] T 2d0e5127e7f548a68dd73858c77f6f84 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.031s user 0.005s sys 0.000s
W20260925 09:58:19.174834 13066 catalog_manager.cc:4821] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): delete failed for tablet b67365d10ba2430594067740623783c9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b67365d10ba2430594067740623783c9 already in progress: creating tablet
I20260925 09:58:19.180056 13076 catalog_manager.cc:5587] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161) does not have the latest schema for tablet 2d0e5127e7f548a68dd73858c77f6f84 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]). Expected version 1 got 0
I20260925 09:58:19.183856 13780 ts_tablet_manager.cc:1918] T 64cfc93464ae4dc1bf6788105513fe8d P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:19.187186 13773 tablet_bootstrap.cc:492] T dea7e34e5932444daa0ba1a11af3d2ea P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
W20260925 09:58:19.185559 13077 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_8f71132b5a004ad89115f7a2278b95d6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:58:19.186015 13080 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:50556 (request call id 13) took 5647 ms. Trace:
I20260925 09:58:19.193076 13781 ts_tablet_manager.cc:1918] T 761a36153d004cef86126c719cfe7946 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:58:19.193418 13781 log.cc:1196] T 761a36153d004cef86126c719cfe7946 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/761a36153d004cef86126c719cfe7946
I20260925 09:58:19.193266 13080 rpcz_store.cc:276] 0925 09:58:13.538215 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:13.540275 (+ 2060us) service_pool.cc:227] Handling call
0925 09:58:13.548651 (+ 8376us) spinlock_profiling.cc:243] Waited 6.87 ms on lock 0x7b5400021554. stack: 00007fc37b38f033 00007fc37b38ef30 00007fc37af10534 00000000004db2b4 00000000004db249 00007fc38229baf0 00007fc38231ef2a 00007fc3841ead6c 00007fc3842e6466 00007fc381e27b34 00007fc381e27ac2 00007fc381e279f1 00007fc381e2796c 00007fc381e26c22 00007fc3800c594c 00007fc3800c4d86
0925 09:58:15.091090 (+1542439us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:58:19.185934 (+4094844us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":3124617,"Hive Metastore.run_cpu_time_us":350,"Hive Metastore.run_wall_time_us":343533,"spinlock_wait_cycles":15805824}
I20260925 09:58:19.194090 13781 ts_tablet_manager.cc:1939] T 761a36153d004cef86126c719cfe7946 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:19.197985 13772 tablet_bootstrap.cc:492] T e75bfd761e314d60b6638eb5174b0227 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:19.199532 13782 raft_consensus.cc:482] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:19.199958 13782 raft_consensus.cc:504] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:19.201936 13782 leader_election.cc:288] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:19.203145 13781 tablet_replica.cc:330] T 780164f28e2e45e38cfb4b47d7a6584c P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:19.204480 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet 761a36153d004cef86126c719cfe7946 (table default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]) successfully deleted
W20260925 09:58:19.208207 13298 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
I20260925 09:58:19.208528 13773 tablet_bootstrap.cc:654] T dea7e34e5932444daa0ba1a11af3d2ea P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
W20260925 09:58:19.209306 13297 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
I20260925 09:58:19.209708 13297 leader_election.cc:302] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:58:19.211505 13781 raft_consensus.cc:2229] T 780164f28e2e45e38cfb4b47d7a6584c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:19.212704 13782 raft_consensus.cc:2726] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:58:19.213114 13781 raft_consensus.cc:2258] T 780164f28e2e45e38cfb4b47d7a6584c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:19.213518 13772 tablet_bootstrap.cc:654] T e75bfd761e314d60b6638eb5174b0227 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:19.216254 13780 log.cc:1196] T 64cfc93464ae4dc1bf6788105513fe8d P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/64cfc93464ae4dc1bf6788105513fe8d
I20260925 09:58:19.217200 13076 catalog_manager.cc:5587] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161) does not have the latest schema for tablet 2d0e5127e7f548a68dd73858c77f6f84 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]). Expected version 1 got 0
I20260925 09:58:19.218149 13781 ts_tablet_manager.cc:1905] T 780164f28e2e45e38cfb4b47d7a6584c P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:19.219278 13780 ts_tablet_manager.cc:1939] T 64cfc93464ae4dc1bf6788105513fe8d P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:19.232108 13780 tablet_replica.cc:330] T 761a36153d004cef86126c719cfe7946 P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:19.227602 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet 64cfc93464ae4dc1bf6788105513fe8d (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]) successfully deleted
I20260925 09:58:19.244643 13773 tablet_bootstrap.cc:492] T dea7e34e5932444daa0ba1a11af3d2ea P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:19.219779 13770 tablet_bootstrap.cc:654] T 2d0e5127e7f548a68dd73858c77f6f84 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:19.245074 13773 ts_tablet_manager.cc:1397] T dea7e34e5932444daa0ba1a11af3d2ea P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.058s user 0.009s sys 0.004s
I20260925 09:58:19.247946 13780 raft_consensus.cc:2229] T 761a36153d004cef86126c719cfe7946 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:58:19.249032 13780 raft_consensus.cc:2258] T 761a36153d004cef86126c719cfe7946 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:19.247364 13773 raft_consensus.cc:348] T dea7e34e5932444daa0ba1a11af3d2ea P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:19.251485 13772 tablet_bootstrap.cc:492] T e75bfd761e314d60b6638eb5174b0227 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:19.251581 13773 raft_consensus.cc:374] T dea7e34e5932444daa0ba1a11af3d2ea P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:19.251854 13772 ts_tablet_manager.cc:1397] T e75bfd761e314d60b6638eb5174b0227 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.054s user 0.005s sys 0.010s
I20260925 09:58:19.251828 13773 raft_consensus.cc:729] T dea7e34e5932444daa0ba1a11af3d2ea P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:19.252431 13773 consensus_queue.cc:260] T dea7e34e5932444daa0ba1a11af3d2ea P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:19.253218 13780 ts_tablet_manager.cc:1905] T 761a36153d004cef86126c719cfe7946 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:19.256747 13773 ts_tablet_manager.cc:1428] T dea7e34e5932444daa0ba1a11af3d2ea P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.011s user 0.004s sys 0.002s
I20260925 09:58:19.256748 13772 raft_consensus.cc:348] T e75bfd761e314d60b6638eb5174b0227 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:19.257402 13772 raft_consensus.cc:374] T e75bfd761e314d60b6638eb5174b0227 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:19.257649 13772 raft_consensus.cc:729] T e75bfd761e314d60b6638eb5174b0227 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:19.258220 13772 consensus_queue.cc:260] T e75bfd761e314d60b6638eb5174b0227 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:19.260535 13770 tablet_bootstrap.cc:492] T 2d0e5127e7f548a68dd73858c77f6f84 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:19.260911 13770 ts_tablet_manager.cc:1397] T 2d0e5127e7f548a68dd73858c77f6f84 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.183s user 0.007s sys 0.001s
I20260925 09:58:19.263274 13770 raft_consensus.cc:348] T 2d0e5127e7f548a68dd73858c77f6f84 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:19.263954 13770 raft_consensus.cc:374] T 2d0e5127e7f548a68dd73858c77f6f84 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:19.249327 13781 ts_tablet_manager.cc:1918] T 780164f28e2e45e38cfb4b47d7a6584c P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:19.264241 13770 raft_consensus.cc:729] T 2d0e5127e7f548a68dd73858c77f6f84 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:19.265152 13781 log.cc:1196] T 780164f28e2e45e38cfb4b47d7a6584c P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/780164f28e2e45e38cfb4b47d7a6584c
I20260925 09:58:19.264937 13770 consensus_queue.cc:260] T 2d0e5127e7f548a68dd73858c77f6f84 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:19.270879 13770 ts_tablet_manager.cc:1428] T 2d0e5127e7f548a68dd73858c77f6f84 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20260925 09:58:19.271703 13770 tablet_bootstrap.cc:492] T e75bfd761e314d60b6638eb5174b0227 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:19.271759 13076 catalog_manager.cc:5587] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731) does not have the latest schema for tablet 2d0e5127e7f548a68dd73858c77f6f84 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]). Expected version 1 got 0
I20260925 09:58:19.265799 13781 ts_tablet_manager.cc:1939] T 780164f28e2e45e38cfb4b47d7a6584c P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:19.280561 13076 catalog_manager.cc:5587] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731) does not have the latest schema for tablet 2d0e5127e7f548a68dd73858c77f6f84 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]). Expected version 1 got 0
I20260925 09:58:19.290603 13772 ts_tablet_manager.cc:1428] T e75bfd761e314d60b6638eb5174b0227 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.039s user 0.003s sys 0.002s
I20260925 09:58:19.292093 13912 raft_consensus.cc:482] T d48c0366f9b8428983f0844437d890a8 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:19.292522 13912 raft_consensus.cc:504] T d48c0366f9b8428983f0844437d890a8 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:19.294348 13912 leader_election.cc:288] T d48c0366f9b8428983f0844437d890a8 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:19.294323 13076 catalog_manager.cc:5587] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161) does not have the latest schema for tablet e75bfd761e314d60b6638eb5174b0227 (table default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]). Expected version 1 got 0
I20260925 09:58:19.296015 13362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d48c0366f9b8428983f0844437d890a8" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:19.296763 13781 tablet_replica.cc:330] T b67365d10ba2430594067740623783c9 P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:19.298189 13781 raft_consensus.cc:2229] T b67365d10ba2430594067740623783c9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260925 09:58:19.298527 13438 leader_election.cc:341] T d48c0366f9b8428983f0844437d890a8 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:19.300238 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d48c0366f9b8428983f0844437d890a8" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:19.301054 13772 tablet_bootstrap.cc:492] T b67365d10ba2430594067740623783c9 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
W20260925 09:58:19.302814 13440 leader_election.cc:341] T d48c0366f9b8428983f0844437d890a8 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:19.303231 13440 leader_election.cc:302] T d48c0366f9b8428983f0844437d890a8 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4d724d4b83e4d1e817a3a505dc24eb9; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, b3082cf15dc74c0480e5d33ecf592843
I20260925 09:58:19.298642 13781 raft_consensus.cc:2258] T b67365d10ba2430594067740623783c9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:19.305270 13912 raft_consensus.cc:2726] T d48c0366f9b8428983f0844437d890a8 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:58:19.308327 13781 ts_tablet_manager.cc:1905] T b67365d10ba2430594067740623783c9 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:19.309064 13076 catalog_manager.cc:5587] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161) does not have the latest schema for tablet e75bfd761e314d60b6638eb5174b0227 (table default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]). Expected version 1 got 0
I20260925 09:58:19.313524 13772 tablet_bootstrap.cc:654] T b67365d10ba2430594067740623783c9 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:19.320669 13770 tablet_bootstrap.cc:654] T e75bfd761e314d60b6638eb5174b0227 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:19.338367 13780 ts_tablet_manager.cc:1918] T 761a36153d004cef86126c719cfe7946 P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:58:19.338873 13780 log.cc:1196] T 761a36153d004cef86126c719cfe7946 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/761a36153d004cef86126c719cfe7946
I20260925 09:58:19.340715 13772 tablet_bootstrap.cc:492] T b67365d10ba2430594067740623783c9 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:19.340757 13781 ts_tablet_manager.cc:1918] T b67365d10ba2430594067740623783c9 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:19.341064 13781 log.cc:1196] T b67365d10ba2430594067740623783c9 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/b67365d10ba2430594067740623783c9
I20260925 09:58:19.341136 13772 ts_tablet_manager.cc:1397] T b67365d10ba2430594067740623783c9 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.040s user 0.001s sys 0.008s
I20260925 09:58:19.341681 13781 ts_tablet_manager.cc:1939] T b67365d10ba2430594067740623783c9 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:19.345279 13770 tablet_bootstrap.cc:492] T e75bfd761e314d60b6638eb5174b0227 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:19.345718 13770 ts_tablet_manager.cc:1397] T e75bfd761e314d60b6638eb5174b0227 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.074s user 0.003s sys 0.006s
I20260925 09:58:19.348140 13770 raft_consensus.cc:348] T e75bfd761e314d60b6638eb5174b0227 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:19.348845 13770 raft_consensus.cc:374] T e75bfd761e314d60b6638eb5174b0227 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:19.349117 13770 raft_consensus.cc:729] T e75bfd761e314d60b6638eb5174b0227 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:19.355132 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet b67365d10ba2430594067740623783c9 (table default.table_4c3437905d9e4b05ac1fb4c6e3d0a417 [id=7e0502ffa5544a5bb252a72020649cd7]) successfully deleted
I20260925 09:58:19.358613 13912 raft_consensus.cc:482] T dea7e34e5932444daa0ba1a11af3d2ea P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:19.359042 13912 raft_consensus.cc:504] T dea7e34e5932444daa0ba1a11af3d2ea P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:19.359712 13770 consensus_queue.cc:260] T e75bfd761e314d60b6638eb5174b0227 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:19.360895 13912 leader_election.cc:288] T dea7e34e5932444daa0ba1a11af3d2ea P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:19.343341 13772 raft_consensus.cc:348] T b67365d10ba2430594067740623783c9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:19.361224 13772 raft_consensus.cc:374] T b67365d10ba2430594067740623783c9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:19.361328 13076 master_service.cc:946] ReplaceTablet: received request to replace tablet af6ace3011c84603b32d0a409d5b8c1c from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:19.361475 13772 raft_consensus.cc:729] T b67365d10ba2430594067740623783c9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:19.363306 13362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea7e34e5932444daa0ba1a11af3d2ea" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:19.363369 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea7e34e5932444daa0ba1a11af3d2ea" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
W20260925 09:58:19.364897 13438 leader_election.cc:341] T dea7e34e5932444daa0ba1a11af3d2ea P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:19.370371 13076 catalog_manager.cc:5587] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731) does not have the latest schema for tablet e75bfd761e314d60b6638eb5174b0227 (table default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]). Expected version 1 got 0
W20260925 09:58:19.370639 13440 leader_election.cc:341] T dea7e34e5932444daa0ba1a11af3d2ea P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:19.371006 13770 ts_tablet_manager.cc:1428] T e75bfd761e314d60b6638eb5174b0227 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.025s user 0.006s sys 0.002s
I20260925 09:58:19.370199 13772 consensus_queue.cc:260] T b67365d10ba2430594067740623783c9 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:19.373813 13080 master_service.cc:946] ReplaceTablet: received request to replace tablet d48c0366f9b8428983f0844437d890a8 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:19.376205 13440 leader_election.cc:302] T dea7e34e5932444daa0ba1a11af3d2ea P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4d724d4b83e4d1e817a3a505dc24eb9; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, b3082cf15dc74c0480e5d33ecf592843
I20260925 09:58:19.380689 13078 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50556:
name: "default.Table_f83185097e7343848b564d3d5881c9ab"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:58:19.381312 13912 raft_consensus.cc:2726] T dea7e34e5932444daa0ba1a11af3d2ea P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:58:19.383317 13780 ts_tablet_manager.cc:1939] T 761a36153d004cef86126c719cfe7946 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:19.386559 13772 ts_tablet_manager.cc:1428] T b67365d10ba2430594067740623783c9 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.045s user 0.005s sys 0.001s
I20260925 09:58:19.387204 13770 tablet_bootstrap.cc:492] T dea7e34e5932444daa0ba1a11af3d2ea P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:19.389524 13079 catalog_manager.cc:5587] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731) does not have the latest schema for tablet e75bfd761e314d60b6638eb5174b0227 (table default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]). Expected version 1 got 0
I20260925 09:58:19.391144 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet 761a36153d004cef86126c719cfe7946 (table default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]) successfully deleted
I20260925 09:58:19.395727 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet b67365d10ba2430594067740623783c9 with delete_type TABLET_DATA_DELETED (replaced by tablet b103b022a84040ea97247d6503002f47) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:19.396384 13777 tablet_replica.cc:330] T b67365d10ba2430594067740623783c9 P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:19.397034 13777 raft_consensus.cc:2229] T b67365d10ba2430594067740623783c9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:19.397464 13777 raft_consensus.cc:2258] T b67365d10ba2430594067740623783c9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:19.398998 13777 ts_tablet_manager.cc:1905] T b67365d10ba2430594067740623783c9 P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:19.399672 13920 raft_consensus.cc:482] T 2d0e5127e7f548a68dd73858c77f6f84 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:19.400118 13920 raft_consensus.cc:504] T 2d0e5127e7f548a68dd73858c77f6f84 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:19.400739 13080 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet d48c0366f9b8428983f0844437d890a8
I20260925 09:58:19.401947 13920 leader_election.cc:288] T 2d0e5127e7f548a68dd73858c77f6f84 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
I20260925 09:58:19.403209 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d0e5127e7f548a68dd73858c77f6f84" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
I20260925 09:58:19.403985 13505 raft_consensus.cc:2454] T 2d0e5127e7f548a68dd73858c77f6f84 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 0.
I20260925 09:58:19.404913 13362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d0e5127e7f548a68dd73858c77f6f84" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:19.405566 13362 raft_consensus.cc:2454] T 2d0e5127e7f548a68dd73858c77f6f84 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 0.
I20260925 09:58:19.407863 13606 leader_election.cc:302] T 2d0e5127e7f548a68dd73858c77f6f84 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:19.408675 13920 raft_consensus.cc:2781] T 2d0e5127e7f548a68dd73858c77f6f84 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:19.408766 13649 tablet_service.cc:1514] Processing DeleteTablet for tablet d48c0366f9b8428983f0844437d890a8 with delete_type TABLET_DATA_DELETED (replaced by tablet cd13c4b390f54a7992438615fd477ea8) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:19.409001 13080 catalog_manager.cc:6565] ReplaceTablet: tablet d48c0366f9b8428983f0844437d890a8 deleted and replaced by tablet cd13c4b390f54a7992438615fd477ea8
I20260925 09:58:19.408981 13920 raft_consensus.cc:482] T 2d0e5127e7f548a68dd73858c77f6f84 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:19.409278 13920 raft_consensus.cc:3037] T 2d0e5127e7f548a68dd73858c77f6f84 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:19.410804 13076 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50556:
table { table_name: "default.Table_fb4b236f93c04bb08319a6915fa8796c" } modify_external_catalogs: true
I20260925 09:58:19.411536 13076 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50556:
table { table_name: "default.Table_fb4b236f93c04bb08319a6915fa8796c" } modify_external_catalogs: true
I20260925 09:58:19.412056 13483 tablet_service.cc:1514] Processing DeleteTablet for tablet d48c0366f9b8428983f0844437d890a8 with delete_type TABLET_DATA_DELETED (replaced by tablet cd13c4b390f54a7992438615fd477ea8) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:19.412583 13780 tablet_replica.cc:330] T 780164f28e2e45e38cfb4b47d7a6584c P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:19.412932 13781 tablet_replica.cc:330] T d48c0366f9b8428983f0844437d890a8 P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:19.413371 13780 raft_consensus.cc:2229] T 780164f28e2e45e38cfb4b47d7a6584c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:58:19.413627 13781 raft_consensus.cc:2229] T d48c0366f9b8428983f0844437d890a8 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:19.414088 13780 pending_rounds.cc:62] T 780164f28e2e45e38cfb4b47d7a6584c P a06c3ef7a6a94f7cbcc03e7cd232f237: Trying to abort 1 pending ops.
I20260925 09:58:19.414064 13781 raft_consensus.cc:2258] T d48c0366f9b8428983f0844437d890a8 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:19.414315 13780 pending_rounds.cc:68] T 780164f28e2e45e38cfb4b47d7a6584c P a06c3ef7a6a94f7cbcc03e7cd232f237: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192903417348096 op_type: NO_OP noop_request { }
I20260925 09:58:19.414824 13780 raft_consensus.cc:2866] T 780164f28e2e45e38cfb4b47d7a6584c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:58:19.415149 13780 raft_consensus.cc:2258] T 780164f28e2e45e38cfb4b47d7a6584c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:19.417222 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet d48c0366f9b8428983f0844437d890a8 with delete_type TABLET_DATA_DELETED (replaced by tablet cd13c4b390f54a7992438615fd477ea8) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:19.417596 13770 tablet_bootstrap.cc:654] T dea7e34e5932444daa0ba1a11af3d2ea P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:19.417986 13780 ts_tablet_manager.cc:1905] T 780164f28e2e45e38cfb4b47d7a6584c P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:19.418963 13781 ts_tablet_manager.cc:1905] T d48c0366f9b8428983f0844437d890a8 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:19.432688 13912 raft_consensus.cc:482] T e75bfd761e314d60b6638eb5174b0227 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:19.433189 13912 raft_consensus.cc:504] T e75bfd761e314d60b6638eb5174b0227 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:19.438154 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e75bfd761e314d60b6638eb5174b0227" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:19.438804 13671 raft_consensus.cc:2454] T e75bfd761e314d60b6638eb5174b0227 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 0.
I20260925 09:58:19.439632 13770 tablet_bootstrap.cc:492] T dea7e34e5932444daa0ba1a11af3d2ea P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:19.440019 13770 ts_tablet_manager.cc:1397] T dea7e34e5932444daa0ba1a11af3d2ea P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.068s user 0.009s sys 0.003s
I20260925 09:58:19.440372 13362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e75bfd761e314d60b6638eb5174b0227" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:19.441043 13362 raft_consensus.cc:2454] T e75bfd761e314d60b6638eb5174b0227 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 0.
I20260925 09:58:19.442271 13440 leader_election.cc:302] T e75bfd761e314d60b6638eb5174b0227 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:19.442420 13770 raft_consensus.cc:348] T dea7e34e5932444daa0ba1a11af3d2ea P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:19.435261 13912 leader_election.cc:288] T e75bfd761e314d60b6638eb5174b0227 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:19.444105 13912 raft_consensus.cc:2781] T e75bfd761e314d60b6638eb5174b0227 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:19.444391 13912 raft_consensus.cc:482] T e75bfd761e314d60b6638eb5174b0227 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:19.444669 13912 raft_consensus.cc:3037] T e75bfd761e314d60b6638eb5174b0227 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:19.443048 13770 raft_consensus.cc:374] T dea7e34e5932444daa0ba1a11af3d2ea P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:19.448786 13770 raft_consensus.cc:729] T dea7e34e5932444daa0ba1a11af3d2ea P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:19.449659 13770 consensus_queue.cc:260] T dea7e34e5932444daa0ba1a11af3d2ea P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:19.450145 13780 ts_tablet_manager.cc:1918] T 780164f28e2e45e38cfb4b47d7a6584c P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:19.450497 13780 log.cc:1196] T 780164f28e2e45e38cfb4b47d7a6584c P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/780164f28e2e45e38cfb4b47d7a6584c
I20260925 09:58:19.451368 13780 ts_tablet_manager.cc:1939] T 780164f28e2e45e38cfb4b47d7a6584c P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:19.454325 13780 tablet_replica.cc:330] T b67365d10ba2430594067740623783c9 P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:19.454974 13780 raft_consensus.cc:2229] T b67365d10ba2430594067740623783c9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:19.454008 13920 raft_consensus.cc:504] T 2d0e5127e7f548a68dd73858c77f6f84 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:19.455359 13780 raft_consensus.cc:2258] T b67365d10ba2430594067740623783c9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:19.457269 13780 ts_tablet_manager.cc:1905] T b67365d10ba2430594067740623783c9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:19.458398 13362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d0e5127e7f548a68dd73858c77f6f84" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843"
I20260925 09:58:19.459034 13362 raft_consensus.cc:3037] T 2d0e5127e7f548a68dd73858c77f6f84 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:19.465772 13362 raft_consensus.cc:2454] T 2d0e5127e7f548a68dd73858c77f6f84 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 1.
I20260925 09:58:19.468806 13770 ts_tablet_manager.cc:1428] T dea7e34e5932444daa0ba1a11af3d2ea P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.028s user 0.006s sys 0.001s
I20260925 09:58:19.474393 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d0e5127e7f548a68dd73858c77f6f84" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9"
I20260925 09:58:19.474957 13505 raft_consensus.cc:3037] T 2d0e5127e7f548a68dd73858c77f6f84 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:19.471549 13920 leader_election.cc:288] T 2d0e5127e7f548a68dd73858c77f6f84 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 election: Requested vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
I20260925 09:58:19.476555 13780 ts_tablet_manager.cc:1918] T b67365d10ba2430594067740623783c9 P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:19.476897 13780 log.cc:1196] T b67365d10ba2430594067740623783c9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/b67365d10ba2430594067740623783c9
I20260925 09:58:19.477443 13780 ts_tablet_manager.cc:1939] T b67365d10ba2430594067740623783c9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:19.479562 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet b67365d10ba2430594067740623783c9 (table default.table_4c3437905d9e4b05ac1fb4c6e3d0a417 [id=7e0502ffa5544a5bb252a72020649cd7]) successfully deleted
W20260925 09:58:19.481801 13068 catalog_manager.cc:4808] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): delete failed for tablet b67365d10ba2430594067740623783c9 because the tablet was not found. No further retry: Not found: Tablet not found: b67365d10ba2430594067740623783c9
W20260925 09:58:19.483310 13068 catalog_manager.cc:4821] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): delete failed for tablet d48c0366f9b8428983f0844437d890a8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d48c0366f9b8428983f0844437d890a8 already in progress: creating tablet
I20260925 09:58:19.483938 13770 tablet_bootstrap.cc:492] T d48c0366f9b8428983f0844437d890a8 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:19.469085 13772 tablet_bootstrap.cc:492] T dea7e34e5932444daa0ba1a11af3d2ea P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:19.494602 13770 tablet_bootstrap.cc:654] T d48c0366f9b8428983f0844437d890a8 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:19.497035 13772 tablet_bootstrap.cc:654] T dea7e34e5932444daa0ba1a11af3d2ea P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:19.501972 13603 leader_election.cc:302] T 2d0e5127e7f548a68dd73858c77f6f84 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, b3082cf15dc74c0480e5d33ecf592843; no voters:
I20260925 09:58:19.504266 13958 raft_consensus.cc:2781] T 2d0e5127e7f548a68dd73858c77f6f84 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:19.504877 13958 raft_consensus.cc:686] T 2d0e5127e7f548a68dd73858c77f6f84 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 LEADER]: Becoming Leader. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Running, Role: LEADER
I20260925 09:58:19.507735 13958 consensus_queue.cc:237] T 2d0e5127e7f548a68dd73858c77f6f84 P a06c3ef7a6a94f7cbcc03e7cd232f237 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:19.512676 13912 raft_consensus.cc:504] T e75bfd761e314d60b6638eb5174b0227 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:19.513828 13505 raft_consensus.cc:2454] T 2d0e5127e7f548a68dd73858c77f6f84 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 1.
I20260925 09:58:19.520119 13772 tablet_bootstrap.cc:492] T dea7e34e5932444daa0ba1a11af3d2ea P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:19.520567 13772 ts_tablet_manager.cc:1397] T dea7e34e5932444daa0ba1a11af3d2ea P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.052s user 0.008s sys 0.004s
I20260925 09:58:19.525808 13362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e75bfd761e314d60b6638eb5174b0227" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843"
I20260925 09:58:19.527146 13342 tablet_service.cc:1467] Processing CreateTablet for tablet cd13c4b390f54a7992438615fd477ea8 (DEFAULT_TABLE table=default.table_3823faf045944cc1a828ce929b8a7275 [id=f6ae78b7a67640f5a6e71e04249b4262]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:19.528174 13342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd13c4b390f54a7992438615fd477ea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:19.529131 13649 tablet_service.cc:1467] Processing CreateTablet for tablet cd13c4b390f54a7992438615fd477ea8 (DEFAULT_TABLE table=default.table_3823faf045944cc1a828ce929b8a7275 [id=f6ae78b7a67640f5a6e71e04249b4262]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:19.529753 13343 tablet_service.cc:1467] Processing CreateTablet for tablet 58670396dbce49baa196550f676a67f3 (DEFAULT_TABLE table=default.table_eadc20bcf90e4a72b8f26e3a4798e259 [id=f196d9d3750b4857905c1332d3ac763e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:19.530153 13649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd13c4b390f54a7992438615fd477ea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:19.530695 13343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58670396dbce49baa196550f676a67f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:19.532236 13483 tablet_service.cc:1467] Processing CreateTablet for tablet cd13c4b390f54a7992438615fd477ea8 (DEFAULT_TABLE table=default.table_3823faf045944cc1a828ce929b8a7275 [id=f6ae78b7a67640f5a6e71e04249b4262]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:19.532388 13341 tablet_service.cc:1467] Processing CreateTablet for tablet b103b022a84040ea97247d6503002f47 (DEFAULT_TABLE table=default.table_4c3437905d9e4b05ac1fb4c6e3d0a417 [id=7e0502ffa5544a5bb252a72020649cd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:19.533239 13483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd13c4b390f54a7992438615fd477ea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:19.533362 13341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b103b022a84040ea97247d6503002f47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:19.533243 13650 tablet_service.cc:1467] Processing CreateTablet for tablet 58670396dbce49baa196550f676a67f3 (DEFAULT_TABLE table=default.table_eadc20bcf90e4a72b8f26e3a4798e259 [id=f196d9d3750b4857905c1332d3ac763e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:19.534152 13650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58670396dbce49baa196550f676a67f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:19.538823 13485 tablet_service.cc:1467] Processing CreateTablet for tablet 58670396dbce49baa196550f676a67f3 (DEFAULT_TABLE table=default.table_eadc20bcf90e4a72b8f26e3a4798e259 [id=f196d9d3750b4857905c1332d3ac763e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:19.540079 13485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58670396dbce49baa196550f676a67f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:19.541884 13484 tablet_service.cc:1467] Processing CreateTablet for tablet b103b022a84040ea97247d6503002f47 (DEFAULT_TABLE table=default.table_4c3437905d9e4b05ac1fb4c6e3d0a417 [id=7e0502ffa5544a5bb252a72020649cd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:19.542161 13651 tablet_service.cc:1467] Processing CreateTablet for tablet b103b022a84040ea97247d6503002f47 (DEFAULT_TABLE table=default.table_4c3437905d9e4b05ac1fb4c6e3d0a417 [id=7e0502ffa5544a5bb252a72020649cd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:19.542945 13484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b103b022a84040ea97247d6503002f47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:19.543210 13651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b103b022a84040ea97247d6503002f47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:19.526440 13362 raft_consensus.cc:3037] T e75bfd761e314d60b6638eb5174b0227 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:19.569175 13770 tablet_bootstrap.cc:492] T d48c0366f9b8428983f0844437d890a8 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:19.569821 13770 ts_tablet_manager.cc:1397] T d48c0366f9b8428983f0844437d890a8 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.086s user 0.008s sys 0.004s
I20260925 09:58:19.573909 13777 ts_tablet_manager.cc:1918] T b67365d10ba2430594067740623783c9 P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:19.574255 13777 log.cc:1196] T b67365d10ba2430594067740623783c9 P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/b67365d10ba2430594067740623783c9
I20260925 09:58:19.579041 13781 ts_tablet_manager.cc:1918] T d48c0366f9b8428983f0844437d890a8 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:19.579233 13773 tablet_bootstrap.cc:492] T cd13c4b390f54a7992438615fd477ea8 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:19.579373 13781 log.cc:1196] T d48c0366f9b8428983f0844437d890a8 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/d48c0366f9b8428983f0844437d890a8
I20260925 09:58:19.582026 13912 leader_election.cc:288] T e75bfd761e314d60b6638eb5174b0227 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Requested vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:19.585582 13773 tablet_bootstrap.cc:654] T cd13c4b390f54a7992438615fd477ea8 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:19.596277 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e75bfd761e314d60b6638eb5174b0227" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
I20260925 09:58:19.596876 13671 raft_consensus.cc:3037] T e75bfd761e314d60b6638eb5174b0227 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:19.598881 13362 raft_consensus.cc:2454] T e75bfd761e314d60b6638eb5174b0227 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:19.603454 13671 raft_consensus.cc:2454] T e75bfd761e314d60b6638eb5174b0227 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:19.603919 13438 leader_election.cc:302] T e75bfd761e314d60b6638eb5174b0227 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:19.605073 13912 raft_consensus.cc:2781] T e75bfd761e314d60b6638eb5174b0227 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:19.605605 13912 raft_consensus.cc:686] T e75bfd761e314d60b6638eb5174b0227 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: Becoming Leader. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Running, Role: LEADER
I20260925 09:58:19.606390 13080 master_service.cc:946] ReplaceTablet: received request to replace tablet 2d0e5127e7f548a68dd73858c77f6f84 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:19.606446 13912 consensus_queue.cc:237] T e75bfd761e314d60b6638eb5174b0227 P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:19.610553 13770 raft_consensus.cc:348] T d48c0366f9b8428983f0844437d890a8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:19.611202 13770 raft_consensus.cc:374] T d48c0366f9b8428983f0844437d890a8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:19.611476 13770 raft_consensus.cc:729] T d48c0366f9b8428983f0844437d890a8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:19.612046 13770 consensus_queue.cc:260] T d48c0366f9b8428983f0844437d890a8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:19.616978 13770 ts_tablet_manager.cc:1428] T d48c0366f9b8428983f0844437d890a8 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.037s user 0.005s sys 0.000s
I20260925 09:58:19.619441 13770 tablet_bootstrap.cc:492] T 58670396dbce49baa196550f676a67f3 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:19.620359 13773 tablet_bootstrap.cc:492] T cd13c4b390f54a7992438615fd477ea8 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:19.620786 13773 ts_tablet_manager.cc:1397] T cd13c4b390f54a7992438615fd477ea8 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.042s user 0.010s sys 0.000s
I20260925 09:58:19.623041 13773 raft_consensus.cc:348] T cd13c4b390f54a7992438615fd477ea8 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } }
I20260925 09:58:19.623721 13773 raft_consensus.cc:374] T cd13c4b390f54a7992438615fd477ea8 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:19.623961 13773 raft_consensus.cc:729] T cd13c4b390f54a7992438615fd477ea8 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:19.582827 13772 raft_consensus.cc:348] T dea7e34e5932444daa0ba1a11af3d2ea P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:19.628046 13772 raft_consensus.cc:374] T dea7e34e5932444daa0ba1a11af3d2ea P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:19.628366 13772 raft_consensus.cc:729] T dea7e34e5932444daa0ba1a11af3d2ea P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:19.629009 13772 consensus_queue.cc:260] T dea7e34e5932444daa0ba1a11af3d2ea P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:19.633859 13773 consensus_queue.cc:260] T cd13c4b390f54a7992438615fd477ea8 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } }
I20260925 09:58:19.634419 13772 ts_tablet_manager.cc:1428] T dea7e34e5932444daa0ba1a11af3d2ea P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.114s user 0.005s sys 0.002s
I20260925 09:58:19.638698 13772 tablet_bootstrap.cc:492] T d48c0366f9b8428983f0844437d890a8 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:19.647142 13770 tablet_bootstrap.cc:654] T 58670396dbce49baa196550f676a67f3 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:19.647836 13080 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet 2d0e5127e7f548a68dd73858c77f6f84
I20260925 09:58:19.650604 13484 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d0e5127e7f548a68dd73858c77f6f84 with delete_type TABLET_DATA_DELETED (replaced by tablet 69ca132c80084e3eabc66a7ba5296e42) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:19.651002 13781 ts_tablet_manager.cc:1939] T d48c0366f9b8428983f0844437d890a8 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:19.651898 13773 ts_tablet_manager.cc:1428] T cd13c4b390f54a7992438615fd477ea8 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.031s user 0.007s sys 0.000s
I20260925 09:58:19.652817 13773 tablet_bootstrap.cc:492] T 58670396dbce49baa196550f676a67f3 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:19.574998 13777 ts_tablet_manager.cc:1939] T b67365d10ba2430594067740623783c9 P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:19.663736 13343 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d0e5127e7f548a68dd73858c77f6f84 with delete_type TABLET_DATA_DELETED (replaced by tablet 69ca132c80084e3eabc66a7ba5296e42) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:19.664055 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet b67365d10ba2430594067740623783c9 (table default.table_4c3437905d9e4b05ac1fb4c6e3d0a417 [id=7e0502ffa5544a5bb252a72020649cd7]) successfully deleted
I20260925 09:58:19.664595 13777 tablet_replica.cc:330] T 2d0e5127e7f548a68dd73858c77f6f84 P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
W20260925 09:58:19.665252 13066 catalog_manager.cc:4821] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): delete failed for tablet d48c0366f9b8428983f0844437d890a8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d48c0366f9b8428983f0844437d890a8 already in progress: creating tablet
I20260925 09:58:19.665414 13080 catalog_manager.cc:6565] ReplaceTablet: tablet 2d0e5127e7f548a68dd73858c77f6f84 deleted and replaced by tablet 69ca132c80084e3eabc66a7ba5296e42
I20260925 09:58:19.665704 13777 raft_consensus.cc:2229] T 2d0e5127e7f548a68dd73858c77f6f84 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:19.666254 13777 raft_consensus.cc:2258] T 2d0e5127e7f548a68dd73858c77f6f84 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:19.669049 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet d48c0366f9b8428983f0844437d890a8 (table default.table_3823faf045944cc1a828ce929b8a7275 [id=f6ae78b7a67640f5a6e71e04249b4262]) successfully deleted
I20260925 09:58:19.669737 13781 tablet_replica.cc:330] T 2d0e5127e7f548a68dd73858c77f6f84 P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:19.670960 13080 master_service.cc:946] ReplaceTablet: received request to replace tablet b103b022a84040ea97247d6503002f47 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:19.671936 13781 raft_consensus.cc:2229] T 2d0e5127e7f548a68dd73858c77f6f84 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:19.674324 13651 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d0e5127e7f548a68dd73858c77f6f84 with delete_type TABLET_DATA_DELETED (replaced by tablet 69ca132c80084e3eabc66a7ba5296e42) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:19.675035 13780 tablet_replica.cc:330] T 2d0e5127e7f548a68dd73858c77f6f84 P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:19.675738 13780 raft_consensus.cc:2229] T 2d0e5127e7f548a68dd73858c77f6f84 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:58:19.676344 13780 pending_rounds.cc:62] T 2d0e5127e7f548a68dd73858c77f6f84 P a06c3ef7a6a94f7cbcc03e7cd232f237: Trying to abort 1 pending ops.
I20260925 09:58:19.676548 13780 pending_rounds.cc:68] T 2d0e5127e7f548a68dd73858c77f6f84 P a06c3ef7a6a94f7cbcc03e7cd232f237: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192906816757760 op_type: NO_OP noop_request { }
I20260925 09:58:19.676877 13780 raft_consensus.cc:2866] T 2d0e5127e7f548a68dd73858c77f6f84 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:58:19.677155 13780 raft_consensus.cc:2258] T 2d0e5127e7f548a68dd73858c77f6f84 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:19.679950 13780 ts_tablet_manager.cc:1905] T 2d0e5127e7f548a68dd73858c77f6f84 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:19.686771 13772 tablet_bootstrap.cc:654] T d48c0366f9b8428983f0844437d890a8 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:19.687906 13651 tablet_service.cc:1514] Processing DeleteTablet for tablet d48c0366f9b8428983f0844437d890a8 with delete_type TABLET_DATA_DELETED (replaced by tablet cd13c4b390f54a7992438615fd477ea8) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:19.689043 13781 raft_consensus.cc:2258] T 2d0e5127e7f548a68dd73858c77f6f84 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:19.693988 13781 ts_tablet_manager.cc:1905] T 2d0e5127e7f548a68dd73858c77f6f84 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:19.668385 13777 ts_tablet_manager.cc:1905] T 2d0e5127e7f548a68dd73858c77f6f84 P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:19.711792 13080 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet b103b022a84040ea97247d6503002f47
I20260925 09:58:19.714641 13341 tablet_service.cc:1514] Processing DeleteTablet for tablet b103b022a84040ea97247d6503002f47 with delete_type TABLET_DATA_DELETED (replaced by tablet 09c76ac2e3a749b3afb478227d8ce7ee) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:19.715523 13651 tablet_service.cc:1514] Processing DeleteTablet for tablet b103b022a84040ea97247d6503002f47 with delete_type TABLET_DATA_DELETED (replaced by tablet 09c76ac2e3a749b3afb478227d8ce7ee) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:19.696976 13773 tablet_bootstrap.cc:654] T 58670396dbce49baa196550f676a67f3 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:19.714670 13484 tablet_service.cc:1514] Processing DeleteTablet for tablet b103b022a84040ea97247d6503002f47 with delete_type TABLET_DATA_DELETED (replaced by tablet 09c76ac2e3a749b3afb478227d8ce7ee) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:19.722659 13080 catalog_manager.cc:6565] ReplaceTablet: tablet b103b022a84040ea97247d6503002f47 deleted and replaced by tablet 09c76ac2e3a749b3afb478227d8ce7ee
I20260925 09:58:19.737648 13912 raft_consensus.cc:482] T cd13c4b390f54a7992438615fd477ea8 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:19.738379 13912 raft_consensus.cc:504] T cd13c4b390f54a7992438615fd477ea8 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } }
I20260925 09:58:19.742472 13362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd13c4b390f54a7992438615fd477ea8" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
W20260925 09:58:19.744415 13438 leader_election.cc:341] T cd13c4b390f54a7992438615fd477ea8 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:19.747242 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd13c4b390f54a7992438615fd477ea8" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
W20260925 09:58:19.749240 13440 leader_election.cc:341] T cd13c4b390f54a7992438615fd477ea8 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:19.749811 13440 leader_election.cc:302] T cd13c4b390f54a7992438615fd477ea8 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4d724d4b83e4d1e817a3a505dc24eb9; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, b3082cf15dc74c0480e5d33ecf592843
I20260925 09:58:19.750847 13956 raft_consensus.cc:2726] T cd13c4b390f54a7992438615fd477ea8 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:58:19.751698 13912 leader_election.cc:288] T cd13c4b390f54a7992438615fd477ea8 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:19.759142 13780 ts_tablet_manager.cc:1918] T 2d0e5127e7f548a68dd73858c77f6f84 P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:19.766592 13341 tablet_service.cc:1467] Processing CreateTablet for tablet 69ca132c80084e3eabc66a7ba5296e42 (DEFAULT_TABLE table=default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:19.766229 13080 catalog_manager.cc:5495] T e75bfd761e314d60b6638eb5174b0227 P c4d724d4b83e4d1e817a3a505dc24eb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2). New cstate: current_term: 1 leader_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: UNKNOWN } } }
I20260925 09:58:19.767633 13080 catalog_manager.cc:5587] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327) does not have the latest schema for tablet e75bfd761e314d60b6638eb5174b0227 (table default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]). Expected version 1 got 0
I20260925 09:58:19.768203 13341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69ca132c80084e3eabc66a7ba5296e42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:19.772171 13773 tablet_bootstrap.cc:492] T 58670396dbce49baa196550f676a67f3 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:19.772614 13773 ts_tablet_manager.cc:1397] T 58670396dbce49baa196550f676a67f3 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.120s user 0.005s sys 0.004s
I20260925 09:58:19.777138 13484 tablet_service.cc:1467] Processing CreateTablet for tablet 69ca132c80084e3eabc66a7ba5296e42 (DEFAULT_TABLE table=default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:19.778290 13484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69ca132c80084e3eabc66a7ba5296e42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:19.779152 13773 raft_consensus.cc:348] T 58670396dbce49baa196550f676a67f3 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } }
I20260925 09:58:19.779843 13773 raft_consensus.cc:374] T 58670396dbce49baa196550f676a67f3 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:19.780073 13773 raft_consensus.cc:729] T 58670396dbce49baa196550f676a67f3 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:19.780679 13773 consensus_queue.cc:260] T 58670396dbce49baa196550f676a67f3 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } }
I20260925 09:58:19.783506 13773 ts_tablet_manager.cc:1428] T 58670396dbce49baa196550f676a67f3 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20260925 09:58:19.784292 13773 tablet_bootstrap.cc:492] T b103b022a84040ea97247d6503002f47 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:19.794412 13651 tablet_service.cc:1467] Processing CreateTablet for tablet 69ca132c80084e3eabc66a7ba5296e42 (DEFAULT_TABLE table=default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:19.795786 13772 tablet_bootstrap.cc:492] T d48c0366f9b8428983f0844437d890a8 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:19.796175 13651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69ca132c80084e3eabc66a7ba5296e42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:19.796352 13772 ts_tablet_manager.cc:1397] T d48c0366f9b8428983f0844437d890a8 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.161s user 0.013s sys 0.002s
I20260925 09:58:19.799047 13770 tablet_bootstrap.cc:492] T 58670396dbce49baa196550f676a67f3 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:19.799453 13770 ts_tablet_manager.cc:1397] T 58670396dbce49baa196550f676a67f3 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.180s user 0.052s sys 0.080s
I20260925 09:58:19.801658 13770 raft_consensus.cc:348] T 58670396dbce49baa196550f676a67f3 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } }
I20260925 09:58:19.802371 13770 raft_consensus.cc:374] T 58670396dbce49baa196550f676a67f3 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:19.802632 13770 raft_consensus.cc:729] T 58670396dbce49baa196550f676a67f3 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:19.811575 13780 log.cc:1196] T 2d0e5127e7f548a68dd73858c77f6f84 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/2d0e5127e7f548a68dd73858c77f6f84
I20260925 09:58:19.810372 13770 consensus_queue.cc:260] T 58670396dbce49baa196550f676a67f3 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } }
I20260925 09:58:19.812783 13780 ts_tablet_manager.cc:1939] T 2d0e5127e7f548a68dd73858c77f6f84 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:19.798858 13772 raft_consensus.cc:348] T d48c0366f9b8428983f0844437d890a8 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:19.815215 13772 raft_consensus.cc:374] T d48c0366f9b8428983f0844437d890a8 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:19.815546 13772 raft_consensus.cc:729] T d48c0366f9b8428983f0844437d890a8 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:19.816226 13772 consensus_queue.cc:260] T d48c0366f9b8428983f0844437d890a8 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:19.818064 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet 2d0e5127e7f548a68dd73858c77f6f84 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]) successfully deleted
I20260925 09:58:19.822402 13770 ts_tablet_manager.cc:1428] T 58670396dbce49baa196550f676a67f3 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.023s user 0.004s sys 0.001s
I20260925 09:58:19.826848 13772 ts_tablet_manager.cc:1428] T d48c0366f9b8428983f0844437d890a8 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.030s user 0.006s sys 0.000s
I20260925 09:58:19.828917 13781 ts_tablet_manager.cc:1918] T 2d0e5127e7f548a68dd73858c77f6f84 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:19.829291 13781 log.cc:1196] T 2d0e5127e7f548a68dd73858c77f6f84 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/2d0e5127e7f548a68dd73858c77f6f84
I20260925 09:58:19.829771 13777 ts_tablet_manager.cc:1918] T 2d0e5127e7f548a68dd73858c77f6f84 P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:19.830124 13777 log.cc:1196] T 2d0e5127e7f548a68dd73858c77f6f84 P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/2d0e5127e7f548a68dd73858c77f6f84
I20260925 09:58:19.830729 13777 ts_tablet_manager.cc:1939] T 2d0e5127e7f548a68dd73858c77f6f84 P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:19.830502 13770 tablet_bootstrap.cc:492] T cd13c4b390f54a7992438615fd477ea8 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:19.841742 13772 tablet_bootstrap.cc:492] T b103b022a84040ea97247d6503002f47 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:19.843611 13770 tablet_bootstrap.cc:654] T cd13c4b390f54a7992438615fd477ea8 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:19.846843 13772 tablet_bootstrap.cc:654] T b103b022a84040ea97247d6503002f47 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:19.848928 13780 tablet_replica.cc:330] T d48c0366f9b8428983f0844437d890a8 P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:19.849072 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet 2d0e5127e7f548a68dd73858c77f6f84 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]) successfully deleted
I20260925 09:58:19.849716 13780 raft_consensus.cc:2229] T d48c0366f9b8428983f0844437d890a8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:19.850163 13780 raft_consensus.cc:2258] T d48c0366f9b8428983f0844437d890a8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260925 09:58:19.851600 13066 catalog_manager.cc:4821] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): delete failed for tablet b103b022a84040ea97247d6503002f47 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b103b022a84040ea97247d6503002f47 already in progress: creating tablet
I20260925 09:58:19.858508 13780 ts_tablet_manager.cc:1905] T d48c0366f9b8428983f0844437d890a8 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:19.860564 13341 tablet_service.cc:1514] Processing DeleteTablet for tablet d48c0366f9b8428983f0844437d890a8 with delete_type TABLET_DATA_DELETED (replaced by tablet cd13c4b390f54a7992438615fd477ea8) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:19.861450 13777 tablet_replica.cc:330] T d48c0366f9b8428983f0844437d890a8 P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:19.862381 13777 raft_consensus.cc:2229] T d48c0366f9b8428983f0844437d890a8 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:19.862862 13777 raft_consensus.cc:2258] T d48c0366f9b8428983f0844437d890a8 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:19.864990 13777 ts_tablet_manager.cc:1905] T d48c0366f9b8428983f0844437d890a8 P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:19.874449 13773 tablet_bootstrap.cc:654] T b103b022a84040ea97247d6503002f47 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:19.881062 13782 raft_consensus.cc:482] T dea7e34e5932444daa0ba1a11af3d2ea P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:19.881517 13782 raft_consensus.cc:504] T dea7e34e5932444daa0ba1a11af3d2ea P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:19.885423 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea7e34e5932444daa0ba1a11af3d2ea" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:19.885548 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea7e34e5932444daa0ba1a11af3d2ea" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
I20260925 09:58:19.886070 13671 raft_consensus.cc:2454] T dea7e34e5932444daa0ba1a11af3d2ea P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 0.
I20260925 09:58:19.886348 13505 raft_consensus.cc:2454] T dea7e34e5932444daa0ba1a11af3d2ea P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 0.
I20260925 09:58:19.887818 13297 leader_election.cc:302] T dea7e34e5932444daa0ba1a11af3d2ea P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, b3082cf15dc74c0480e5d33ecf592843; no voters:
I20260925 09:58:19.883297 13782 leader_election.cc:288] T dea7e34e5932444daa0ba1a11af3d2ea P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:19.890352 13782 raft_consensus.cc:2781] T dea7e34e5932444daa0ba1a11af3d2ea P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:19.890771 13782 raft_consensus.cc:482] T dea7e34e5932444daa0ba1a11af3d2ea P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:19.891139 13782 raft_consensus.cc:3037] T dea7e34e5932444daa0ba1a11af3d2ea P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:19.829913 13781 ts_tablet_manager.cc:1939] T 2d0e5127e7f548a68dd73858c77f6f84 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:19.894570 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet 2d0e5127e7f548a68dd73858c77f6f84 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]) successfully deleted
W20260925 09:58:19.895293 13069 catalog_manager.cc:4821] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): delete failed for tablet b103b022a84040ea97247d6503002f47 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b103b022a84040ea97247d6503002f47 already in progress: creating tablet
I20260925 09:58:19.901852 13782 raft_consensus.cc:504] T dea7e34e5932444daa0ba1a11af3d2ea P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:19.905758 13073 catalog_manager.cc:3411] renamed table default.table_4c3437905d9e4b05ac1fb4c6e3d0a417 [id=7e0502ffa5544a5bb252a72020649cd7] in HMS: new name default.table_e59da12e7b95405787a2f9f73efdc76b
I20260925 09:58:19.911201 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea7e34e5932444daa0ba1a11af3d2ea" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9"
I20260925 09:58:19.911827 13505 raft_consensus.cc:3037] T dea7e34e5932444daa0ba1a11af3d2ea P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:19.913692 13079 master_service.cc:946] ReplaceTablet: received request to replace tablet d48c0366f9b8428983f0844437d890a8 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:19.918772 13505 raft_consensus.cc:2454] T dea7e34e5932444daa0ba1a11af3d2ea P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 1.
I20260925 09:58:19.919329 13341 tablet_service.cc:1467] Processing CreateTablet for tablet 09c76ac2e3a749b3afb478227d8ce7ee (DEFAULT_TABLE table=default.table_4c3437905d9e4b05ac1fb4c6e3d0a417 [id=7e0502ffa5544a5bb252a72020649cd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:19.919987 13298 leader_election.cc:302] T dea7e34e5932444daa0ba1a11af3d2ea P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:19.920480 13341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09c76ac2e3a749b3afb478227d8ce7ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:19.922444 13772 tablet_bootstrap.cc:492] T b103b022a84040ea97247d6503002f47 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:19.922820 13772 ts_tablet_manager.cc:1397] T b103b022a84040ea97247d6503002f47 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.081s user 0.008s sys 0.000s
I20260925 09:58:19.925015 13772 raft_consensus.cc:348] T b103b022a84040ea97247d6503002f47 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:19.925698 13772 raft_consensus.cc:374] T b103b022a84040ea97247d6503002f47 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:19.925988 13772 raft_consensus.cc:729] T b103b022a84040ea97247d6503002f47 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:19.926674 13772 consensus_queue.cc:260] T b103b022a84040ea97247d6503002f47 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:19.927831 13969 raft_consensus.cc:2781] T dea7e34e5932444daa0ba1a11af3d2ea P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:19.933076 13782 leader_election.cc:288] T dea7e34e5932444daa0ba1a11af3d2ea P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Requested vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:19.936991 13969 raft_consensus.cc:686] T dea7e34e5932444daa0ba1a11af3d2ea P b3082cf15dc74c0480e5d33ecf592843 [term 1 LEADER]: Becoming Leader. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Running, Role: LEADER
I20260925 09:58:19.937839 13969 consensus_queue.cc:237] T dea7e34e5932444daa0ba1a11af3d2ea P b3082cf15dc74c0480e5d33ecf592843 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:19.944005 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea7e34e5932444daa0ba1a11af3d2ea" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
I20260925 09:58:19.944581 13671 raft_consensus.cc:3037] T dea7e34e5932444daa0ba1a11af3d2ea P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:19.947741 13770 tablet_bootstrap.cc:492] T cd13c4b390f54a7992438615fd477ea8 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:19.948141 13770 ts_tablet_manager.cc:1397] T cd13c4b390f54a7992438615fd477ea8 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.118s user 0.011s sys 0.020s
W20260925 09:58:19.948446 13067 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 2d0e5127e7f548a68dd73858c77f6f84 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 2d0e5127e7f548a68dd73858c77f6f84 consensus state has no leader
I20260925 09:58:19.950325 13770 raft_consensus.cc:348] T cd13c4b390f54a7992438615fd477ea8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } }
I20260925 09:58:19.950970 13770 raft_consensus.cc:374] T cd13c4b390f54a7992438615fd477ea8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:19.954968 13671 raft_consensus.cc:2454] T dea7e34e5932444daa0ba1a11af3d2ea P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 1.
I20260925 09:58:19.956523 13772 ts_tablet_manager.cc:1428] T b103b022a84040ea97247d6503002f47 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.033s user 0.004s sys 0.002s
I20260925 09:58:19.951200 13770 raft_consensus.cc:729] T cd13c4b390f54a7992438615fd477ea8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:19.957399 13770 consensus_queue.cc:260] T cd13c4b390f54a7992438615fd477ea8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } }
I20260925 09:58:19.962111 13772 tablet_bootstrap.cc:492] T 58670396dbce49baa196550f676a67f3 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:19.966053 13080 master_service.cc:946] ReplaceTablet: received request to replace tablet cd13c4b390f54a7992438615fd477ea8 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:19.970103 13777 ts_tablet_manager.cc:1918] T d48c0366f9b8428983f0844437d890a8 P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:19.970525 13777 log.cc:1196] T d48c0366f9b8428983f0844437d890a8 P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/d48c0366f9b8428983f0844437d890a8
I20260925 09:58:19.973310 13651 tablet_service.cc:1467] Processing CreateTablet for tablet 09c76ac2e3a749b3afb478227d8ce7ee (DEFAULT_TABLE table=default.table_4c3437905d9e4b05ac1fb4c6e3d0a417 [id=7e0502ffa5544a5bb252a72020649cd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:19.974627 13651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09c76ac2e3a749b3afb478227d8ce7ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:19.980847 13341 tablet_service.cc:1514] Processing DeleteTablet for tablet b103b022a84040ea97247d6503002f47 with delete_type TABLET_DATA_DELETED (replaced by tablet 09c76ac2e3a749b3afb478227d8ce7ee) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:19.959259 13770 ts_tablet_manager.cc:1428] T cd13c4b390f54a7992438615fd477ea8 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.011s user 0.004s sys 0.000s
I20260925 09:58:19.984825 13485 tablet_service.cc:1467] Processing CreateTablet for tablet 09c76ac2e3a749b3afb478227d8ce7ee (DEFAULT_TABLE table=default.table_4c3437905d9e4b05ac1fb4c6e3d0a417 [id=7e0502ffa5544a5bb252a72020649cd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:19.985945 13485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09c76ac2e3a749b3afb478227d8ce7ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:19.997174 13773 tablet_bootstrap.cc:492] T b103b022a84040ea97247d6503002f47 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:19.997608 13773 ts_tablet_manager.cc:1397] T b103b022a84040ea97247d6503002f47 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.213s user 0.011s sys 0.002s
I20260925 09:58:20.000641 13772 tablet_bootstrap.cc:654] T 58670396dbce49baa196550f676a67f3 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:19.997663 13770 tablet_bootstrap.cc:492] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:20.005899 13080 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet cd13c4b390f54a7992438615fd477ea8
I20260925 09:58:20.010140 13780 ts_tablet_manager.cc:1918] T d48c0366f9b8428983f0844437d890a8 P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:20.014780 13341 tablet_service.cc:1514] Processing DeleteTablet for tablet cd13c4b390f54a7992438615fd477ea8 with delete_type TABLET_DATA_DELETED (replaced by tablet 59af1da7da284d1a81c7cc3eda283997) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:20.015373 13773 raft_consensus.cc:348] T b103b022a84040ea97247d6503002f47 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:20.016079 13773 raft_consensus.cc:374] T b103b022a84040ea97247d6503002f47 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:20.016340 13773 raft_consensus.cc:729] T b103b022a84040ea97247d6503002f47 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:20.017035 13773 consensus_queue.cc:260] T b103b022a84040ea97247d6503002f47 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:20.020103 13649 tablet_service.cc:1514] Processing DeleteTablet for tablet cd13c4b390f54a7992438615fd477ea8 with delete_type TABLET_DATA_DELETED (replaced by tablet 59af1da7da284d1a81c7cc3eda283997) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:20.016498 13780 log.cc:1196] T d48c0366f9b8428983f0844437d890a8 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/d48c0366f9b8428983f0844437d890a8
I20260925 09:58:20.026934 13780 ts_tablet_manager.cc:1939] T d48c0366f9b8428983f0844437d890a8 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:20.029244 13958 raft_consensus.cc:482] T cd13c4b390f54a7992438615fd477ea8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:20.029289 13484 tablet_service.cc:1514] Processing DeleteTablet for tablet cd13c4b390f54a7992438615fd477ea8 with delete_type TABLET_DATA_DELETED (replaced by tablet 59af1da7da284d1a81c7cc3eda283997) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:20.029985 13781 tablet_replica.cc:330] T cd13c4b390f54a7992438615fd477ea8 P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:20.029707 13958 raft_consensus.cc:504] T cd13c4b390f54a7992438615fd477ea8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } }
I20260925 09:58:20.017653 13770 tablet_bootstrap.cc:654] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:20.030943 13777 ts_tablet_manager.cc:1939] T d48c0366f9b8428983f0844437d890a8 P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:20.032567 13362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd13c4b390f54a7992438615fd477ea8" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:20.032944 13958 leader_election.cc:288] T cd13c4b390f54a7992438615fd477ea8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
I20260925 09:58:20.034200 13777 tablet_replica.cc:330] T b103b022a84040ea97247d6503002f47 P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:20.034418 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet d48c0366f9b8428983f0844437d890a8 (table default.table_3823faf045944cc1a828ce929b8a7275 [id=f6ae78b7a67640f5a6e71e04249b4262]) successfully deleted
I20260925 09:58:20.034549 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd13c4b390f54a7992438615fd477ea8" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
I20260925 09:58:20.034858 13777 raft_consensus.cc:2229] T b103b022a84040ea97247d6503002f47 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:20.035004 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet d48c0366f9b8428983f0844437d890a8 (table default.table_3823faf045944cc1a828ce929b8a7275 [id=f6ae78b7a67640f5a6e71e04249b4262]) successfully deleted
I20260925 09:58:20.035202 13505 raft_consensus.cc:2454] T cd13c4b390f54a7992438615fd477ea8 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 0.
I20260925 09:58:20.028777 13080 catalog_manager.cc:6565] ReplaceTablet: tablet cd13c4b390f54a7992438615fd477ea8 deleted and replaced by tablet 59af1da7da284d1a81c7cc3eda283997
W20260925 09:58:20.036450 13603 leader_election.cc:341] T cd13c4b390f54a7992438615fd477ea8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:20.037474 13606 leader_election.cc:302] T cd13c4b390f54a7992438615fd477ea8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9; no voters: b3082cf15dc74c0480e5d33ecf592843
I20260925 09:58:20.038604 13958 raft_consensus.cc:2781] T cd13c4b390f54a7992438615fd477ea8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:20.038942 13958 raft_consensus.cc:482] T cd13c4b390f54a7992438615fd477ea8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:20.040592 13958 raft_consensus.cc:3037] T cd13c4b390f54a7992438615fd477ea8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:20.048146 13958 raft_consensus.cc:504] T cd13c4b390f54a7992438615fd477ea8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } }
I20260925 09:58:20.051646 13781 raft_consensus.cc:2229] T cd13c4b390f54a7992438615fd477ea8 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:20.052173 13781 raft_consensus.cc:2258] T cd13c4b390f54a7992438615fd477ea8 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260925 09:58:20.052588 13068 catalog_manager.cc:4821] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): delete failed for tablet b103b022a84040ea97247d6503002f47 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b103b022a84040ea97247d6503002f47 already in progress: creating tablet
I20260925 09:58:20.054388 13781 ts_tablet_manager.cc:1905] T cd13c4b390f54a7992438615fd477ea8 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:20.055402 13780 tablet_replica.cc:330] T cd13c4b390f54a7992438615fd477ea8 P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:20.056211 13362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd13c4b390f54a7992438615fd477ea8" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843"
I20260925 09:58:20.057487 13780 raft_consensus.cc:2229] T cd13c4b390f54a7992438615fd477ea8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260925 09:58:20.057732 13603 leader_election.cc:341] T cd13c4b390f54a7992438615fd477ea8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:20.064061 13773 ts_tablet_manager.cc:1428] T b103b022a84040ea97247d6503002f47 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.066s user 0.003s sys 0.004s
I20260925 09:58:20.067435 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd13c4b390f54a7992438615fd477ea8" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9"
I20260925 09:58:20.067986 13772 tablet_bootstrap.cc:492] T 58670396dbce49baa196550f676a67f3 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
W20260925 09:58:20.069205 13606 leader_election.cc:341] T cd13c4b390f54a7992438615fd477ea8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Illegal state: Tablet not RUNNING: STOPPED
I20260925 09:58:20.069628 13606 leader_election.cc:302] T cd13c4b390f54a7992438615fd477ea8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237; no voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:58:20.070996 13958 leader_election.cc:288] T cd13c4b390f54a7992438615fd477ea8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 election: Requested vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
I20260925 09:58:20.068523 13772 ts_tablet_manager.cc:1397] T 58670396dbce49baa196550f676a67f3 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.107s user 0.010s sys 0.000s
I20260925 09:58:20.072321 13080 catalog_manager.cc:5495] T dea7e34e5932444daa0ba1a11af3d2ea P b3082cf15dc74c0480e5d33ecf592843 reported cstate change: term changed from 0 to 1, leader changed from <none> to b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b3082cf15dc74c0480e5d33ecf592843" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: UNKNOWN } } }
I20260925 09:58:20.075435 13780 raft_consensus.cc:2258] T cd13c4b390f54a7992438615fd477ea8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:20.075814 13958 raft_consensus.cc:2684] T cd13c4b390f54a7992438615fd477ea8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Received election callback for term 1 while not running: Illegal state: RaftConsensus is not running: State = Stopped
I20260925 09:58:20.076267 13770 tablet_bootstrap.cc:492] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:20.035283 13777 raft_consensus.cc:2258] T b103b022a84040ea97247d6503002f47 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:20.076671 13770 ts_tablet_manager.cc:1397] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.093s user 0.009s sys 0.000s
I20260925 09:58:20.077454 13773 tablet_bootstrap.cc:492] T 69ca132c80084e3eabc66a7ba5296e42 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:20.078325 13341 tablet_service.cc:1467] Processing CreateTablet for tablet 59af1da7da284d1a81c7cc3eda283997 (DEFAULT_TABLE table=default.table_3823faf045944cc1a828ce929b8a7275 [id=f6ae78b7a67640f5a6e71e04249b4262]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:20.079460 13341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59af1da7da284d1a81c7cc3eda283997. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:20.085059 13780 ts_tablet_manager.cc:1905] T cd13c4b390f54a7992438615fd477ea8 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:20.085623 13777 ts_tablet_manager.cc:1905] T b103b022a84040ea97247d6503002f47 P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:20.074864 13772 raft_consensus.cc:348] T 58670396dbce49baa196550f676a67f3 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } }
I20260925 09:58:20.078868 13770 raft_consensus.cc:348] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:20.091804 13772 raft_consensus.cc:374] T 58670396dbce49baa196550f676a67f3 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:20.091991 13770 raft_consensus.cc:374] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:20.090164 13485 tablet_service.cc:1467] Processing CreateTablet for tablet 59af1da7da284d1a81c7cc3eda283997 (DEFAULT_TABLE table=default.table_3823faf045944cc1a828ce929b8a7275 [id=f6ae78b7a67640f5a6e71e04249b4262]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:20.092072 13772 raft_consensus.cc:729] T 58670396dbce49baa196550f676a67f3 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:20.092236 13770 raft_consensus.cc:729] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:20.092757 13772 consensus_queue.cc:260] T 58670396dbce49baa196550f676a67f3 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } }
I20260925 09:58:20.095798 13485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59af1da7da284d1a81c7cc3eda283997. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:20.096272 13772 ts_tablet_manager.cc:1428] T 58670396dbce49baa196550f676a67f3 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.024s user 0.004s sys 0.001s
I20260925 09:58:20.102169 13772 tablet_bootstrap.cc:492] T cd13c4b390f54a7992438615fd477ea8 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:20.105443 13773 tablet_bootstrap.cc:654] T 69ca132c80084e3eabc66a7ba5296e42 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:20.105269 13649 tablet_service.cc:1467] Processing CreateTablet for tablet 59af1da7da284d1a81c7cc3eda283997 (DEFAULT_TABLE table=default.table_3823faf045944cc1a828ce929b8a7275 [id=f6ae78b7a67640f5a6e71e04249b4262]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:20.106330 13649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59af1da7da284d1a81c7cc3eda283997. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:20.094262 13770 consensus_queue.cc:260] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:20.120780 13770 ts_tablet_manager.cc:1428] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.044s user 0.006s sys 0.000s
I20260925 09:58:20.125641 13772 tablet_bootstrap.cc:654] T cd13c4b390f54a7992438615fd477ea8 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:20.128927 13770 tablet_bootstrap.cc:492] T 69ca132c80084e3eabc66a7ba5296e42 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:20.137751 13671 raft_consensus.cc:1261] T e75bfd761e314d60b6638eb5174b0227 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Refusing update from remote peer c4d724d4b83e4d1e817a3a505dc24eb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:58:20.140694 13770 tablet_bootstrap.cc:654] T 69ca132c80084e3eabc66a7ba5296e42 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:20.141654 13362 raft_consensus.cc:1261] T e75bfd761e314d60b6638eb5174b0227 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Refusing update from remote peer c4d724d4b83e4d1e817a3a505dc24eb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:58:20.143502 13956 consensus_queue.cc:1035] T e75bfd761e314d60b6638eb5174b0227 P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:20.144848 13956 consensus_queue.cc:1035] T e75bfd761e314d60b6638eb5174b0227 P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:20.151835 13484 tablet_service.cc:1514] Processing DeleteTablet for tablet b103b022a84040ea97247d6503002f47 with delete_type TABLET_DATA_DELETED (replaced by tablet 09c76ac2e3a749b3afb478227d8ce7ee) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:20.153359 13786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 09:58:20.156013 13786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 09:58:20.164170 13770 tablet_bootstrap.cc:492] T 69ca132c80084e3eabc66a7ba5296e42 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:20.164562 13770 ts_tablet_manager.cc:1397] T 69ca132c80084e3eabc66a7ba5296e42 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.036s user 0.004s sys 0.006s
I20260925 09:58:20.167074 13079 master_service.cc:946] ReplaceTablet: received request to replace tablet 69ca132c80084e3eabc66a7ba5296e42 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:20.167480 13780 ts_tablet_manager.cc:1918] T cd13c4b390f54a7992438615fd477ea8 P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:20.167910 13780 log.cc:1196] T cd13c4b390f54a7992438615fd477ea8 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/cd13c4b390f54a7992438615fd477ea8
I20260925 09:58:20.168910 13781 ts_tablet_manager.cc:1918] T cd13c4b390f54a7992438615fd477ea8 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:20.169370 13781 log.cc:1196] T cd13c4b390f54a7992438615fd477ea8 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/cd13c4b390f54a7992438615fd477ea8
I20260925 09:58:20.170151 13770 raft_consensus.cc:348] T 69ca132c80084e3eabc66a7ba5296e42 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:20.170083 13781 ts_tablet_manager.cc:1939] T cd13c4b390f54a7992438615fd477ea8 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:20.170819 13770 raft_consensus.cc:374] T 69ca132c80084e3eabc66a7ba5296e42 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:20.171308 13770 raft_consensus.cc:729] T 69ca132c80084e3eabc66a7ba5296e42 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:20.172283 13770 consensus_queue.cc:260] T 69ca132c80084e3eabc66a7ba5296e42 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:20.180869 13770 ts_tablet_manager.cc:1428] T 69ca132c80084e3eabc66a7ba5296e42 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.016s user 0.006s sys 0.000s
I20260925 09:58:20.182945 13786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 09:58:20.184875 13770 tablet_bootstrap.cc:492] T 09c76ac2e3a749b3afb478227d8ce7ee P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:20.195199 13781 tablet_replica.cc:330] T b103b022a84040ea97247d6503002f47 P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:20.196204 13781 raft_consensus.cc:2229] T b103b022a84040ea97247d6503002f47 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:20.198594 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet cd13c4b390f54a7992438615fd477ea8 (table default.table_3823faf045944cc1a828ce929b8a7275 [id=f6ae78b7a67640f5a6e71e04249b4262]) successfully deleted
I20260925 09:58:20.203666 13770 tablet_bootstrap.cc:654] T 09c76ac2e3a749b3afb478227d8ce7ee P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:20.209916 13777 ts_tablet_manager.cc:1918] T b103b022a84040ea97247d6503002f47 P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:20.210536 13781 raft_consensus.cc:2258] T b103b022a84040ea97247d6503002f47 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:20.212608 13781 ts_tablet_manager.cc:1905] T b103b022a84040ea97247d6503002f47 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:20.206502 13773 tablet_bootstrap.cc:492] T 69ca132c80084e3eabc66a7ba5296e42 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:20.213224 13773 ts_tablet_manager.cc:1397] T 69ca132c80084e3eabc66a7ba5296e42 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.148s user 0.004s sys 0.006s
W20260925 09:58:20.214280 13080 catalog_manager.cc:5328] Ignoring report from unknown tablet b103b022a84040ea97247d6503002f47
I20260925 09:58:20.215363 13080 catalog_manager.cc:5587] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731) does not have the latest schema for tablet e75bfd761e314d60b6638eb5174b0227 (table default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]). Expected version 1 got 0
I20260925 09:58:20.216701 13773 raft_consensus.cc:348] T 69ca132c80084e3eabc66a7ba5296e42 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:20.217311 13773 raft_consensus.cc:374] T 69ca132c80084e3eabc66a7ba5296e42 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:20.217551 13773 raft_consensus.cc:729] T 69ca132c80084e3eabc66a7ba5296e42 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:20.218197 13773 consensus_queue.cc:260] T 69ca132c80084e3eabc66a7ba5296e42 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:20.229360 13983 tablet.cc:1722] T e75bfd761e314d60b6638eb5174b0227 P a06c3ef7a6a94f7cbcc03e7cd232f237: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:58:20.232297 13982 tablet.cc:1722] T e75bfd761e314d60b6638eb5174b0227 P c4d724d4b83e4d1e817a3a505dc24eb9: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:58:20.248571 13780 ts_tablet_manager.cc:1939] T cd13c4b390f54a7992438615fd477ea8 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:20.238509 13079 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet 69ca132c80084e3eabc66a7ba5296e42
I20260925 09:58:20.252700 13484 tablet_service.cc:1514] Processing DeleteTablet for tablet 69ca132c80084e3eabc66a7ba5296e42 with delete_type TABLET_DATA_DELETED (replaced by tablet 3735920fa0a64b588480af485d0fd6f0) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:20.253974 13079 catalog_manager.cc:6565] ReplaceTablet: tablet 69ca132c80084e3eabc66a7ba5296e42 deleted and replaced by tablet 3735920fa0a64b588480af485d0fd6f0
I20260925 09:58:20.256850 13649 tablet_service.cc:1514] Processing DeleteTablet for tablet 69ca132c80084e3eabc66a7ba5296e42 with delete_type TABLET_DATA_DELETED (replaced by tablet 3735920fa0a64b588480af485d0fd6f0) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:20.264528 13912 raft_consensus.cc:482] T 58670396dbce49baa196550f676a67f3 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:20.265028 13912 raft_consensus.cc:504] T 58670396dbce49baa196550f676a67f3 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } }
I20260925 09:58:20.269369 13912 leader_election.cc:288] T 58670396dbce49baa196550f676a67f3 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:20.270121 13773 ts_tablet_manager.cc:1428] T 69ca132c80084e3eabc66a7ba5296e42 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.056s user 0.004s sys 0.001s
I20260925 09:58:20.271064 13773 tablet_bootstrap.cc:492] T 09c76ac2e3a749b3afb478227d8ce7ee P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:20.278245 13770 tablet_bootstrap.cc:492] T 09c76ac2e3a749b3afb478227d8ce7ee P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:20.278787 13770 ts_tablet_manager.cc:1397] T 09c76ac2e3a749b3afb478227d8ce7ee P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.094s user 0.017s sys 0.025s
I20260925 09:58:20.281028 13670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58670396dbce49baa196550f676a67f3" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:20.281793 13670 raft_consensus.cc:2454] T 58670396dbce49baa196550f676a67f3 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 0.
I20260925 09:58:20.283048 13440 leader_election.cc:302] T 58670396dbce49baa196550f676a67f3 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:20.281258 13770 raft_consensus.cc:348] T 09c76ac2e3a749b3afb478227d8ce7ee P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:20.289216 13770 raft_consensus.cc:374] T 09c76ac2e3a749b3afb478227d8ce7ee P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:20.289501 13770 raft_consensus.cc:729] T 09c76ac2e3a749b3afb478227d8ce7ee P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:20.254514 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet 69ca132c80084e3eabc66a7ba5296e42 with delete_type TABLET_DATA_DELETED (replaced by tablet 3735920fa0a64b588480af485d0fd6f0) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:20.291651 13912 raft_consensus.cc:2781] T 58670396dbce49baa196550f676a67f3 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:20.291970 13777 log.cc:1196] T b103b022a84040ea97247d6503002f47 P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/b103b022a84040ea97247d6503002f47
I20260925 09:58:20.292121 13912 raft_consensus.cc:482] T 58670396dbce49baa196550f676a67f3 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:20.292060 13770 consensus_queue.cc:260] T 09c76ac2e3a749b3afb478227d8ce7ee P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:20.292536 13912 raft_consensus.cc:3037] T 58670396dbce49baa196550f676a67f3 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:20.292954 13777 ts_tablet_manager.cc:1939] T b103b022a84040ea97247d6503002f47 P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:20.259263 13772 tablet_bootstrap.cc:492] T cd13c4b390f54a7992438615fd477ea8 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:20.267967 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58670396dbce49baa196550f676a67f3" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:20.295727 13772 ts_tablet_manager.cc:1397] T cd13c4b390f54a7992438615fd477ea8 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.194s user 0.011s sys 0.002s
I20260925 09:58:20.295778 13363 raft_consensus.cc:2454] T 58670396dbce49baa196550f676a67f3 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 0.
I20260925 09:58:20.298022 13772 raft_consensus.cc:348] T cd13c4b390f54a7992438615fd477ea8 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } }
I20260925 09:58:20.298785 13772 raft_consensus.cc:374] T cd13c4b390f54a7992438615fd477ea8 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:20.299043 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet b103b022a84040ea97247d6503002f47 (table default.table_4c3437905d9e4b05ac1fb4c6e3d0a417 [id=7e0502ffa5544a5bb252a72020649cd7]) successfully deleted
I20260925 09:58:20.299093 13772 raft_consensus.cc:729] T cd13c4b390f54a7992438615fd477ea8 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:20.307886 13780 tablet_replica.cc:330] T 69ca132c80084e3eabc66a7ba5296e42 P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:20.308102 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet cd13c4b390f54a7992438615fd477ea8 (table default.table_3823faf045944cc1a828ce929b8a7275 [id=f6ae78b7a67640f5a6e71e04249b4262]) successfully deleted
I20260925 09:58:20.308737 13772 consensus_queue.cc:260] T cd13c4b390f54a7992438615fd477ea8 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } }
I20260925 09:58:20.316151 13772 ts_tablet_manager.cc:1428] T cd13c4b390f54a7992438615fd477ea8 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.020s user 0.006s sys 0.000s
I20260925 09:58:20.317014 13772 tablet_bootstrap.cc:492] T 69ca132c80084e3eabc66a7ba5296e42 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:20.320825 13770 ts_tablet_manager.cc:1428] T 09c76ac2e3a749b3afb478227d8ce7ee P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.042s user 0.006s sys 0.000s
I20260925 09:58:20.321290 13780 raft_consensus.cc:2229] T 69ca132c80084e3eabc66a7ba5296e42 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:20.321754 13780 raft_consensus.cc:2258] T 69ca132c80084e3eabc66a7ba5296e42 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:20.326669 13770 tablet_bootstrap.cc:492] T 59af1da7da284d1a81c7cc3eda283997 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:20.334976 13342 tablet_service.cc:1467] Processing CreateTablet for tablet 3735920fa0a64b588480af485d0fd6f0 (DEFAULT_TABLE table=default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:20.336153 13342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3735920fa0a64b588480af485d0fd6f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:20.337841 13079 catalog_manager.cc:5587] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161) does not have the latest schema for tablet e75bfd761e314d60b6638eb5174b0227 (table default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]). Expected version 1 got 0
I20260925 09:58:20.341148 13912 raft_consensus.cc:504] T 58670396dbce49baa196550f676a67f3 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } }
I20260925 09:58:20.345276 13362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58670396dbce49baa196550f676a67f3" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843"
I20260925 09:58:20.345902 13362 raft_consensus.cc:3037] T 58670396dbce49baa196550f676a67f3 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:20.351676 13485 tablet_service.cc:1467] Processing CreateTablet for tablet 3735920fa0a64b588480af485d0fd6f0 (DEFAULT_TABLE table=default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:20.352717 13485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3735920fa0a64b588480af485d0fd6f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:20.352361 13649 tablet_service.cc:1467] Processing CreateTablet for tablet 3735920fa0a64b588480af485d0fd6f0 (DEFAULT_TABLE table=default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:20.353569 13649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3735920fa0a64b588480af485d0fd6f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:20.341603 13780 ts_tablet_manager.cc:1905] T 69ca132c80084e3eabc66a7ba5296e42 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:20.364032 13362 raft_consensus.cc:2454] T 58670396dbce49baa196550f676a67f3 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:20.365410 13438 leader_election.cc:302] T 58670396dbce49baa196550f676a67f3 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:20.367024 13773 tablet_bootstrap.cc:654] T 09c76ac2e3a749b3afb478227d8ce7ee P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:20.367820 13079 master_service.cc:946] ReplaceTablet: received request to replace tablet 09c76ac2e3a749b3afb478227d8ce7ee from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:20.341598 13770 tablet_bootstrap.cc:654] T 59af1da7da284d1a81c7cc3eda283997 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:20.371147 13772 tablet_bootstrap.cc:654] T 69ca132c80084e3eabc66a7ba5296e42 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:20.378780 13990 raft_consensus.cc:2781] T 58670396dbce49baa196550f676a67f3 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:20.383278 13990 raft_consensus.cc:686] T 58670396dbce49baa196550f676a67f3 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: Becoming Leader. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Running, Role: LEADER
I20260925 09:58:20.384502 13990 consensus_queue.cc:237] T 58670396dbce49baa196550f676a67f3 P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } }
I20260925 09:58:20.385725 13670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58670396dbce49baa196550f676a67f3" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
I20260925 09:58:20.386289 13670 raft_consensus.cc:3037] T 58670396dbce49baa196550f676a67f3 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:20.394484 13670 raft_consensus.cc:2454] T 58670396dbce49baa196550f676a67f3 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:20.398357 13912 leader_election.cc:288] T 58670396dbce49baa196550f676a67f3 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Requested vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:20.371411 13777 tablet_replica.cc:330] T cd13c4b390f54a7992438615fd477ea8 P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:20.418542 13077 catalog_manager.cc:2148] created HMS catalog entry for table [id=0cdc8d8fef5445668c0b0107243e0ae9]
W20260925 09:58:20.423130 13081 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_1238f9df428247dc9b9ca7c59823bfe4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 09:58:20.425398 13075 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_4d3f82d91a5749679343f60e3dbbc422 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:58:20.428184 13777 raft_consensus.cc:2229] T cd13c4b390f54a7992438615fd477ea8 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:20.428707 13777 raft_consensus.cc:2258] T cd13c4b390f54a7992438615fd477ea8 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:20.388824 13991 tablet.cc:1722] T e75bfd761e314d60b6638eb5174b0227 P b3082cf15dc74c0480e5d33ecf592843: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:58:20.436642 13777 ts_tablet_manager.cc:1905] T cd13c4b390f54a7992438615fd477ea8 P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:20.447366 13770 tablet_bootstrap.cc:492] T 59af1da7da284d1a81c7cc3eda283997 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:20.449079 13770 ts_tablet_manager.cc:1397] T 59af1da7da284d1a81c7cc3eda283997 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.123s user 0.009s sys 0.001s
I20260925 09:58:20.459734 13770 raft_consensus.cc:348] T 59af1da7da284d1a81c7cc3eda283997 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:20.461592 13770 raft_consensus.cc:374] T 59af1da7da284d1a81c7cc3eda283997 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:20.462268 13770 raft_consensus.cc:729] T 59af1da7da284d1a81c7cc3eda283997 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:20.471194 13077 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:50556 (request call id 45) took 2016 ms. Trace:
I20260925 09:58:20.471639 13077 rpcz_store.cc:276] 0925 09:58:18.454512 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:18.454617 (+ 105us) service_pool.cc:227] Handling call
0925 09:58:19.192451 (+737834us) catalog_manager.cc:2075] Acquired catalog manager lock
0925 09:58:19.195043 (+ 2592us) catalog_manager.cc:2121] Created new table and tablet info
0925 09:58:20.418499 (+1223456us) catalog_manager.cc:2147] Created new table in HMS catalog
0925 09:58:20.468169 (+ 49670us) catalog_manager.cc:2181] Wrote table and tablets to system table
0925 09:58:20.470852 (+ 2683us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0925 09:58:20.471133 (+ 281us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:58:20.420193 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:58:20.420523 (+ 330us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:58:20.420545 (+ 22us) write_op.cc:435] Acquired schema lock
0925 09:58:20.420557 (+ 12us) tablet.cc:660] Decoding operations
0925 09:58:20.420798 (+ 241us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:58:20.420833 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0925 09:58:20.420855 (+ 22us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:58:20.421045 (+ 190us) tablet.cc:699] Row locks acquired
0925 09:58:20.421059 (+ 14us) write_op.cc:260] PREPARE: finished
0925 09:58:20.421164 (+ 105us) write_op.cc:270] Start()
0925 09:58:20.421240 (+ 76us) write_op.cc:276] Timestamp: P: 1790330300421127 usec, L: 0
0925 09:58:20.421255 (+ 15us) op_driver.cc:331] REPLICATION: starting
0925 09:58:20.421694 (+ 439us) log.cc:842] Serialized 432 byte log entry
0925 09:58:20.465298 (+ 43604us) op_driver.cc:444] REPLICATION: finished
0925 09:58:20.465558 (+ 260us) write_op.cc:301] APPLY: starting
0925 09:58:20.465610 (+ 52us) tablet.cc:1365] starting BulkCheckPresence
0925 09:58:20.465709 (+ 99us) tablet.cc:1368] finished BulkCheckPresence
0925 09:58:20.465789 (+ 80us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:58:20.466153 (+ 364us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:58:20.466284 (+ 131us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 09:58:20.466323 (+ 39us) write_op.cc:312] APPLY: finished
0925 09:58:20.466761 (+ 438us) log.cc:842] Serialized 28 byte log entry
0925 09:58:20.466914 (+ 153us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:20.466994 (+ 80us) write_op.cc:454] Released schema lock
0925 09:58:20.467028 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0925 09:58:20.467210 (+ 182us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:20.467230 (+ 20us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":709650,"Hive Metastore.run_cpu_time_us":645,"Hive Metastore.run_wall_time_us":465440,"child_traces":[["op",{"apply.queue_time_us":105,"apply.run_cpu_time_us":1614,"apply.run_wall_time_us":1816,"num_ops":2,"prepare.queue_time_us":88,"prepare.run_cpu_time_us":1694,"prepare.run_wall_time_us":1694,"replication_time_us":43958}]]}
I20260925 09:58:20.479815 13773 tablet_bootstrap.cc:492] T 09c76ac2e3a749b3afb478227d8ce7ee P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:20.480264 13773 ts_tablet_manager.cc:1397] T 09c76ac2e3a749b3afb478227d8ce7ee P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.209s user 0.011s sys 0.000s
W20260925 09:58:20.477782 13067 consensus_peers.cc:487] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 -> Peer 61e29afab0cf46819cbffc91248bf761 (127.10.9.60:42083): Couldn't send request to peer 61e29afab0cf46819cbffc91248bf761. Status: Network error: Client connection negotiation failed: client connection to 127.10.9.60:42083: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260925 09:58:20.482640 13773 raft_consensus.cc:348] T 09c76ac2e3a749b3afb478227d8ce7ee P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:20.478506 13079 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet 09c76ac2e3a749b3afb478227d8ce7ee
I20260925 09:58:20.484855 13777 ts_tablet_manager.cc:1918] T cd13c4b390f54a7992438615fd477ea8 P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:20.485203 13777 log.cc:1196] T cd13c4b390f54a7992438615fd477ea8 P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/cd13c4b390f54a7992438615fd477ea8
I20260925 09:58:20.485836 13777 ts_tablet_manager.cc:1939] T cd13c4b390f54a7992438615fd477ea8 P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:20.492650 13772 tablet_bootstrap.cc:492] T 69ca132c80084e3eabc66a7ba5296e42 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:20.493244 13772 ts_tablet_manager.cc:1397] T 69ca132c80084e3eabc66a7ba5296e42 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.176s user 0.009s sys 0.005s
I20260925 09:58:20.500738 13781 ts_tablet_manager.cc:1918] T b103b022a84040ea97247d6503002f47 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:20.501124 13781 log.cc:1196] T b103b022a84040ea97247d6503002f47 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/b103b022a84040ea97247d6503002f47
I20260925 09:58:20.501775 13781 ts_tablet_manager.cc:1939] T b103b022a84040ea97247d6503002f47 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:20.499203 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet cd13c4b390f54a7992438615fd477ea8 (table default.table_3823faf045944cc1a828ce929b8a7275 [id=f6ae78b7a67640f5a6e71e04249b4262]) successfully deleted
W20260925 09:58:20.502496 13080 catalog_manager.cc:5328] Ignoring report from unknown tablet cd13c4b390f54a7992438615fd477ea8
W20260925 09:58:20.503163 13066 catalog_manager.cc:4821] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): delete failed for tablet 69ca132c80084e3eabc66a7ba5296e42 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 69ca132c80084e3eabc66a7ba5296e42 already in progress: creating tablet
I20260925 09:58:20.503770 13079 catalog_manager.cc:6565] ReplaceTablet: tablet 09c76ac2e3a749b3afb478227d8ce7ee deleted and replaced by tablet a9d515adc84446e6b46b0332d3ea10db
I20260925 09:58:20.504294 13781 tablet_replica.cc:330] T 69ca132c80084e3eabc66a7ba5296e42 P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:20.504931 13649 tablet_service.cc:1514] Processing DeleteTablet for tablet 09c76ac2e3a749b3afb478227d8ce7ee with delete_type TABLET_DATA_DELETED (replaced by tablet a9d515adc84446e6b46b0332d3ea10db) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:20.505034 13781 raft_consensus.cc:2229] T 69ca132c80084e3eabc66a7ba5296e42 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:20.505524 13781 raft_consensus.cc:2258] T 69ca132c80084e3eabc66a7ba5296e42 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:20.463873 13770 consensus_queue.cc:260] T 59af1da7da284d1a81c7cc3eda283997 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:20.506289 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet 09c76ac2e3a749b3afb478227d8ce7ee with delete_type TABLET_DATA_DELETED (replaced by tablet a9d515adc84446e6b46b0332d3ea10db) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:20.506428 13485 tablet_service.cc:1514] Processing DeleteTablet for tablet 09c76ac2e3a749b3afb478227d8ce7ee with delete_type TABLET_DATA_DELETED (replaced by tablet a9d515adc84446e6b46b0332d3ea10db) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:20.502976 13077 catalog_manager.cc:5495] T 58670396dbce49baa196550f676a67f3 P c4d724d4b83e4d1e817a3a505dc24eb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2). New cstate: current_term: 1 leader_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: HEALTHY } } }
W20260925 09:58:20.507825 13066 catalog_manager.cc:4821] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): delete failed for tablet 09c76ac2e3a749b3afb478227d8ce7ee because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 09c76ac2e3a749b3afb478227d8ce7ee already in progress: creating tablet
I20260925 09:58:20.505331 13079 master_service.cc:946] ReplaceTablet: received request to replace tablet dea7e34e5932444daa0ba1a11af3d2ea from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:20.506474 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet b103b022a84040ea97247d6503002f47 (table default.table_4c3437905d9e4b05ac1fb4c6e3d0a417 [id=7e0502ffa5544a5bb252a72020649cd7]) successfully deleted
I20260925 09:58:20.501437 13772 raft_consensus.cc:348] T 69ca132c80084e3eabc66a7ba5296e42 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:20.518184 13770 ts_tablet_manager.cc:1428] T 59af1da7da284d1a81c7cc3eda283997 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.068s user 0.006s sys 0.000s
I20260925 09:58:20.518581 13772 raft_consensus.cc:374] T 69ca132c80084e3eabc66a7ba5296e42 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:20.518934 13772 raft_consensus.cc:729] T 69ca132c80084e3eabc66a7ba5296e42 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:20.520571 13772 consensus_queue.cc:260] T 69ca132c80084e3eabc66a7ba5296e42 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:20.526816 13780 ts_tablet_manager.cc:1918] T 69ca132c80084e3eabc66a7ba5296e42 P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:20.524082 13772 ts_tablet_manager.cc:1428] T 69ca132c80084e3eabc66a7ba5296e42 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.030s user 0.006s sys 0.000s
I20260925 09:58:20.528776 13780 log.cc:1196] T 69ca132c80084e3eabc66a7ba5296e42 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/69ca132c80084e3eabc66a7ba5296e42
I20260925 09:58:20.529549 13772 tablet_bootstrap.cc:492] T 09c76ac2e3a749b3afb478227d8ce7ee P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:20.531553 13780 ts_tablet_manager.cc:1939] T 69ca132c80084e3eabc66a7ba5296e42 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:20.536732 13772 tablet_bootstrap.cc:654] T 09c76ac2e3a749b3afb478227d8ce7ee P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:20.541468 13780 tablet_replica.cc:330] T 09c76ac2e3a749b3afb478227d8ce7ee P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:20.541616 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet 69ca132c80084e3eabc66a7ba5296e42 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]) successfully deleted
I20260925 09:58:20.542274 13780 raft_consensus.cc:2229] T 09c76ac2e3a749b3afb478227d8ce7ee P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:20.542701 13780 raft_consensus.cc:2258] T 09c76ac2e3a749b3afb478227d8ce7ee P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:20.483289 13773 raft_consensus.cc:374] T 09c76ac2e3a749b3afb478227d8ce7ee P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:20.547721 13773 raft_consensus.cc:729] T 09c76ac2e3a749b3afb478227d8ce7ee P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:20.548478 13773 consensus_queue.cc:260] T 09c76ac2e3a749b3afb478227d8ce7ee P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:20.556016 13079 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet dea7e34e5932444daa0ba1a11af3d2ea
I20260925 09:58:20.557562 13079 catalog_manager.cc:6565] ReplaceTablet: tablet dea7e34e5932444daa0ba1a11af3d2ea deleted and replaced by tablet 3da072989f69498e99e5e3b32bdb6859
I20260925 09:58:20.557929 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet dea7e34e5932444daa0ba1a11af3d2ea with delete_type TABLET_DATA_DELETED (replaced by tablet 3da072989f69498e99e5e3b32bdb6859) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:20.558614 13777 tablet_replica.cc:330] T dea7e34e5932444daa0ba1a11af3d2ea P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:20.559262 13777 raft_consensus.cc:2229] T dea7e34e5932444daa0ba1a11af3d2ea P b3082cf15dc74c0480e5d33ecf592843 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:58:20.559937 13777 pending_rounds.cc:62] T dea7e34e5932444daa0ba1a11af3d2ea P b3082cf15dc74c0480e5d33ecf592843: Trying to abort 1 pending ops.
I20260925 09:58:20.560139 13777 pending_rounds.cc:68] T dea7e34e5932444daa0ba1a11af3d2ea P b3082cf15dc74c0480e5d33ecf592843: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192908575084544 op_type: NO_OP noop_request { }
I20260925 09:58:20.560480 13777 raft_consensus.cc:2866] T dea7e34e5932444daa0ba1a11af3d2ea P b3082cf15dc74c0480e5d33ecf592843 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:58:20.560782 13777 raft_consensus.cc:2258] T dea7e34e5932444daa0ba1a11af3d2ea P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:20.561259 13649 tablet_service.cc:1514] Processing DeleteTablet for tablet dea7e34e5932444daa0ba1a11af3d2ea with delete_type TABLET_DATA_DELETED (replaced by tablet 3da072989f69498e99e5e3b32bdb6859) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:20.562448 13780 ts_tablet_manager.cc:1905] T 09c76ac2e3a749b3afb478227d8ce7ee P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:20.567651 13777 ts_tablet_manager.cc:1905] T dea7e34e5932444daa0ba1a11af3d2ea P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:20.569813 13485 tablet_service.cc:1514] Processing DeleteTablet for tablet dea7e34e5932444daa0ba1a11af3d2ea with delete_type TABLET_DATA_DELETED (replaced by tablet 3da072989f69498e99e5e3b32bdb6859) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:20.569963 13080 catalog_manager.cc:5587] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161) does not have the latest schema for tablet e75bfd761e314d60b6638eb5174b0227 (table default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]). Expected version 1 got 0
I20260925 09:58:20.573081 13770 tablet_bootstrap.cc:492] T 3735920fa0a64b588480af485d0fd6f0 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:20.589535 13773 ts_tablet_manager.cc:1428] T 09c76ac2e3a749b3afb478227d8ce7ee P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.109s user 0.006s sys 0.000s
I20260925 09:58:20.592967 13773 tablet_bootstrap.cc:492] T 59af1da7da284d1a81c7cc3eda283997 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:20.598099 13773 tablet_bootstrap.cc:654] T 59af1da7da284d1a81c7cc3eda283997 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:20.608651 13770 tablet_bootstrap.cc:654] T 3735920fa0a64b588480af485d0fd6f0 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:20.613746 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet 69ca132c80084e3eabc66a7ba5296e42 with delete_type TABLET_DATA_DELETED (replaced by tablet 3735920fa0a64b588480af485d0fd6f0) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:20.626724 13342 tablet_service.cc:1467] Processing CreateTablet for tablet 1db61f5876974206be17c93462236bb9 (DEFAULT_TABLE table=default.table_8f71132b5a004ad89115f7a2278b95d6 [id=0cdc8d8fef5445668c0b0107243e0ae9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:20.627995 13342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1db61f5876974206be17c93462236bb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:20.629184 13780 ts_tablet_manager.cc:1918] T 09c76ac2e3a749b3afb478227d8ce7ee P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:20.629609 13780 log.cc:1196] T 09c76ac2e3a749b3afb478227d8ce7ee P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/09c76ac2e3a749b3afb478227d8ce7ee
W20260925 09:58:20.630606 13269 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a9d515adc84446e6b46b0332d3ea10db consensus state has no leader
I20260925 09:58:20.632577 13780 ts_tablet_manager.cc:1939] T 09c76ac2e3a749b3afb478227d8ce7ee P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:20.634857 13780 tablet_replica.cc:330] T dea7e34e5932444daa0ba1a11af3d2ea P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:20.635118 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet 09c76ac2e3a749b3afb478227d8ce7ee (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]) successfully deleted
I20260925 09:58:20.635561 13780 raft_consensus.cc:2229] T dea7e34e5932444daa0ba1a11af3d2ea P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:20.636022 13780 raft_consensus.cc:2258] T dea7e34e5932444daa0ba1a11af3d2ea P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:20.600016 13781 ts_tablet_manager.cc:1905] T 69ca132c80084e3eabc66a7ba5296e42 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:20.643312 13780 ts_tablet_manager.cc:1905] T dea7e34e5932444daa0ba1a11af3d2ea P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:20.644273 13649 tablet_service.cc:1467] Processing CreateTablet for tablet 1db61f5876974206be17c93462236bb9 (DEFAULT_TABLE table=default.table_8f71132b5a004ad89115f7a2278b95d6 [id=0cdc8d8fef5445668c0b0107243e0ae9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:20.645216 13649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1db61f5876974206be17c93462236bb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:20.664556 13485 tablet_service.cc:1467] Processing CreateTablet for tablet 1db61f5876974206be17c93462236bb9 (DEFAULT_TABLE table=default.table_8f71132b5a004ad89115f7a2278b95d6 [id=0cdc8d8fef5445668c0b0107243e0ae9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:20.665753 13485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1db61f5876974206be17c93462236bb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:20.671531 13781 ts_tablet_manager.cc:1918] T 69ca132c80084e3eabc66a7ba5296e42 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:20.673712 13781 log.cc:1196] T 69ca132c80084e3eabc66a7ba5296e42 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/69ca132c80084e3eabc66a7ba5296e42
I20260925 09:58:20.674226 13341 tablet_service.cc:1467] Processing CreateTablet for tablet a9d515adc84446e6b46b0332d3ea10db (DEFAULT_TABLE table=default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:20.674945 13781 ts_tablet_manager.cc:1939] T 69ca132c80084e3eabc66a7ba5296e42 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:20.675446 13341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9d515adc84446e6b46b0332d3ea10db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:20.677227 13770 tablet_bootstrap.cc:492] T 3735920fa0a64b588480af485d0fd6f0 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:20.677606 13770 ts_tablet_manager.cc:1397] T 3735920fa0a64b588480af485d0fd6f0 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.105s user 0.003s sys 0.009s
I20260925 09:58:20.673401 13773 tablet_bootstrap.cc:492] T 59af1da7da284d1a81c7cc3eda283997 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:20.679538 13772 tablet_bootstrap.cc:492] T 09c76ac2e3a749b3afb478227d8ce7ee P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:20.680049 13772 ts_tablet_manager.cc:1397] T 09c76ac2e3a749b3afb478227d8ce7ee P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.151s user 0.009s sys 0.011s
I20260925 09:58:20.686295 13773 ts_tablet_manager.cc:1397] T 59af1da7da284d1a81c7cc3eda283997 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.093s user 0.007s sys 0.005s
I20260925 09:58:20.686765 13079 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50556:
name: "default.Table_2c3c283350d644f39f1903713ab06589"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:58:20.689479 13773 raft_consensus.cc:348] T 59af1da7da284d1a81c7cc3eda283997 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:20.684763 13772 raft_consensus.cc:348] T 09c76ac2e3a749b3afb478227d8ce7ee P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:20.690146 13773 raft_consensus.cc:374] T 59af1da7da284d1a81c7cc3eda283997 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:20.690414 13773 raft_consensus.cc:729] T 59af1da7da284d1a81c7cc3eda283997 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:20.690596 13772 raft_consensus.cc:374] T 09c76ac2e3a749b3afb478227d8ce7ee P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:20.691108 13772 raft_consensus.cc:729] T 09c76ac2e3a749b3afb478227d8ce7ee P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:20.690996 13773 consensus_queue.cc:260] T 59af1da7da284d1a81c7cc3eda283997 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
W20260925 09:58:20.692776 13078 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_f83185097e7343848b564d3d5881c9ab in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:58:20.693490 13073 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:50556 (request call id 47) took 2125 ms. Trace:
I20260925 09:58:20.693794 13073 rpcz_store.cc:276] 0925 09:58:18.568402 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:18.568564 (+ 162us) service_pool.cc:227] Handling call
0925 09:58:19.186426 (+617862us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:58:20.691855 (+1505429us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:58:20.692707 (+ 852us) catalog_manager.cc:3616] Apply alter schema
0925 09:58:20.693427 (+ 720us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":12110,"Hive Metastore.run_cpu_time_us":1796,"Hive Metastore.run_wall_time_us":705768}
I20260925 09:58:20.694252 13772 consensus_queue.cc:260] T 09c76ac2e3a749b3afb478227d8ce7ee P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:20.696993 13651 tablet_service.cc:1467] Processing CreateTablet for tablet a9d515adc84446e6b46b0332d3ea10db (DEFAULT_TABLE table=default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:20.698200 13651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9d515adc84446e6b46b0332d3ea10db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:20.708158 13772 ts_tablet_manager.cc:1428] T 09c76ac2e3a749b3afb478227d8ce7ee P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.028s user 0.004s sys 0.002s
I20260925 09:58:20.710176 13772 tablet_bootstrap.cc:492] T 59af1da7da284d1a81c7cc3eda283997 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:20.712056 13971 raft_consensus.cc:482] T 09c76ac2e3a749b3afb478227d8ce7ee P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:20.712577 13971 raft_consensus.cc:504] T 09c76ac2e3a749b3afb478227d8ce7ee P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:20.714569 13971 leader_election.cc:288] T 09c76ac2e3a749b3afb478227d8ce7ee P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:20.716984 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09c76ac2e3a749b3afb478227d8ce7ee" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
I20260925 09:58:20.717638 13505 raft_consensus.cc:2454] T 09c76ac2e3a749b3afb478227d8ce7ee P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 0.
I20260925 09:58:20.718887 13298 leader_election.cc:302] T 09c76ac2e3a749b3afb478227d8ce7ee P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
W20260925 09:58:20.721127 13297 leader_election.cc:341] T 09c76ac2e3a749b3afb478227d8ce7ee P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Not found: Tablet not found: 09c76ac2e3a749b3afb478227d8ce7ee
I20260925 09:58:20.722900 13971 raft_consensus.cc:2781] T 09c76ac2e3a749b3afb478227d8ce7ee P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:20.723251 13971 raft_consensus.cc:482] T 09c76ac2e3a749b3afb478227d8ce7ee P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:20.723415 13777 ts_tablet_manager.cc:1918] T dea7e34e5932444daa0ba1a11af3d2ea P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:20.723621 13971 raft_consensus.cc:3037] T 09c76ac2e3a749b3afb478227d8ce7ee P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:20.686884 13770 raft_consensus.cc:348] T 3735920fa0a64b588480af485d0fd6f0 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:20.730366 13770 raft_consensus.cc:374] T 3735920fa0a64b588480af485d0fd6f0 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:20.730667 13770 raft_consensus.cc:729] T 3735920fa0a64b588480af485d0fd6f0 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:20.731859 13780 ts_tablet_manager.cc:1918] T dea7e34e5932444daa0ba1a11af3d2ea P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:20.732237 13780 log.cc:1196] T dea7e34e5932444daa0ba1a11af3d2ea P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/dea7e34e5932444daa0ba1a11af3d2ea
I20260925 09:58:20.732926 13780 ts_tablet_manager.cc:1939] T dea7e34e5932444daa0ba1a11af3d2ea P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:20.733145 13781 tablet_replica.cc:330] T 09c76ac2e3a749b3afb478227d8ce7ee P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:20.733525 13777 log.cc:1196] T dea7e34e5932444daa0ba1a11af3d2ea P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/dea7e34e5932444daa0ba1a11af3d2ea
I20260925 09:58:20.733865 13781 raft_consensus.cc:2229] T 09c76ac2e3a749b3afb478227d8ce7ee P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:20.734735 13777 ts_tablet_manager.cc:1939] T dea7e34e5932444daa0ba1a11af3d2ea P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:20.739006 13770 consensus_queue.cc:260] T 3735920fa0a64b588480af485d0fd6f0 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:20.740731 13772 tablet_bootstrap.cc:654] T 59af1da7da284d1a81c7cc3eda283997 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:20.738307 13971 raft_consensus.cc:504] T 09c76ac2e3a749b3afb478227d8ce7ee P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:20.742493 13971 leader_election.cc:288] T 09c76ac2e3a749b3afb478227d8ce7ee P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Requested vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:20.743338 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09c76ac2e3a749b3afb478227d8ce7ee" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9"
W20260925 09:58:20.744388 13297 leader_election.cc:341] T 09c76ac2e3a749b3afb478227d8ce7ee P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Not found: Tablet not found: 09c76ac2e3a749b3afb478227d8ce7ee
W20260925 09:58:20.746623 13298 leader_election.cc:341] T 09c76ac2e3a749b3afb478227d8ce7ee P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:20.750164 13298 leader_election.cc:302] T 09c76ac2e3a749b3afb478227d8ce7ee P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:58:20.751588 13969 raft_consensus.cc:2726] T 09c76ac2e3a749b3afb478227d8ce7ee P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260925 09:58:20.752363 13770 ts_tablet_manager.cc:1428] T 3735920fa0a64b588480af485d0fd6f0 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.075s user 0.006s sys 0.000s
I20260925 09:58:20.745055 13777 tablet_replica.cc:330] T 69ca132c80084e3eabc66a7ba5296e42 P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:20.754051 13772 tablet_bootstrap.cc:492] T 59af1da7da284d1a81c7cc3eda283997 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:20.754460 13772 ts_tablet_manager.cc:1397] T 59af1da7da284d1a81c7cc3eda283997 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.044s user 0.009s sys 0.000s
I20260925 09:58:20.756690 13777 raft_consensus.cc:2229] T 69ca132c80084e3eabc66a7ba5296e42 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:20.757412 13777 raft_consensus.cc:2258] T 69ca132c80084e3eabc66a7ba5296e42 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:20.762568 13777 ts_tablet_manager.cc:1905] T 69ca132c80084e3eabc66a7ba5296e42 P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:20.766325 13770 tablet_bootstrap.cc:492] T 1db61f5876974206be17c93462236bb9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:20.773146 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet dea7e34e5932444daa0ba1a11af3d2ea (table default.table_a29aa5b33fd644a2ab80242e289ed25d [id=8fa5a3419e394be2a52d1b13713a04d9]) successfully deleted
I20260925 09:58:20.775343 13773 ts_tablet_manager.cc:1428] T 59af1da7da284d1a81c7cc3eda283997 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.088s user 0.006s sys 0.000s
I20260925 09:58:20.775870 13781 raft_consensus.cc:2258] T 09c76ac2e3a749b3afb478227d8ce7ee P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:20.777053 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet dea7e34e5932444daa0ba1a11af3d2ea (table default.table_a29aa5b33fd644a2ab80242e289ed25d [id=8fa5a3419e394be2a52d1b13713a04d9]) successfully deleted
I20260925 09:58:20.781575 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet 69ca132c80084e3eabc66a7ba5296e42 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]) successfully deleted
I20260925 09:58:20.783942 13484 tablet_service.cc:1467] Processing CreateTablet for tablet a9d515adc84446e6b46b0332d3ea10db (DEFAULT_TABLE table=default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:20.788774 13484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9d515adc84446e6b46b0332d3ea10db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:20.792021 13773 tablet_bootstrap.cc:492] T 3735920fa0a64b588480af485d0fd6f0 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:20.793655 13772 raft_consensus.cc:348] T 59af1da7da284d1a81c7cc3eda283997 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:20.794394 13772 raft_consensus.cc:374] T 59af1da7da284d1a81c7cc3eda283997 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:20.794662 13772 raft_consensus.cc:729] T 59af1da7da284d1a81c7cc3eda283997 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:20.795300 13772 consensus_queue.cc:260] T 59af1da7da284d1a81c7cc3eda283997 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:20.799646 13976 raft_consensus.cc:482] T 59af1da7da284d1a81c7cc3eda283997 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:20.800108 13976 raft_consensus.cc:504] T 59af1da7da284d1a81c7cc3eda283997 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:20.800882 13773 tablet_bootstrap.cc:654] T 3735920fa0a64b588480af485d0fd6f0 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:20.802274 13976 leader_election.cc:288] T 59af1da7da284d1a81c7cc3eda283997 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
I20260925 09:58:20.805038 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59af1da7da284d1a81c7cc3eda283997" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
I20260925 09:58:20.805692 13505 raft_consensus.cc:2454] T 59af1da7da284d1a81c7cc3eda283997 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 0.
I20260925 09:58:20.805961 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59af1da7da284d1a81c7cc3eda283997" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:20.806619 13363 raft_consensus.cc:2454] T 59af1da7da284d1a81c7cc3eda283997 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 0.
I20260925 09:58:20.807093 13073 master_service.cc:946] ReplaceTablet: received request to replace tablet 58670396dbce49baa196550f676a67f3 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:20.807891 13603 leader_election.cc:302] T 59af1da7da284d1a81c7cc3eda283997 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, b3082cf15dc74c0480e5d33ecf592843; no voters:
I20260925 09:58:20.811872 13976 raft_consensus.cc:2781] T 59af1da7da284d1a81c7cc3eda283997 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:20.814327 13976 raft_consensus.cc:482] T 59af1da7da284d1a81c7cc3eda283997 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:20.814756 13976 raft_consensus.cc:3037] T 59af1da7da284d1a81c7cc3eda283997 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:20.825448 13341 tablet_service.cc:1514] Processing DeleteTablet for tablet 09c76ac2e3a749b3afb478227d8ce7ee with delete_type TABLET_DATA_DELETED (replaced by tablet a9d515adc84446e6b46b0332d3ea10db) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:20.827266 13770 tablet_bootstrap.cc:654] T 1db61f5876974206be17c93462236bb9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:20.827447 13081 catalog_manager.cc:2148] created HMS catalog entry for table [id=5eb4ef6bf2c344bdb26e429f484c59a4]
I20260925 09:58:20.827520 13976 raft_consensus.cc:504] T 59af1da7da284d1a81c7cc3eda283997 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:20.831552 13781 ts_tablet_manager.cc:1905] T 09c76ac2e3a749b3afb478227d8ce7ee P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:20.831682 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59af1da7da284d1a81c7cc3eda283997" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843"
I20260925 09:58:20.832304 13363 raft_consensus.cc:3037] T 59af1da7da284d1a81c7cc3eda283997 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:20.840905 13363 raft_consensus.cc:2454] T 59af1da7da284d1a81c7cc3eda283997 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 1.
I20260925 09:58:20.845965 13603 leader_election.cc:302] T 59af1da7da284d1a81c7cc3eda283997 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, b3082cf15dc74c0480e5d33ecf592843; no voters:
I20260925 09:58:20.847556 13770 tablet_bootstrap.cc:492] T 1db61f5876974206be17c93462236bb9 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:20.847971 13770 ts_tablet_manager.cc:1397] T 1db61f5876974206be17c93462236bb9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.082s user 0.012s sys 0.000s
I20260925 09:58:20.848763 13772 ts_tablet_manager.cc:1428] T 59af1da7da284d1a81c7cc3eda283997 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.094s user 0.005s sys 0.000s
I20260925 09:58:20.850165 13770 raft_consensus.cc:348] T 1db61f5876974206be17c93462236bb9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:20.850884 13770 raft_consensus.cc:374] T 1db61f5876974206be17c93462236bb9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:20.851120 13770 raft_consensus.cc:729] T 1db61f5876974206be17c93462236bb9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:20.853266 13773 tablet_bootstrap.cc:492] T 3735920fa0a64b588480af485d0fd6f0 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:20.853691 13773 ts_tablet_manager.cc:1397] T 3735920fa0a64b588480af485d0fd6f0 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.062s user 0.007s sys 0.003s
I20260925 09:58:20.856182 13773 raft_consensus.cc:348] T 3735920fa0a64b588480af485d0fd6f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:20.863713 13772 tablet_bootstrap.cc:492] T 3735920fa0a64b588480af485d0fd6f0 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:20.864423 13773 raft_consensus.cc:374] T 3735920fa0a64b588480af485d0fd6f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:20.864768 13773 raft_consensus.cc:729] T 3735920fa0a64b588480af485d0fd6f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:20.865507 13773 consensus_queue.cc:260] T 3735920fa0a64b588480af485d0fd6f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:20.864084 14018 raft_consensus.cc:2781] T 59af1da7da284d1a81c7cc3eda283997 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:20.873121 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59af1da7da284d1a81c7cc3eda283997" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9"
I20260925 09:58:20.873631 13505 raft_consensus.cc:3037] T 59af1da7da284d1a81c7cc3eda283997 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:20.874751 13772 tablet_bootstrap.cc:654] T 3735920fa0a64b588480af485d0fd6f0 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:20.879595 13073 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet 58670396dbce49baa196550f676a67f3
I20260925 09:58:20.880890 13073 catalog_manager.cc:6565] ReplaceTablet: tablet 58670396dbce49baa196550f676a67f3 deleted and replaced by tablet 3cde6676790b4e3f89f05ce60796b36e
I20260925 09:58:20.882629 13505 raft_consensus.cc:2454] T 59af1da7da284d1a81c7cc3eda283997 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 1.
I20260925 09:58:20.864322 13770 consensus_queue.cc:260] T 1db61f5876974206be17c93462236bb9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:20.885546 13651 tablet_service.cc:1514] Processing DeleteTablet for tablet 58670396dbce49baa196550f676a67f3 with delete_type TABLET_DATA_DELETED (replaced by tablet 3cde6676790b4e3f89f05ce60796b36e) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:20.886662 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet 58670396dbce49baa196550f676a67f3 with delete_type TABLET_DATA_DELETED (replaced by tablet 3cde6676790b4e3f89f05ce60796b36e) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:20.887571 13773 ts_tablet_manager.cc:1428] T 3735920fa0a64b588480af485d0fd6f0 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.034s user 0.007s sys 0.001s
I20260925 09:58:20.887856 13649 tablet_service.cc:1467] Processing CreateTablet for tablet 3da072989f69498e99e5e3b32bdb6859 (DEFAULT_TABLE table=default.table_a29aa5b33fd644a2ab80242e289ed25d [id=8fa5a3419e394be2a52d1b13713a04d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:20.888922 13649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3da072989f69498e99e5e3b32bdb6859. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:20.892534 13780 tablet_replica.cc:330] T 58670396dbce49baa196550f676a67f3 P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:20.893232 13780 raft_consensus.cc:2229] T 58670396dbce49baa196550f676a67f3 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:20.893663 13780 raft_consensus.cc:2258] T 58670396dbce49baa196550f676a67f3 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:20.895933 13485 tablet_service.cc:1514] Processing DeleteTablet for tablet 58670396dbce49baa196550f676a67f3 with delete_type TABLET_DATA_DELETED (replaced by tablet 3cde6676790b4e3f89f05ce60796b36e) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:20.897250 13773 tablet_bootstrap.cc:492] T 1db61f5876974206be17c93462236bb9 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:20.898072 13990 raft_consensus.cc:482] T 3735920fa0a64b588480af485d0fd6f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:20.898455 13990 raft_consensus.cc:504] T 3735920fa0a64b588480af485d0fd6f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:20.900141 13780 ts_tablet_manager.cc:1905] T 58670396dbce49baa196550f676a67f3 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:20.900203 13990 leader_election.cc:288] T 3735920fa0a64b588480af485d0fd6f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:20.904954 13770 ts_tablet_manager.cc:1428] T 1db61f5876974206be17c93462236bb9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.057s user 0.007s sys 0.000s
I20260925 09:58:20.907058 13341 tablet_service.cc:1467] Processing CreateTablet for tablet 3da072989f69498e99e5e3b32bdb6859 (DEFAULT_TABLE table=default.table_a29aa5b33fd644a2ab80242e289ed25d [id=8fa5a3419e394be2a52d1b13713a04d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:20.907745 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3735920fa0a64b588480af485d0fd6f0" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:20.908511 13341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3da072989f69498e99e5e3b32bdb6859. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:58:20.909104 13438 leader_election.cc:341] T 3735920fa0a64b588480af485d0fd6f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:20.910331 14018 raft_consensus.cc:686] T 59af1da7da284d1a81c7cc3eda283997 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 LEADER]: Becoming Leader. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Running, Role: LEADER
I20260925 09:58:20.911273 13770 tablet_bootstrap.cc:492] T a9d515adc84446e6b46b0332d3ea10db P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:20.911293 14018 consensus_queue.cc:237] T 59af1da7da284d1a81c7cc3eda283997 P a06c3ef7a6a94f7cbcc03e7cd232f237 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:20.921094 13077 catalog_manager.cc:5587] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731) does not have the latest schema for tablet 3735920fa0a64b588480af485d0fd6f0 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]). Expected version 1 got 0
I20260925 09:58:20.921713 13773 tablet_bootstrap.cc:654] T 1db61f5876974206be17c93462236bb9 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
W20260925 09:58:20.922712 13077 catalog_manager.cc:5687] Failed to send AlterTable RPC for tablet 3735920fa0a64b588480af485d0fd6f0 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 3735920fa0a64b588480af485d0fd6f0 consensus state has no leader
I20260925 09:58:20.926193 13080 master_service.cc:946] ReplaceTablet: received request to replace tablet 59af1da7da284d1a81c7cc3eda283997 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:20.928503 13077 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:50556:
table { table_name: "default.Table_eadc20bcf90e4a72b8f26e3a4798e259" } new_table_name: "default.Table_d4409ab5331140e9a4bb995d57be4ec4" modify_external_catalogs: true
I20260925 09:58:20.932091 13484 tablet_service.cc:1467] Processing CreateTablet for tablet 3da072989f69498e99e5e3b32bdb6859 (DEFAULT_TABLE table=default.table_a29aa5b33fd644a2ab80242e289ed25d [id=8fa5a3419e394be2a52d1b13713a04d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:20.933332 13484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3da072989f69498e99e5e3b32bdb6859. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:20.943255 13081 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:50556 (request call id 50) took 2057 ms. Trace:
I20260925 09:58:20.943643 13081 rpcz_store.cc:276] 0925 09:58:18.886143 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:18.886269 (+ 126us) service_pool.cc:227] Handling call
0925 09:58:20.469203 (+1582934us) catalog_manager.cc:2075] Acquired catalog manager lock
0925 09:58:20.471105 (+ 1902us) catalog_manager.cc:2121] Created new table and tablet info
0925 09:58:20.827203 (+356098us) catalog_manager.cc:2147] Created new table in HMS catalog
0925 09:58:20.941763 (+114560us) catalog_manager.cc:2181] Wrote table and tablets to system table
0925 09:58:20.942234 (+ 471us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0925 09:58:20.943164 (+ 930us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:58:20.888917 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:58:20.889277 (+ 360us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:58:20.889298 (+ 21us) write_op.cc:435] Acquired schema lock
0925 09:58:20.889310 (+ 12us) tablet.cc:660] Decoding operations
0925 09:58:20.889683 (+ 373us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:58:20.889721 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0925 09:58:20.889742 (+ 21us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:58:20.889958 (+ 216us) tablet.cc:699] Row locks acquired
0925 09:58:20.889972 (+ 14us) write_op.cc:260] PREPARE: finished
0925 09:58:20.890069 (+ 97us) write_op.cc:270] Start()
0925 09:58:20.890143 (+ 74us) write_op.cc:276] Timestamp: P: 1790330300889998 usec, L: 0
0925 09:58:20.890158 (+ 15us) op_driver.cc:331] REPLICATION: starting
0925 09:58:20.890608 (+ 450us) log.cc:842] Serialized 432 byte log entry
0925 09:58:20.938597 (+ 47989us) op_driver.cc:444] REPLICATION: finished
0925 09:58:20.940331 (+ 1734us) write_op.cc:301] APPLY: starting
0925 09:58:20.940383 (+ 52us) tablet.cc:1365] starting BulkCheckPresence
0925 09:58:20.940504 (+ 121us) tablet.cc:1368] finished BulkCheckPresence
0925 09:58:20.940602 (+ 98us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:58:20.940984 (+ 382us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:58:20.941115 (+ 131us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 09:58:20.941157 (+ 42us) write_op.cc:312] APPLY: finished
0925 09:58:20.941316 (+ 159us) log.cc:842] Serialized 28 byte log entry
0925 09:58:20.941458 (+ 142us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:20.941529 (+ 71us) write_op.cc:454] Released schema lock
0925 09:58:20.941560 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0925 09:58:20.941724 (+ 164us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:20.941749 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":16465,"Hive Metastore.run_cpu_time_us":1162,"Hive Metastore.run_wall_time_us":329122,"child_traces":[["op",{"apply.queue_time_us":1577,"apply.run_cpu_time_us":1564,"apply.run_wall_time_us":1564,"num_ops":2,"prepare.queue_time_us":59226,"prepare.run_cpu_time_us":1936,"prepare.run_wall_time_us":56738,"raft.queue_time_us":102,"raft.run_cpu_time_us":630,"raft.run_wall_time_us":631,"replication_time_us":48376}]]}
I20260925 09:58:20.887878 13976 leader_election.cc:288] T 59af1da7da284d1a81c7cc3eda283997 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 election: Requested vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
I20260925 09:58:20.956732 13770 tablet_bootstrap.cc:654] T a9d515adc84446e6b46b0332d3ea10db P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:20.967555 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet 59af1da7da284d1a81c7cc3eda283997 with delete_type TABLET_DATA_DELETED (replaced by tablet 27b03ccddd364f2f96b9a303ae6fda91) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:20.968194 13080 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet 59af1da7da284d1a81c7cc3eda283997
I20260925 09:58:20.969854 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet 59af1da7da284d1a81c7cc3eda283997 with delete_type TABLET_DATA_DELETED (replaced by tablet 27b03ccddd364f2f96b9a303ae6fda91) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:20.970556 13080 catalog_manager.cc:6565] ReplaceTablet: tablet 59af1da7da284d1a81c7cc3eda283997 deleted and replaced by tablet 27b03ccddd364f2f96b9a303ae6fda91
I20260925 09:58:20.971050 13485 tablet_service.cc:1514] Processing DeleteTablet for tablet 59af1da7da284d1a81c7cc3eda283997 with delete_type TABLET_DATA_DELETED (replaced by tablet 27b03ccddd364f2f96b9a303ae6fda91) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:20.973192 13773 tablet_bootstrap.cc:492] T 1db61f5876974206be17c93462236bb9 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:20.973630 13773 ts_tablet_manager.cc:1397] T 1db61f5876974206be17c93462236bb9 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.077s user 0.011s sys 0.000s
I20260925 09:58:20.975835 13773 raft_consensus.cc:348] T 1db61f5876974206be17c93462236bb9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:20.977560 13773 raft_consensus.cc:374] T 1db61f5876974206be17c93462236bb9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:20.977952 13773 raft_consensus.cc:729] T 1db61f5876974206be17c93462236bb9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:20.978544 13773 consensus_queue.cc:260] T 1db61f5876974206be17c93462236bb9 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:20.982379 13777 ts_tablet_manager.cc:1918] T 69ca132c80084e3eabc66a7ba5296e42 P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:20.995738 13773 ts_tablet_manager.cc:1428] T 1db61f5876974206be17c93462236bb9 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.022s user 0.005s sys 0.000s
I20260925 09:58:21.008563 13773 tablet_bootstrap.cc:492] T a9d515adc84446e6b46b0332d3ea10db P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
W20260925 09:58:21.011704 13066 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 3735920fa0a64b588480af485d0fd6f0 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 3735920fa0a64b588480af485d0fd6f0 consensus state has no leader
I20260925 09:58:21.001706 13772 tablet_bootstrap.cc:492] T 3735920fa0a64b588480af485d0fd6f0 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:21.017426 13777 log.cc:1196] T 69ca132c80084e3eabc66a7ba5296e42 P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/69ca132c80084e3eabc66a7ba5296e42
I20260925 09:58:21.017565 13772 ts_tablet_manager.cc:1397] T 3735920fa0a64b588480af485d0fd6f0 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.154s user 0.011s sys 0.001s
I20260925 09:58:21.018119 13777 ts_tablet_manager.cc:1939] T 69ca132c80084e3eabc66a7ba5296e42 P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:21.019701 13772 raft_consensus.cc:348] T 3735920fa0a64b588480af485d0fd6f0 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.020292 13772 raft_consensus.cc:374] T 3735920fa0a64b588480af485d0fd6f0 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:21.020512 13772 raft_consensus.cc:729] T 3735920fa0a64b588480af485d0fd6f0 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:21.021505 13781 ts_tablet_manager.cc:1918] T 09c76ac2e3a749b3afb478227d8ce7ee P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:21.022085 13772 consensus_queue.cc:260] T 3735920fa0a64b588480af485d0fd6f0 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.025343 13772 ts_tablet_manager.cc:1428] T 3735920fa0a64b588480af485d0fd6f0 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.008s user 0.004s sys 0.000s
I20260925 09:58:21.027683 13781 log.cc:1196] T 09c76ac2e3a749b3afb478227d8ce7ee P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/09c76ac2e3a749b3afb478227d8ce7ee
I20260925 09:58:21.027644 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet 69ca132c80084e3eabc66a7ba5296e42 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]) successfully deleted
I20260925 09:58:21.028509 13781 ts_tablet_manager.cc:1939] T 09c76ac2e3a749b3afb478227d8ce7ee P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:21.030369 13772 tablet_bootstrap.cc:492] T 1db61f5876974206be17c93462236bb9 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:21.030982 13777 tablet_replica.cc:330] T 09c76ac2e3a749b3afb478227d8ce7ee P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:21.033954 13777 raft_consensus.cc:2229] T 09c76ac2e3a749b3afb478227d8ce7ee P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:21.030647 13773 tablet_bootstrap.cc:654] T a9d515adc84446e6b46b0332d3ea10db P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:21.049546 13777 raft_consensus.cc:2258] T 09c76ac2e3a749b3afb478227d8ce7ee P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:21.051018 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet 09c76ac2e3a749b3afb478227d8ce7ee (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]) successfully deleted
I20260925 09:58:21.053547 13081 catalog_manager.cc:5587] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327) does not have the latest schema for tablet 3735920fa0a64b588480af485d0fd6f0 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]). Expected version 1 got 0
I20260925 09:58:21.056046 13781 tablet_replica.cc:330] T dea7e34e5932444daa0ba1a11af3d2ea P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:21.056255 13073 catalog_manager.cc:5587] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161) does not have the latest schema for tablet 3735920fa0a64b588480af485d0fd6f0 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]). Expected version 1 got 0
I20260925 09:58:21.060290 13341 tablet_service.cc:1467] Processing CreateTablet for tablet 8313c1ae006341bc84438085dd38f8ad (DEFAULT_TABLE table=default.table_1238f9df428247dc9b9ca7c59823bfe4 [id=5eb4ef6bf2c344bdb26e429f484c59a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:21.061509 13781 raft_consensus.cc:2229] T dea7e34e5932444daa0ba1a11af3d2ea P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:21.062337 13781 raft_consensus.cc:2258] T dea7e34e5932444daa0ba1a11af3d2ea P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:21.062901 13772 tablet_bootstrap.cc:654] T 1db61f5876974206be17c93462236bb9 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:21.065156 13781 ts_tablet_manager.cc:1905] T dea7e34e5932444daa0ba1a11af3d2ea P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:21.066035 13081 catalog_manager.cc:5587] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161) does not have the latest schema for tablet 3735920fa0a64b588480af485d0fd6f0 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]). Expected version 1 got 0
I20260925 09:58:21.068638 13342 tablet_service.cc:1467] Processing CreateTablet for tablet 3cde6676790b4e3f89f05ce60796b36e (DEFAULT_TABLE table=default.table_eadc20bcf90e4a72b8f26e3a4798e259 [id=f196d9d3750b4857905c1332d3ac763e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:21.072047 13651 tablet_service.cc:1514] Processing DeleteTablet for tablet 59af1da7da284d1a81c7cc3eda283997 with delete_type TABLET_DATA_DELETED (replaced by tablet 27b03ccddd364f2f96b9a303ae6fda91) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:21.073961 13670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3735920fa0a64b588480af485d0fd6f0" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:21.075330 13670 raft_consensus.cc:2454] T 3735920fa0a64b588480af485d0fd6f0 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 0.
I20260925 09:58:21.077531 13440 leader_election.cc:302] T 3735920fa0a64b588480af485d0fd6f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9; no voters: b3082cf15dc74c0480e5d33ecf592843
I20260925 09:58:21.078534 13990 raft_consensus.cc:2781] T 3735920fa0a64b588480af485d0fd6f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:21.078827 13990 raft_consensus.cc:482] T 3735920fa0a64b588480af485d0fd6f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:21.079113 13990 raft_consensus.cc:3037] T 3735920fa0a64b588480af485d0fd6f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:21.065301 13484 tablet_service.cc:1467] Processing CreateTablet for tablet 8313c1ae006341bc84438085dd38f8ad (DEFAULT_TABLE table=default.table_1238f9df428247dc9b9ca7c59823bfe4 [id=5eb4ef6bf2c344bdb26e429f484c59a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:21.080057 13484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8313c1ae006341bc84438085dd38f8ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:21.084322 13342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3cde6676790b4e3f89f05ce60796b36e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:21.085989 13990 raft_consensus.cc:504] T 3735920fa0a64b588480af485d0fd6f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.088045 13777 ts_tablet_manager.cc:1905] T 09c76ac2e3a749b3afb478227d8ce7ee P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:21.089272 13485 tablet_service.cc:1467] Processing CreateTablet for tablet 3cde6676790b4e3f89f05ce60796b36e (DEFAULT_TABLE table=default.table_eadc20bcf90e4a72b8f26e3a4798e259 [id=f196d9d3750b4857905c1332d3ac763e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:21.090258 13485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3cde6676790b4e3f89f05ce60796b36e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:21.092227 13990 leader_election.cc:288] T 3735920fa0a64b588480af485d0fd6f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Requested vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:21.092576 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3735920fa0a64b588480af485d0fd6f0" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843"
I20260925 09:58:21.093083 13363 raft_consensus.cc:3037] T 3735920fa0a64b588480af485d0fd6f0 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:21.098649 13976 raft_consensus.cc:482] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:21.099339 13976 raft_consensus.cc:504] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.102437 13976 leader_election.cc:288] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
W20260925 09:58:21.105751 13603 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
W20260925 09:58:21.106676 13606 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
I20260925 09:58:21.106617 13670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3735920fa0a64b588480af485d0fd6f0" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
I20260925 09:58:21.104562 13770 tablet_bootstrap.cc:492] T a9d515adc84446e6b46b0332d3ea10db P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:21.108062 13670 raft_consensus.cc:3037] T 3735920fa0a64b588480af485d0fd6f0 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:21.108657 13770 ts_tablet_manager.cc:1397] T a9d515adc84446e6b46b0332d3ea10db P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.198s user 0.008s sys 0.002s
I20260925 09:58:21.070683 13650 tablet_service.cc:1467] Processing CreateTablet for tablet 8313c1ae006341bc84438085dd38f8ad (DEFAULT_TABLE table=default.table_1238f9df428247dc9b9ca7c59823bfe4 [id=5eb4ef6bf2c344bdb26e429f484c59a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:21.105618 13651 tablet_service.cc:1467] Processing CreateTablet for tablet 3cde6676790b4e3f89f05ce60796b36e (DEFAULT_TABLE table=default.table_eadc20bcf90e4a72b8f26e3a4798e259 [id=f196d9d3750b4857905c1332d3ac763e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:21.109952 13363 raft_consensus.cc:2454] T 3735920fa0a64b588480af485d0fd6f0 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:21.110999 13438 leader_election.cc:302] T 3735920fa0a64b588480af485d0fd6f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:21.114063 13990 raft_consensus.cc:2781] T 3735920fa0a64b588480af485d0fd6f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:21.116405 13650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8313c1ae006341bc84438085dd38f8ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:21.120352 13780 ts_tablet_manager.cc:1918] T 58670396dbce49baa196550f676a67f3 P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:21.120801 13780 log.cc:1196] T 58670396dbce49baa196550f676a67f3 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/58670396dbce49baa196550f676a67f3
I20260925 09:58:21.121615 13780 ts_tablet_manager.cc:1939] T 58670396dbce49baa196550f676a67f3 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:21.122191 13081 master_service.cc:946] ReplaceTablet: received request to replace tablet 3735920fa0a64b588480af485d0fd6f0 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:21.107215 13606 leader_election.cc:302] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237; no voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:58:21.125550 13670 raft_consensus.cc:2454] T 3735920fa0a64b588480af485d0fd6f0 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:21.133180 13341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8313c1ae006341bc84438085dd38f8ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:21.134521 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet 58670396dbce49baa196550f676a67f3 (table default.table_eadc20bcf90e4a72b8f26e3a4798e259 [id=f196d9d3750b4857905c1332d3ac763e]) successfully deleted
I20260925 09:58:21.143066 13990 raft_consensus.cc:686] T 3735920fa0a64b588480af485d0fd6f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: Becoming Leader. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Running, Role: LEADER
I20260925 09:58:21.143950 13990 consensus_queue.cc:237] T 3735920fa0a64b588480af485d0fd6f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.152678 13772 tablet_bootstrap.cc:492] T 1db61f5876974206be17c93462236bb9 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:21.153081 13772 ts_tablet_manager.cc:1397] T 1db61f5876974206be17c93462236bb9 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.123s user 0.004s sys 0.006s
I20260925 09:58:21.154924 13080 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50556:
name: "default.Table_41304460d5a345c88866a7429d816aca"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:58:21.126552 13976 raft_consensus.cc:2726] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:58:21.128737 13770 raft_consensus.cc:348] T a9d515adc84446e6b46b0332d3ea10db P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.131115 13651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3cde6676790b4e3f89f05ce60796b36e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:21.132562 13780 tablet_replica.cc:330] T 59af1da7da284d1a81c7cc3eda283997 P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:21.163144 13770 raft_consensus.cc:374] T a9d515adc84446e6b46b0332d3ea10db P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:21.163477 13770 raft_consensus.cc:729] T a9d515adc84446e6b46b0332d3ea10db P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:21.163674 13780 raft_consensus.cc:2229] T 59af1da7da284d1a81c7cc3eda283997 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:58:21.164522 13780 pending_rounds.cc:62] T 59af1da7da284d1a81c7cc3eda283997 P a06c3ef7a6a94f7cbcc03e7cd232f237: Trying to abort 1 pending ops.
I20260925 09:58:21.164059 13770 consensus_queue.cc:260] T a9d515adc84446e6b46b0332d3ea10db P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.164762 13780 pending_rounds.cc:68] T 59af1da7da284d1a81c7cc3eda283997 P a06c3ef7a6a94f7cbcc03e7cd232f237: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192912558116864 op_type: NO_OP noop_request { }
I20260925 09:58:21.165283 13780 raft_consensus.cc:2866] T 59af1da7da284d1a81c7cc3eda283997 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:58:21.165618 13780 raft_consensus.cc:2258] T 59af1da7da284d1a81c7cc3eda283997 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:21.166215 13770 ts_tablet_manager.cc:1428] T a9d515adc84446e6b46b0332d3ea10db P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.057s user 0.001s sys 0.005s
I20260925 09:58:21.167173 13770 tablet_bootstrap.cc:492] T 3da072989f69498e99e5e3b32bdb6859 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:21.171267 13780 ts_tablet_manager.cc:1905] T 59af1da7da284d1a81c7cc3eda283997 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:21.179597 13073 catalog_manager.cc:5587] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731) does not have the latest schema for tablet a9d515adc84446e6b46b0332d3ea10db (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]). Expected version 1 got 0
I20260925 09:58:21.184614 13772 raft_consensus.cc:348] T 1db61f5876974206be17c93462236bb9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.185288 13772 raft_consensus.cc:374] T 1db61f5876974206be17c93462236bb9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:21.185546 13772 raft_consensus.cc:729] T 1db61f5876974206be17c93462236bb9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:21.186594 13081 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet 3735920fa0a64b588480af485d0fd6f0
I20260925 09:58:21.189572 13772 consensus_queue.cc:260] T 1db61f5876974206be17c93462236bb9 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.187588 13073 master_service.cc:946] ReplaceTablet: received request to replace tablet 3da072989f69498e99e5e3b32bdb6859 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:21.193792 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet 3735920fa0a64b588480af485d0fd6f0 with delete_type TABLET_DATA_DELETED (replaced by tablet e3027f1b971f483d91c9046163f62abe) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:21.195467 14044 consensus_queue.cc:1035] T 58670396dbce49baa196550f676a67f3 P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:21.202273 13483 tablet_service.cc:1514] Processing DeleteTablet for tablet 3735920fa0a64b588480af485d0fd6f0 with delete_type TABLET_DATA_DELETED (replaced by tablet e3027f1b971f483d91c9046163f62abe) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:21.203511 13081 catalog_manager.cc:6565] ReplaceTablet: tablet 3735920fa0a64b588480af485d0fd6f0 deleted and replaced by tablet e3027f1b971f483d91c9046163f62abe
I20260925 09:58:21.203621 13649 tablet_service.cc:1514] Processing DeleteTablet for tablet 3735920fa0a64b588480af485d0fd6f0 with delete_type TABLET_DATA_DELETED (replaced by tablet e3027f1b971f483d91c9046163f62abe) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:21.204276 13772 ts_tablet_manager.cc:1428] T 1db61f5876974206be17c93462236bb9 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.051s user 0.004s sys 0.002s
I20260925 09:58:21.206447 13773 tablet_bootstrap.cc:492] T a9d515adc84446e6b46b0332d3ea10db P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:21.206940 13773 ts_tablet_manager.cc:1397] T a9d515adc84446e6b46b0332d3ea10db P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.205s user 0.008s sys 0.011s
I20260925 09:58:21.209125 13773 raft_consensus.cc:348] T a9d515adc84446e6b46b0332d3ea10db P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.209740 13773 raft_consensus.cc:374] T a9d515adc84446e6b46b0332d3ea10db P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:21.210000 13773 raft_consensus.cc:729] T a9d515adc84446e6b46b0332d3ea10db P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
W20260925 09:58:21.213091 13440 consensus_peers.cc:487] T 58670396dbce49baa196550f676a67f3 P c4d724d4b83e4d1e817a3a505dc24eb9 -> Peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Couldn't send request to peer a06c3ef7a6a94f7cbcc03e7cd232f237. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 58670396dbce49baa196550f676a67f3. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:58:21.224897 13770 tablet_bootstrap.cc:654] T 3da072989f69498e99e5e3b32bdb6859 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:21.229477 13773 consensus_queue.cc:260] T a9d515adc84446e6b46b0332d3ea10db P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.230531 13772 tablet_bootstrap.cc:492] T a9d515adc84446e6b46b0332d3ea10db P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:21.238504 13772 tablet_bootstrap.cc:654] T a9d515adc84446e6b46b0332d3ea10db P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:21.247704 13773 ts_tablet_manager.cc:1428] T a9d515adc84446e6b46b0332d3ea10db P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.041s user 0.006s sys 0.000s
I20260925 09:58:21.254302 13773 tablet_bootstrap.cc:492] T 3da072989f69498e99e5e3b32bdb6859 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:21.255545 13781 ts_tablet_manager.cc:1918] T dea7e34e5932444daa0ba1a11af3d2ea P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:21.259184 13781 log.cc:1196] T dea7e34e5932444daa0ba1a11af3d2ea P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/dea7e34e5932444daa0ba1a11af3d2ea
I20260925 09:58:21.260222 13781 ts_tablet_manager.cc:1939] T dea7e34e5932444daa0ba1a11af3d2ea P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:21.263083 13073 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet 3da072989f69498e99e5e3b32bdb6859
I20260925 09:58:21.265828 13073 catalog_manager.cc:6565] ReplaceTablet: tablet 3da072989f69498e99e5e3b32bdb6859 deleted and replaced by tablet 963c1e91c6224e5dae4ecc9a6f48f1ac
I20260925 09:58:21.265933 13781 tablet_replica.cc:330] T 58670396dbce49baa196550f676a67f3 P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:21.265944 13770 tablet_bootstrap.cc:492] T 3da072989f69498e99e5e3b32bdb6859 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:21.266049 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet dea7e34e5932444daa0ba1a11af3d2ea (table default.table_a29aa5b33fd644a2ab80242e289ed25d [id=8fa5a3419e394be2a52d1b13713a04d9]) successfully deleted
I20260925 09:58:21.266319 13770 ts_tablet_manager.cc:1397] T 3da072989f69498e99e5e3b32bdb6859 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.099s user 0.010s sys 0.003s
I20260925 09:58:21.266723 13650 tablet_service.cc:1514] Processing DeleteTablet for tablet 3da072989f69498e99e5e3b32bdb6859 with delete_type TABLET_DATA_DELETED (replaced by tablet 963c1e91c6224e5dae4ecc9a6f48f1ac) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:21.265486 13484 tablet_service.cc:1514] Processing DeleteTablet for tablet 3da072989f69498e99e5e3b32bdb6859 with delete_type TABLET_DATA_DELETED (replaced by tablet 963c1e91c6224e5dae4ecc9a6f48f1ac) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:21.268631 13770 raft_consensus.cc:348] T 3da072989f69498e99e5e3b32bdb6859 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.269836 13770 raft_consensus.cc:374] T 3da072989f69498e99e5e3b32bdb6859 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:21.270087 13770 raft_consensus.cc:729] T 3da072989f69498e99e5e3b32bdb6859 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:21.276153 13484 tablet_service.cc:1467] Processing CreateTablet for tablet 27b03ccddd364f2f96b9a303ae6fda91 (DEFAULT_TABLE table=default.table_3823faf045944cc1a828ce929b8a7275 [id=f6ae78b7a67640f5a6e71e04249b4262]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:21.277347 13484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27b03ccddd364f2f96b9a303ae6fda91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:21.278411 13780 ts_tablet_manager.cc:1918] T 59af1da7da284d1a81c7cc3eda283997 P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:21.278746 13780 log.cc:1196] T 59af1da7da284d1a81c7cc3eda283997 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/59af1da7da284d1a81c7cc3eda283997
I20260925 09:58:21.279444 13780 ts_tablet_manager.cc:1939] T 59af1da7da284d1a81c7cc3eda283997 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:21.279963 13082 catalog_manager.cc:2685] deleted HMS catalog entry for table default.table_3823faf045944cc1a828ce929b8a7275 [id=f6ae78b7a67640f5a6e71e04249b4262]
I20260925 09:58:21.240935 13777 ts_tablet_manager.cc:1918] T 09c76ac2e3a749b3afb478227d8ce7ee P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:21.282176 13777 log.cc:1196] T 09c76ac2e3a749b3afb478227d8ce7ee P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/09c76ac2e3a749b3afb478227d8ce7ee
I20260925 09:58:21.282802 13777 ts_tablet_manager.cc:1939] T 09c76ac2e3a749b3afb478227d8ce7ee P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:21.284943 13777 tablet_replica.cc:330] T 58670396dbce49baa196550f676a67f3 P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:21.284950 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet 09c76ac2e3a749b3afb478227d8ce7ee (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]) successfully deleted
I20260925 09:58:21.285081 13781 raft_consensus.cc:2229] T 58670396dbce49baa196550f676a67f3 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:58:21.285580 13777 raft_consensus.cc:2229] T 58670396dbce49baa196550f676a67f3 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:21.286005 13777 raft_consensus.cc:2258] T 58670396dbce49baa196550f676a67f3 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:21.269081 13341 tablet_service.cc:1514] Processing DeleteTablet for tablet 3da072989f69498e99e5e3b32bdb6859 with delete_type TABLET_DATA_DELETED (replaced by tablet 963c1e91c6224e5dae4ecc9a6f48f1ac) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:21.276823 13342 tablet_service.cc:1467] Processing CreateTablet for tablet 27b03ccddd364f2f96b9a303ae6fda91 (DEFAULT_TABLE table=default.table_3823faf045944cc1a828ce929b8a7275 [id=f6ae78b7a67640f5a6e71e04249b4262]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:21.287185 13342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27b03ccddd364f2f96b9a303ae6fda91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:21.287554 13781 raft_consensus.cc:2258] T 58670396dbce49baa196550f676a67f3 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:21.261725 13773 tablet_bootstrap.cc:654] T 3da072989f69498e99e5e3b32bdb6859 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:21.292624 13777 ts_tablet_manager.cc:1905] T 58670396dbce49baa196550f676a67f3 P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 09:58:21.297151 13073 catalog_manager.cc:5328] Ignoring report from unknown tablet 58670396dbce49baa196550f676a67f3
I20260925 09:58:21.297891 13781 ts_tablet_manager.cc:1905] T 58670396dbce49baa196550f676a67f3 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:21.299553 13650 tablet_service.cc:1467] Processing CreateTablet for tablet 27b03ccddd364f2f96b9a303ae6fda91 (DEFAULT_TABLE table=default.table_3823faf045944cc1a828ce929b8a7275 [id=f6ae78b7a67640f5a6e71e04249b4262]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:21.300282 13976 raft_consensus.cc:482] T 1db61f5876974206be17c93462236bb9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:21.300511 13650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27b03ccddd364f2f96b9a303ae6fda91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:21.300676 13976 raft_consensus.cc:504] T 1db61f5876974206be17c93462236bb9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.302417 13976 leader_election.cc:288] T 1db61f5876974206be17c93462236bb9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
I20260925 09:58:21.303524 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1db61f5876974206be17c93462236bb9" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
I20260925 09:58:21.304095 13505 raft_consensus.cc:2454] T 1db61f5876974206be17c93462236bb9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 0.
I20260925 09:58:21.304930 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1db61f5876974206be17c93462236bb9" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:21.305496 13363 raft_consensus.cc:2454] T 1db61f5876974206be17c93462236bb9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 0.
I20260925 09:58:21.306540 13603 leader_election.cc:302] T 1db61f5876974206be17c93462236bb9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, b3082cf15dc74c0480e5d33ecf592843; no voters:
I20260925 09:58:21.308218 13976 raft_consensus.cc:2781] T 1db61f5876974206be17c93462236bb9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:21.308508 13976 raft_consensus.cc:482] T 1db61f5876974206be17c93462236bb9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:21.308768 13976 raft_consensus.cc:3037] T 1db61f5876974206be17c93462236bb9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:21.270678 13770 consensus_queue.cc:260] T 3da072989f69498e99e5e3b32bdb6859 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
W20260925 09:58:21.311255 13081 catalog_manager.cc:5328] Ignoring report from unknown tablet 58670396dbce49baa196550f676a67f3
I20260925 09:58:21.315287 13773 tablet_bootstrap.cc:492] T 3da072989f69498e99e5e3b32bdb6859 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:21.315701 13773 ts_tablet_manager.cc:1397] T 3da072989f69498e99e5e3b32bdb6859 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.062s user 0.005s sys 0.005s
I20260925 09:58:21.317862 13770 ts_tablet_manager.cc:1428] T 3da072989f69498e99e5e3b32bdb6859 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.051s user 0.006s sys 0.000s
I20260925 09:58:21.317842 13773 raft_consensus.cc:348] T 3da072989f69498e99e5e3b32bdb6859 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.318472 13773 raft_consensus.cc:374] T 3da072989f69498e99e5e3b32bdb6859 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:21.318722 13773 raft_consensus.cc:729] T 3da072989f69498e99e5e3b32bdb6859 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:21.319180 13081 catalog_manager.cc:5587] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327) does not have the latest schema for tablet a9d515adc84446e6b46b0332d3ea10db (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]). Expected version 1 got 0
I20260925 09:58:21.322501 13976 raft_consensus.cc:504] T 1db61f5876974206be17c93462236bb9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.327734 13773 consensus_queue.cc:260] T 3da072989f69498e99e5e3b32bdb6859 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.327994 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet 59af1da7da284d1a81c7cc3eda283997 (table default.table_3823faf045944cc1a828ce929b8a7275 [id=f6ae78b7a67640f5a6e71e04249b4262]) successfully deleted
I20260925 09:58:21.331549 13770 tablet_bootstrap.cc:492] T 8313c1ae006341bc84438085dd38f8ad P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:21.332307 13773 ts_tablet_manager.cc:1428] T 3da072989f69498e99e5e3b32bdb6859 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.016s user 0.004s sys 0.001s
I20260925 09:58:21.334074 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1db61f5876974206be17c93462236bb9" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843"
I20260925 09:58:21.334643 13363 raft_consensus.cc:3037] T 1db61f5876974206be17c93462236bb9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:21.337618 13773 tablet_bootstrap.cc:492] T 3cde6676790b4e3f89f05ce60796b36e P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:21.328649 13780 tablet_replica.cc:330] T 3735920fa0a64b588480af485d0fd6f0 P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:21.339919 13780 raft_consensus.cc:2229] T 3735920fa0a64b588480af485d0fd6f0 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:21.340310 13780 raft_consensus.cc:2258] T 3735920fa0a64b588480af485d0fd6f0 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:21.341876 13780 ts_tablet_manager.cc:1905] T 3735920fa0a64b588480af485d0fd6f0 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:21.345196 13363 raft_consensus.cc:2454] T 1db61f5876974206be17c93462236bb9 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 1.
I20260925 09:58:21.345373 13772 tablet_bootstrap.cc:492] T a9d515adc84446e6b46b0332d3ea10db P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:21.345837 13772 ts_tablet_manager.cc:1397] T a9d515adc84446e6b46b0332d3ea10db P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.115s user 0.007s sys 0.002s
I20260925 09:58:21.346385 13603 leader_election.cc:302] T 1db61f5876974206be17c93462236bb9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, b3082cf15dc74c0480e5d33ecf592843; no voters:
I20260925 09:58:21.347260 14018 raft_consensus.cc:2781] T 1db61f5876974206be17c93462236bb9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:21.347708 14018 raft_consensus.cc:686] T 1db61f5876974206be17c93462236bb9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 LEADER]: Becoming Leader. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Running, Role: LEADER
I20260925 09:58:21.348443 13976 leader_election.cc:288] T 1db61f5876974206be17c93462236bb9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 election: Requested vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
I20260925 09:58:21.348930 13773 tablet_bootstrap.cc:654] T 3cde6676790b4e3f89f05ce60796b36e P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
W20260925 09:58:21.351611 13081 catalog_manager.cc:5328] Ignoring report from unknown tablet 58670396dbce49baa196550f676a67f3
I20260925 09:58:21.352054 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1db61f5876974206be17c93462236bb9" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9"
I20260925 09:58:21.352592 13505 raft_consensus.cc:3037] T 1db61f5876974206be17c93462236bb9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:21.348464 14018 consensus_queue.cc:237] T 1db61f5876974206be17c93462236bb9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.360052 13781 ts_tablet_manager.cc:1918] T 58670396dbce49baa196550f676a67f3 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:21.360407 13781 log.cc:1196] T 58670396dbce49baa196550f676a67f3 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/58670396dbce49baa196550f676a67f3
I20260925 09:58:21.361099 13781 ts_tablet_manager.cc:1939] T 58670396dbce49baa196550f676a67f3 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:21.362811 13505 raft_consensus.cc:2454] T 1db61f5876974206be17c93462236bb9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 1.
I20260925 09:58:21.364033 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet 58670396dbce49baa196550f676a67f3 (table default.table_eadc20bcf90e4a72b8f26e3a4798e259 [id=f196d9d3750b4857905c1332d3ac763e]) successfully deleted
I20260925 09:58:21.372097 13777 ts_tablet_manager.cc:1918] T 58670396dbce49baa196550f676a67f3 P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:21.372445 13777 log.cc:1196] T 58670396dbce49baa196550f676a67f3 P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/58670396dbce49baa196550f676a67f3
I20260925 09:58:21.373224 13777 ts_tablet_manager.cc:1939] T 58670396dbce49baa196550f676a67f3 P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:21.375516 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet 58670396dbce49baa196550f676a67f3 (table default.table_eadc20bcf90e4a72b8f26e3a4798e259 [id=f196d9d3750b4857905c1332d3ac763e]) successfully deleted
I20260925 09:58:21.376405 13777 tablet_replica.cc:330] T 59af1da7da284d1a81c7cc3eda283997 P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:21.373175 13772 raft_consensus.cc:348] T a9d515adc84446e6b46b0332d3ea10db P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.377104 13777 raft_consensus.cc:2229] T 59af1da7da284d1a81c7cc3eda283997 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:21.377365 13772 raft_consensus.cc:374] T a9d515adc84446e6b46b0332d3ea10db P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:21.377668 13777 raft_consensus.cc:2258] T 59af1da7da284d1a81c7cc3eda283997 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:21.379125 13770 tablet_bootstrap.cc:654] T 8313c1ae006341bc84438085dd38f8ad P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:21.379171 13777 ts_tablet_manager.cc:1905] T 59af1da7da284d1a81c7cc3eda283997 P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:21.379235 13781 tablet_replica.cc:330] T 59af1da7da284d1a81c7cc3eda283997 P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:21.379962 13781 raft_consensus.cc:2229] T 59af1da7da284d1a81c7cc3eda283997 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:21.380429 13781 raft_consensus.cc:2258] T 59af1da7da284d1a81c7cc3eda283997 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:21.383453 13772 raft_consensus.cc:729] T a9d515adc84446e6b46b0332d3ea10db P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:21.387373 13773 tablet_bootstrap.cc:492] T 3cde6676790b4e3f89f05ce60796b36e P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:21.387840 13773 ts_tablet_manager.cc:1397] T 3cde6676790b4e3f89f05ce60796b36e P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.050s user 0.009s sys 0.000s
I20260925 09:58:21.388134 13772 consensus_queue.cc:260] T a9d515adc84446e6b46b0332d3ea10db P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.390478 13773 raft_consensus.cc:348] T 3cde6676790b4e3f89f05ce60796b36e P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.395556 13773 raft_consensus.cc:374] T 3cde6676790b4e3f89f05ce60796b36e P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:21.395848 13773 raft_consensus.cc:729] T 3cde6676790b4e3f89f05ce60796b36e P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:21.396636 13773 consensus_queue.cc:260] T 3cde6676790b4e3f89f05ce60796b36e P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.400035 13773 ts_tablet_manager.cc:1428] T 3cde6676790b4e3f89f05ce60796b36e P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.012s user 0.004s sys 0.001s
I20260925 09:58:21.402995 13772 ts_tablet_manager.cc:1428] T a9d515adc84446e6b46b0332d3ea10db P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.057s user 0.006s sys 0.001s
I20260925 09:58:21.404989 13772 tablet_bootstrap.cc:492] T 3da072989f69498e99e5e3b32bdb6859 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:21.405910 13773 tablet_bootstrap.cc:492] T 8313c1ae006341bc84438085dd38f8ad P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:21.411312 13777 ts_tablet_manager.cc:1918] T 59af1da7da284d1a81c7cc3eda283997 P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:21.411489 13781 ts_tablet_manager.cc:1905] T 59af1da7da284d1a81c7cc3eda283997 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:21.411971 13777 log.cc:1196] T 59af1da7da284d1a81c7cc3eda283997 P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/59af1da7da284d1a81c7cc3eda283997
I20260925 09:58:21.412621 13777 ts_tablet_manager.cc:1939] T 59af1da7da284d1a81c7cc3eda283997 P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:21.413931 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet 59af1da7da284d1a81c7cc3eda283997 (table default.table_3823faf045944cc1a828ce929b8a7275 [id=f6ae78b7a67640f5a6e71e04249b4262]) successfully deleted
I20260925 09:58:21.410627 13773 tablet_bootstrap.cc:654] T 8313c1ae006341bc84438085dd38f8ad P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
W20260925 09:58:21.416500 13066 catalog_manager.cc:4808] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): delete failed for tablet 59af1da7da284d1a81c7cc3eda283997 because the tablet was not found. No further retry: Not found: Tablet not found: 59af1da7da284d1a81c7cc3eda283997
I20260925 09:58:21.418923 13777 tablet_replica.cc:330] T 3735920fa0a64b588480af485d0fd6f0 P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:21.419620 13777 raft_consensus.cc:2229] T 3735920fa0a64b588480af485d0fd6f0 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:21.420113 13777 raft_consensus.cc:2258] T 3735920fa0a64b588480af485d0fd6f0 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:21.421802 13777 ts_tablet_manager.cc:1905] T 3735920fa0a64b588480af485d0fd6f0 P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:21.429744 13772 tablet_bootstrap.cc:654] T 3da072989f69498e99e5e3b32bdb6859 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:21.432920 13770 tablet_bootstrap.cc:492] T 8313c1ae006341bc84438085dd38f8ad P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:21.433408 13770 ts_tablet_manager.cc:1397] T 8313c1ae006341bc84438085dd38f8ad P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.102s user 0.005s sys 0.003s
I20260925 09:58:21.437311 13342 tablet_service.cc:1467] Processing CreateTablet for tablet 963c1e91c6224e5dae4ecc9a6f48f1ac (DEFAULT_TABLE table=default.table_a29aa5b33fd644a2ab80242e289ed25d [id=8fa5a3419e394be2a52d1b13713a04d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:21.438333 13342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 963c1e91c6224e5dae4ecc9a6f48f1ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:21.440179 13081 catalog_manager.cc:5587] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161) does not have the latest schema for tablet a9d515adc84446e6b46b0332d3ea10db (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]). Expected version 1 got 0
I20260925 09:58:21.443243 13073 catalog_manager.cc:5495] T 1db61f5876974206be17c93462236bb9 P a06c3ef7a6a94f7cbcc03e7cd232f237 reported cstate change: term changed from 0 to 1, leader changed from <none> to a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3). New cstate: current_term: 1 leader_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: HEALTHY } } }
I20260925 09:58:21.452272 13770 raft_consensus.cc:348] T 8313c1ae006341bc84438085dd38f8ad P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.453059 13770 raft_consensus.cc:374] T 8313c1ae006341bc84438085dd38f8ad P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:21.453368 13770 raft_consensus.cc:729] T 8313c1ae006341bc84438085dd38f8ad P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:21.454229 13484 tablet_service.cc:1467] Processing CreateTablet for tablet 963c1e91c6224e5dae4ecc9a6f48f1ac (DEFAULT_TABLE table=default.table_a29aa5b33fd644a2ab80242e289ed25d [id=8fa5a3419e394be2a52d1b13713a04d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:21.454236 13770 consensus_queue.cc:260] T 8313c1ae006341bc84438085dd38f8ad P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.455456 13484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 963c1e91c6224e5dae4ecc9a6f48f1ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:21.460688 13781 ts_tablet_manager.cc:1918] T 59af1da7da284d1a81c7cc3eda283997 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:21.461035 13781 log.cc:1196] T 59af1da7da284d1a81c7cc3eda283997 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/59af1da7da284d1a81c7cc3eda283997
I20260925 09:58:21.461627 13781 ts_tablet_manager.cc:1939] T 59af1da7da284d1a81c7cc3eda283997 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:21.463152 13777 ts_tablet_manager.cc:1918] T 3735920fa0a64b588480af485d0fd6f0 P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:21.463537 13777 log.cc:1196] T 3735920fa0a64b588480af485d0fd6f0 P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/3735920fa0a64b588480af485d0fd6f0
I20260925 09:58:21.463909 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet 59af1da7da284d1a81c7cc3eda283997 (table default.table_3823faf045944cc1a828ce929b8a7275 [id=f6ae78b7a67640f5a6e71e04249b4262]) successfully deleted
I20260925 09:58:21.464138 13777 ts_tablet_manager.cc:1939] T 3735920fa0a64b588480af485d0fd6f0 P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:21.471227 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet 3735920fa0a64b588480af485d0fd6f0 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]) successfully deleted
W20260925 09:58:21.473152 13066 catalog_manager.cc:4821] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): delete failed for tablet 3da072989f69498e99e5e3b32bdb6859 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3da072989f69498e99e5e3b32bdb6859 already in progress: creating tablet
I20260925 09:58:21.475293 13781 tablet_replica.cc:330] T 3735920fa0a64b588480af485d0fd6f0 P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:21.477067 13781 raft_consensus.cc:2229] T 3735920fa0a64b588480af485d0fd6f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:58:21.477679 13781 pending_rounds.cc:62] T 3735920fa0a64b588480af485d0fd6f0 P c4d724d4b83e4d1e817a3a505dc24eb9: Trying to abort 1 pending ops.
I20260925 09:58:21.477874 13781 pending_rounds.cc:68] T 3735920fa0a64b588480af485d0fd6f0 P c4d724d4b83e4d1e817a3a505dc24eb9: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192913493147648 op_type: NO_OP noop_request { }
I20260925 09:58:21.477866 13650 tablet_service.cc:1467] Processing CreateTablet for tablet 963c1e91c6224e5dae4ecc9a6f48f1ac (DEFAULT_TABLE table=default.table_a29aa5b33fd644a2ab80242e289ed25d [id=8fa5a3419e394be2a52d1b13713a04d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:21.478266 13781 raft_consensus.cc:2866] T 3735920fa0a64b588480af485d0fd6f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:58:21.478789 13650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 963c1e91c6224e5dae4ecc9a6f48f1ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:21.489374 13773 tablet_bootstrap.cc:492] T 8313c1ae006341bc84438085dd38f8ad P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:21.489799 13773 ts_tablet_manager.cc:1397] T 8313c1ae006341bc84438085dd38f8ad P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.084s user 0.020s sys 0.011s
I20260925 09:58:21.490029 13781 raft_consensus.cc:2258] T 3735920fa0a64b588480af485d0fd6f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:21.490584 13772 tablet_bootstrap.cc:492] T 3da072989f69498e99e5e3b32bdb6859 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:21.490974 13772 ts_tablet_manager.cc:1397] T 3da072989f69498e99e5e3b32bdb6859 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.086s user 0.006s sys 0.001s
I20260925 09:58:21.491873 13770 ts_tablet_manager.cc:1428] T 8313c1ae006341bc84438085dd38f8ad P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.058s user 0.004s sys 0.004s
I20260925 09:58:21.492381 13781 ts_tablet_manager.cc:1905] T 3735920fa0a64b588480af485d0fd6f0 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:21.492755 13770 tablet_bootstrap.cc:492] T 3cde6676790b4e3f89f05ce60796b36e P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:21.492017 13773 raft_consensus.cc:348] T 8313c1ae006341bc84438085dd38f8ad P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.497471 13773 raft_consensus.cc:374] T 8313c1ae006341bc84438085dd38f8ad P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:21.497855 13773 raft_consensus.cc:729] T 8313c1ae006341bc84438085dd38f8ad P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:21.497658 13772 raft_consensus.cc:348] T 3da072989f69498e99e5e3b32bdb6859 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.498304 13772 raft_consensus.cc:374] T 3da072989f69498e99e5e3b32bdb6859 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:21.498545 13772 raft_consensus.cc:729] T 3da072989f69498e99e5e3b32bdb6859 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:21.498531 13773 consensus_queue.cc:260] T 8313c1ae006341bc84438085dd38f8ad P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.501571 13342 tablet_service.cc:1467] Processing CreateTablet for tablet e3027f1b971f483d91c9046163f62abe (DEFAULT_TABLE table=default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:21.502099 13485 tablet_service.cc:1467] Processing CreateTablet for tablet e3027f1b971f483d91c9046163f62abe (DEFAULT_TABLE table=default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:21.502473 13342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3027f1b971f483d91c9046163f62abe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:21.502980 13485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3027f1b971f483d91c9046163f62abe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:21.504835 13773 ts_tablet_manager.cc:1428] T 8313c1ae006341bc84438085dd38f8ad P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.015s user 0.006s sys 0.001s
I20260925 09:58:21.505606 13773 tablet_bootstrap.cc:492] T 27b03ccddd364f2f96b9a303ae6fda91 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:21.510258 13773 tablet_bootstrap.cc:654] T 27b03ccddd364f2f96b9a303ae6fda91 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:21.521181 13781 ts_tablet_manager.cc:1918] T 3735920fa0a64b588480af485d0fd6f0 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:21.500068 13780 ts_tablet_manager.cc:1918] T 3735920fa0a64b588480af485d0fd6f0 P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:21.523669 13780 log.cc:1196] T 3735920fa0a64b588480af485d0fd6f0 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/3735920fa0a64b588480af485d0fd6f0
I20260925 09:58:21.524308 13780 ts_tablet_manager.cc:1939] T 3735920fa0a64b588480af485d0fd6f0 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:21.526664 13780 tablet_replica.cc:330] T 3da072989f69498e99e5e3b32bdb6859 P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:21.529071 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet 3735920fa0a64b588480af485d0fd6f0 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]) successfully deleted
I20260925 09:58:21.529601 13770 tablet_bootstrap.cc:654] T 3cde6676790b4e3f89f05ce60796b36e P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:21.531730 13772 consensus_queue.cc:260] T 3da072989f69498e99e5e3b32bdb6859 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.537860 13772 ts_tablet_manager.cc:1428] T 3da072989f69498e99e5e3b32bdb6859 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.047s user 0.002s sys 0.005s
I20260925 09:58:21.539170 13772 tablet_bootstrap.cc:492] T 3cde6676790b4e3f89f05ce60796b36e P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:21.543663 13781 log.cc:1196] T 3735920fa0a64b588480af485d0fd6f0 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/3735920fa0a64b588480af485d0fd6f0
I20260925 09:58:21.544780 14047 raft_consensus.cc:482] T 8313c1ae006341bc84438085dd38f8ad P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:21.545217 14047 raft_consensus.cc:504] T 8313c1ae006341bc84438085dd38f8ad P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.545624 13971 raft_consensus.cc:482] T a9d515adc84446e6b46b0332d3ea10db P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:21.546023 13971 raft_consensus.cc:504] T a9d515adc84446e6b46b0332d3ea10db P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.547107 14047 leader_election.cc:288] T 8313c1ae006341bc84438085dd38f8ad P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:21.527257 13780 raft_consensus.cc:2229] T 3da072989f69498e99e5e3b32bdb6859 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:21.547746 13780 raft_consensus.cc:2258] T 3da072989f69498e99e5e3b32bdb6859 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:21.547811 13971 leader_election.cc:288] T a9d515adc84446e6b46b0332d3ea10db P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:21.548779 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8313c1ae006341bc84438085dd38f8ad" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:21.549415 13670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9d515adc84446e6b46b0332d3ea10db" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:21.549612 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8313c1ae006341bc84438085dd38f8ad" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:21.549990 13670 raft_consensus.cc:2454] T a9d515adc84446e6b46b0332d3ea10db P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 0.
I20260925 09:58:21.550181 13671 raft_consensus.cc:2454] T 8313c1ae006341bc84438085dd38f8ad P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 0.
W20260925 09:58:21.550873 13438 leader_election.cc:341] T 8313c1ae006341bc84438085dd38f8ad P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:21.551193 13297 leader_election.cc:302] T a9d515adc84446e6b46b0332d3ea10db P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, b3082cf15dc74c0480e5d33ecf592843; no voters:
I20260925 09:58:21.551489 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9d515adc84446e6b46b0332d3ea10db" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
I20260925 09:58:21.551935 13971 raft_consensus.cc:2781] T a9d515adc84446e6b46b0332d3ea10db P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:21.552062 13505 raft_consensus.cc:2454] T a9d515adc84446e6b46b0332d3ea10db P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 0.
I20260925 09:58:21.552227 13971 raft_consensus.cc:482] T a9d515adc84446e6b46b0332d3ea10db P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:21.552511 13971 raft_consensus.cc:3037] T a9d515adc84446e6b46b0332d3ea10db P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:21.553351 13440 leader_election.cc:302] T 8313c1ae006341bc84438085dd38f8ad P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9; no voters: b3082cf15dc74c0480e5d33ecf592843
I20260925 09:58:21.555999 14047 raft_consensus.cc:2781] T 8313c1ae006341bc84438085dd38f8ad P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:21.556301 14047 raft_consensus.cc:482] T 8313c1ae006341bc84438085dd38f8ad P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:21.556620 14047 raft_consensus.cc:3037] T 8313c1ae006341bc84438085dd38f8ad P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:21.560640 13780 ts_tablet_manager.cc:1905] T 3da072989f69498e99e5e3b32bdb6859 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:21.558884 13971 raft_consensus.cc:504] T a9d515adc84446e6b46b0332d3ea10db P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.564035 13781 ts_tablet_manager.cc:1939] T 3735920fa0a64b588480af485d0fd6f0 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:21.565802 13772 tablet_bootstrap.cc:654] T 3cde6676790b4e3f89f05ce60796b36e P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:21.566591 13971 leader_election.cc:288] T a9d515adc84446e6b46b0332d3ea10db P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Requested vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:21.567660 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9d515adc84446e6b46b0332d3ea10db" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
I20260925 09:58:21.568403 13671 raft_consensus.cc:3037] T a9d515adc84446e6b46b0332d3ea10db P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:21.569303 13770 tablet_bootstrap.cc:492] T 3cde6676790b4e3f89f05ce60796b36e P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:21.569850 13770 ts_tablet_manager.cc:1397] T 3cde6676790b4e3f89f05ce60796b36e P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.077s user 0.007s sys 0.001s
I20260925 09:58:21.572078 13770 raft_consensus.cc:348] T 3cde6676790b4e3f89f05ce60796b36e P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.572665 13770 raft_consensus.cc:374] T 3cde6676790b4e3f89f05ce60796b36e P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:21.572911 13770 raft_consensus.cc:729] T 3cde6676790b4e3f89f05ce60796b36e P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:21.573982 13770 consensus_queue.cc:260] T 3cde6676790b4e3f89f05ce60796b36e P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.575760 13770 ts_tablet_manager.cc:1428] T 3cde6676790b4e3f89f05ce60796b36e P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260925 09:58:21.566632 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9d515adc84446e6b46b0332d3ea10db" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9"
I20260925 09:58:21.576248 13505 raft_consensus.cc:3037] T a9d515adc84446e6b46b0332d3ea10db P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:21.583688 13505 raft_consensus.cc:2454] T a9d515adc84446e6b46b0332d3ea10db P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 1.
I20260925 09:58:21.584918 13298 leader_election.cc:302] T a9d515adc84446e6b46b0332d3ea10db P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:21.587261 13073 catalog_manager.cc:5587] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161) does not have the latest schema for tablet a9d515adc84446e6b46b0332d3ea10db (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]). Expected version 1 got 0
I20260925 09:58:21.587874 13781 tablet_replica.cc:330] T 3da072989f69498e99e5e3b32bdb6859 P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:21.588084 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet 3735920fa0a64b588480af485d0fd6f0 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]) successfully deleted
I20260925 09:58:21.588526 13781 raft_consensus.cc:2229] T 3da072989f69498e99e5e3b32bdb6859 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:21.588915 13781 raft_consensus.cc:2258] T 3da072989f69498e99e5e3b32bdb6859 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:21.590638 13651 tablet_service.cc:1467] Processing CreateTablet for tablet e3027f1b971f483d91c9046163f62abe (DEFAULT_TABLE table=default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:21.591167 13671 raft_consensus.cc:2454] T a9d515adc84446e6b46b0332d3ea10db P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 1.
I20260925 09:58:21.593353 13651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3027f1b971f483d91c9046163f62abe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:21.595564 13971 raft_consensus.cc:2781] T a9d515adc84446e6b46b0332d3ea10db P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:21.596629 14047 raft_consensus.cc:504] T 8313c1ae006341bc84438085dd38f8ad P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.598449 14047 leader_election.cc:288] T 8313c1ae006341bc84438085dd38f8ad P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Requested vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:21.601434 13773 tablet_bootstrap.cc:492] T 27b03ccddd364f2f96b9a303ae6fda91 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:21.601830 13773 ts_tablet_manager.cc:1397] T 27b03ccddd364f2f96b9a303ae6fda91 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.096s user 0.014s sys 0.002s
I20260925 09:58:21.601666 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8313c1ae006341bc84438085dd38f8ad" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843"
W20260925 09:58:21.602952 13438 leader_election.cc:341] T 8313c1ae006341bc84438085dd38f8ad P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:21.604781 13773 raft_consensus.cc:348] T 27b03ccddd364f2f96b9a303ae6fda91 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.605454 13773 raft_consensus.cc:374] T 27b03ccddd364f2f96b9a303ae6fda91 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:21.605705 13773 raft_consensus.cc:729] T 27b03ccddd364f2f96b9a303ae6fda91 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:21.609019 13781 ts_tablet_manager.cc:1905] T 3da072989f69498e99e5e3b32bdb6859 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:21.612048 13770 tablet_bootstrap.cc:492] T 27b03ccddd364f2f96b9a303ae6fda91 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:21.611589 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8313c1ae006341bc84438085dd38f8ad" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
I20260925 09:58:21.618171 13671 raft_consensus.cc:3037] T 8313c1ae006341bc84438085dd38f8ad P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:21.619120 13971 raft_consensus.cc:686] T a9d515adc84446e6b46b0332d3ea10db P b3082cf15dc74c0480e5d33ecf592843 [term 1 LEADER]: Becoming Leader. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Running, Role: LEADER
I20260925 09:58:21.619927 13971 consensus_queue.cc:237] T a9d515adc84446e6b46b0332d3ea10db P b3082cf15dc74c0480e5d33ecf592843 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.620473 13773 consensus_queue.cc:260] T 27b03ccddd364f2f96b9a303ae6fda91 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.620998 13772 tablet_bootstrap.cc:492] T 3cde6676790b4e3f89f05ce60796b36e P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:21.621356 13772 ts_tablet_manager.cc:1397] T 3cde6676790b4e3f89f05ce60796b36e P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.082s user 0.004s sys 0.006s
I20260925 09:58:21.622402 13773 ts_tablet_manager.cc:1428] T 27b03ccddd364f2f96b9a303ae6fda91 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.020s user 0.005s sys 0.000s
I20260925 09:58:21.623924 13772 raft_consensus.cc:348] T 3cde6676790b4e3f89f05ce60796b36e P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.624523 13772 raft_consensus.cc:374] T 3cde6676790b4e3f89f05ce60796b36e P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:21.624755 13772 raft_consensus.cc:729] T 3cde6676790b4e3f89f05ce60796b36e P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:21.628922 13780 ts_tablet_manager.cc:1918] T 3da072989f69498e99e5e3b32bdb6859 P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:21.629277 13780 log.cc:1196] T 3da072989f69498e99e5e3b32bdb6859 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/3da072989f69498e99e5e3b32bdb6859
I20260925 09:58:21.633064 13770 tablet_bootstrap.cc:654] T 27b03ccddd364f2f96b9a303ae6fda91 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:21.635324 13772 consensus_queue.cc:260] T 3cde6676790b4e3f89f05ce60796b36e P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.635918 13773 tablet_bootstrap.cc:492] T e3027f1b971f483d91c9046163f62abe P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:21.637072 13081 master_service.cc:946] ReplaceTablet: received request to replace tablet af6ace3011c84603b32d0a409d5b8c1c from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:21.641808 13772 ts_tablet_manager.cc:1428] T 3cde6676790b4e3f89f05ce60796b36e P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.020s user 0.002s sys 0.003s
I20260925 09:58:21.642659 13772 tablet_bootstrap.cc:492] T 8313c1ae006341bc84438085dd38f8ad P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:21.644905 13780 ts_tablet_manager.cc:1939] T 3da072989f69498e99e5e3b32bdb6859 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:21.648072 13773 tablet_bootstrap.cc:654] T e3027f1b971f483d91c9046163f62abe P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:21.652614 13671 raft_consensus.cc:2454] T 8313c1ae006341bc84438085dd38f8ad P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:21.657083 13075 catalog_manager.cc:2148] created HMS catalog entry for table [id=dc9c88f862a64597aab300baf32f6f76]
I20260925 09:58:21.659166 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet 3da072989f69498e99e5e3b32bdb6859 (table default.table_a29aa5b33fd644a2ab80242e289ed25d [id=8fa5a3419e394be2a52d1b13713a04d9]) successfully deleted
I20260925 09:58:21.665577 13772 tablet_bootstrap.cc:654] T 8313c1ae006341bc84438085dd38f8ad P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:21.690349 13772 tablet_bootstrap.cc:492] T 8313c1ae006341bc84438085dd38f8ad P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:21.670336 13770 tablet_bootstrap.cc:492] T 27b03ccddd364f2f96b9a303ae6fda91 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:21.691123 13770 ts_tablet_manager.cc:1397] T 27b03ccddd364f2f96b9a303ae6fda91 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.098s user 0.008s sys 0.001s
I20260925 09:58:21.693498 13770 raft_consensus.cc:348] T 27b03ccddd364f2f96b9a303ae6fda91 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.693917 13075 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:50556 (request call id 52) took 2650 ms. Trace:
I20260925 09:58:21.694131 13770 raft_consensus.cc:374] T 27b03ccddd364f2f96b9a303ae6fda91 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:21.694386 13770 raft_consensus.cc:729] T 27b03ccddd364f2f96b9a303ae6fda91 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:21.695108 13971 raft_consensus.cc:482] T 3cde6676790b4e3f89f05ce60796b36e P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:21.695547 13770 consensus_queue.cc:260] T 27b03ccddd364f2f96b9a303ae6fda91 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.695571 13971 raft_consensus.cc:504] T 3cde6676790b4e3f89f05ce60796b36e P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.697453 13971 leader_election.cc:288] T 3cde6676790b4e3f89f05ce60796b36e P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:21.694223 13075 rpcz_store.cc:276] 0925 09:58:19.043320 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:19.048666 (+ 5346us) service_pool.cc:227] Handling call
0925 09:58:20.472537 (+1423871us) catalog_manager.cc:2075] Acquired catalog manager lock
0925 09:58:20.474556 (+ 2019us) catalog_manager.cc:2121] Created new table and tablet info
0925 09:58:21.656923 (+1182367us) catalog_manager.cc:2147] Created new table in HMS catalog
0925 09:58:21.672872 (+ 15949us) catalog_manager.cc:2181] Wrote table and tablets to system table
0925 09:58:21.693513 (+ 20641us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0925 09:58:21.693860 (+ 347us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:58:21.658815 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:58:21.659149 (+ 334us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:58:21.659178 (+ 29us) write_op.cc:435] Acquired schema lock
0925 09:58:21.659190 (+ 12us) tablet.cc:660] Decoding operations
0925 09:58:21.659520 (+ 330us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:58:21.659559 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0925 09:58:21.659580 (+ 21us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:58:21.659772 (+ 192us) tablet.cc:699] Row locks acquired
0925 09:58:21.659784 (+ 12us) write_op.cc:260] PREPARE: finished
0925 09:58:21.659883 (+ 99us) write_op.cc:270] Start()
0925 09:58:21.659958 (+ 75us) write_op.cc:276] Timestamp: P: 1790330301659769 usec, L: 0
0925 09:58:21.659973 (+ 15us) op_driver.cc:331] REPLICATION: starting
0925 09:58:21.660455 (+ 482us) log.cc:842] Serialized 432 byte log entry
0925 09:58:21.670682 (+ 10227us) op_driver.cc:444] REPLICATION: finished
0925 09:58:21.671088 (+ 406us) write_op.cc:301] APPLY: starting
0925 09:58:21.671142 (+ 54us) tablet.cc:1365] starting BulkCheckPresence
0925 09:58:21.671239 (+ 97us) tablet.cc:1368] finished BulkCheckPresence
0925 09:58:21.671328 (+ 89us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:58:21.671760 (+ 432us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:58:21.671896 (+ 136us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 09:58:21.671939 (+ 43us) write_op.cc:312] APPLY: finished
0925 09:58:21.672113 (+ 174us) log.cc:842] Serialized 28 byte log entry
0925 09:58:21.672243 (+ 130us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:21.672311 (+ 68us) write_op.cc:454] Released schema lock
0925 09:58:21.672341 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0925 09:58:21.672508 (+ 167us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:21.672529 (+ 21us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":804029,"Hive Metastore.run_cpu_time_us":767,"Hive Metastore.run_wall_time_us":377308,"child_traces":[["op",{"apply.queue_time_us":245,"apply.run_cpu_time_us":1583,"apply.run_wall_time_us":1582,"num_ops":2,"prepare.queue_time_us":127,"prepare.run_cpu_time_us":1909,"prepare.run_wall_time_us":1910,"raft.queue_time_us":2160,"raft.run_cpu_time_us":652,"raft.run_wall_time_us":652,"replication_time_us":10637}]]}
I20260925 09:58:21.698928 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cde6676790b4e3f89f05ce60796b36e" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
I20260925 09:58:21.699481 13670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cde6676790b4e3f89f05ce60796b36e" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:21.699554 13505 raft_consensus.cc:2454] T 3cde6676790b4e3f89f05ce60796b36e P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 0.
I20260925 09:58:21.700027 13770 ts_tablet_manager.cc:1428] T 27b03ccddd364f2f96b9a303ae6fda91 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20260925 09:58:21.700858 13770 tablet_bootstrap.cc:492] T 963c1e91c6224e5dae4ecc9a6f48f1ac P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:21.700668 13298 leader_election.cc:302] T 3cde6676790b4e3f89f05ce60796b36e P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:21.701689 13772 ts_tablet_manager.cc:1397] T 8313c1ae006341bc84438085dd38f8ad P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.059s user 0.009s sys 0.000s
I20260925 09:58:21.700030 13670 raft_consensus.cc:2454] T 3cde6676790b4e3f89f05ce60796b36e P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 0.
I20260925 09:58:21.702412 13971 raft_consensus.cc:2781] T 3cde6676790b4e3f89f05ce60796b36e P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:21.702745 13971 raft_consensus.cc:482] T 3cde6676790b4e3f89f05ce60796b36e P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:21.703064 13971 raft_consensus.cc:3037] T 3cde6676790b4e3f89f05ce60796b36e P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:21.702765 13075 catalog_manager.cc:5495] T a9d515adc84446e6b46b0332d3ea10db P b3082cf15dc74c0480e5d33ecf592843 reported cstate change: term changed from 0 to 1, leader changed from <none> to b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b3082cf15dc74c0480e5d33ecf592843" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: UNKNOWN } } }
I20260925 09:58:21.704174 13075 catalog_manager.cc:5587] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161) does not have the latest schema for tablet a9d515adc84446e6b46b0332d3ea10db (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]). Expected version 1 got 0
I20260925 09:58:21.709442 13971 raft_consensus.cc:504] T 3cde6676790b4e3f89f05ce60796b36e P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.714351 13772 raft_consensus.cc:348] T 8313c1ae006341bc84438085dd38f8ad P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.712092 13781 ts_tablet_manager.cc:1918] T 3da072989f69498e99e5e3b32bdb6859 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:21.714802 13773 tablet_bootstrap.cc:492] T e3027f1b971f483d91c9046163f62abe P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:21.714996 13772 raft_consensus.cc:374] T 8313c1ae006341bc84438085dd38f8ad P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:21.715229 13772 raft_consensus.cc:729] T 8313c1ae006341bc84438085dd38f8ad P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:21.715337 13773 ts_tablet_manager.cc:1397] T e3027f1b971f483d91c9046163f62abe P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.080s user 0.011s sys 0.000s
I20260925 09:58:21.716627 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cde6676790b4e3f89f05ce60796b36e" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9"
I20260925 09:58:21.717162 13505 raft_consensus.cc:3037] T 3cde6676790b4e3f89f05ce60796b36e P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:21.717427 13773 raft_consensus.cc:348] T e3027f1b971f483d91c9046163f62abe P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.718034 13773 raft_consensus.cc:374] T e3027f1b971f483d91c9046163f62abe P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:21.718247 13773 raft_consensus.cc:729] T e3027f1b971f483d91c9046163f62abe P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:21.719506 13773 consensus_queue.cc:260] T e3027f1b971f483d91c9046163f62abe P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.723460 13772 consensus_queue.cc:260] T 8313c1ae006341bc84438085dd38f8ad P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.725373 13770 tablet_bootstrap.cc:654] T 963c1e91c6224e5dae4ecc9a6f48f1ac P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:21.726027 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet 3da072989f69498e99e5e3b32bdb6859 with delete_type TABLET_DATA_DELETED (replaced by tablet 963c1e91c6224e5dae4ecc9a6f48f1ac) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:21.726061 13505 raft_consensus.cc:2454] T 3cde6676790b4e3f89f05ce60796b36e P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 1.
I20260925 09:58:21.726729 13777 tablet_replica.cc:330] T 3da072989f69498e99e5e3b32bdb6859 P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:21.727099 13298 leader_election.cc:302] T 3cde6676790b4e3f89f05ce60796b36e P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:21.727429 13777 raft_consensus.cc:2229] T 3da072989f69498e99e5e3b32bdb6859 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:21.727784 13781 log.cc:1196] T 3da072989f69498e99e5e3b32bdb6859 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/3da072989f69498e99e5e3b32bdb6859
I20260925 09:58:21.728487 13773 ts_tablet_manager.cc:1428] T e3027f1b971f483d91c9046163f62abe P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.013s user 0.004s sys 0.000s
I20260925 09:58:21.730361 13670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cde6676790b4e3f89f05ce60796b36e" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
I20260925 09:58:21.730975 13670 raft_consensus.cc:3037] T 3cde6676790b4e3f89f05ce60796b36e P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:21.735044 14079 raft_consensus.cc:2781] T 3cde6676790b4e3f89f05ce60796b36e P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:21.735488 13773 tablet_bootstrap.cc:492] T 963c1e91c6224e5dae4ecc9a6f48f1ac P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:21.735940 13777 raft_consensus.cc:2258] T 3da072989f69498e99e5e3b32bdb6859 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:21.737859 13777 ts_tablet_manager.cc:1905] T 3da072989f69498e99e5e3b32bdb6859 P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:21.740916 14079 raft_consensus.cc:686] T 3cde6676790b4e3f89f05ce60796b36e P b3082cf15dc74c0480e5d33ecf592843 [term 1 LEADER]: Becoming Leader. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Running, Role: LEADER
I20260925 09:58:21.741633 13440 leader_election.cc:302] T 8313c1ae006341bc84438085dd38f8ad P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9; no voters: b3082cf15dc74c0480e5d33ecf592843
I20260925 09:58:21.741744 14079 consensus_queue.cc:237] T 3cde6676790b4e3f89f05ce60796b36e P b3082cf15dc74c0480e5d33ecf592843 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.742734 14047 raft_consensus.cc:2781] T 8313c1ae006341bc84438085dd38f8ad P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:21.743134 14047 raft_consensus.cc:686] T 8313c1ae006341bc84438085dd38f8ad P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: Becoming Leader. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Running, Role: LEADER
I20260925 09:58:21.743887 14047 consensus_queue.cc:237] T 8313c1ae006341bc84438085dd38f8ad P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.743911 13670 raft_consensus.cc:2454] T 3cde6676790b4e3f89f05ce60796b36e P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 1.
I20260925 09:58:21.746846 13971 leader_election.cc:288] T 3cde6676790b4e3f89f05ce60796b36e P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Requested vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:21.747813 13773 tablet_bootstrap.cc:654] T 963c1e91c6224e5dae4ecc9a6f48f1ac P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:21.747938 13781 ts_tablet_manager.cc:1939] T 3da072989f69498e99e5e3b32bdb6859 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:21.751748 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet 3da072989f69498e99e5e3b32bdb6859 (table default.table_a29aa5b33fd644a2ab80242e289ed25d [id=8fa5a3419e394be2a52d1b13713a04d9]) successfully deleted
I20260925 09:58:21.754545 13772 ts_tablet_manager.cc:1428] T 8313c1ae006341bc84438085dd38f8ad P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.053s user 0.005s sys 0.000s
I20260925 09:58:21.760882 13772 tablet_bootstrap.cc:492] T 27b03ccddd364f2f96b9a303ae6fda91 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:21.766750 13081 catalog_manager.cc:5587] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327) does not have the latest schema for tablet e3027f1b971f483d91c9046163f62abe (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]). Expected version 1 got 0
I20260925 09:58:21.772322 13772 tablet_bootstrap.cc:654] T 27b03ccddd364f2f96b9a303ae6fda91 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:21.783804 13773 tablet_bootstrap.cc:492] T 963c1e91c6224e5dae4ecc9a6f48f1ac P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:21.784106 13770 tablet_bootstrap.cc:492] T 963c1e91c6224e5dae4ecc9a6f48f1ac P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:21.784265 13773 ts_tablet_manager.cc:1397] T 963c1e91c6224e5dae4ecc9a6f48f1ac P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.049s user 0.009s sys 0.000s
I20260925 09:58:21.784541 13770 ts_tablet_manager.cc:1397] T 963c1e91c6224e5dae4ecc9a6f48f1ac P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.084s user 0.008s sys 0.001s
I20260925 09:58:21.786657 13773 raft_consensus.cc:348] T 963c1e91c6224e5dae4ecc9a6f48f1ac P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.787453 13773 raft_consensus.cc:374] T 963c1e91c6224e5dae4ecc9a6f48f1ac P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:21.787741 13773 raft_consensus.cc:729] T 963c1e91c6224e5dae4ecc9a6f48f1ac P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:21.788277 13777 ts_tablet_manager.cc:1918] T 3da072989f69498e99e5e3b32bdb6859 P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:21.788508 13073 master_service.cc:946] ReplaceTablet: received request to replace tablet af6ace3011c84603b32d0a409d5b8c1c from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:21.786834 13770 raft_consensus.cc:348] T 963c1e91c6224e5dae4ecc9a6f48f1ac P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.788854 13770 raft_consensus.cc:374] T 963c1e91c6224e5dae4ecc9a6f48f1ac P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:21.789125 13770 raft_consensus.cc:729] T 963c1e91c6224e5dae4ecc9a6f48f1ac P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
W20260925 09:58:21.787787 13081 catalog_manager.cc:5687] Failed to send AlterTable RPC for tablet e3027f1b971f483d91c9046163f62abe (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet e3027f1b971f483d91c9046163f62abe consensus state has no leader
I20260925 09:58:21.795565 13777 log.cc:1196] T 3da072989f69498e99e5e3b32bdb6859 P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/3da072989f69498e99e5e3b32bdb6859
I20260925 09:58:21.798238 13777 ts_tablet_manager.cc:1939] T 3da072989f69498e99e5e3b32bdb6859 P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:21.798764 13770 consensus_queue.cc:260] T 963c1e91c6224e5dae4ecc9a6f48f1ac P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.793054 13773 consensus_queue.cc:260] T 963c1e91c6224e5dae4ecc9a6f48f1ac P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.802282 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet 3da072989f69498e99e5e3b32bdb6859 (table default.table_a29aa5b33fd644a2ab80242e289ed25d [id=8fa5a3419e394be2a52d1b13713a04d9]) successfully deleted
I20260925 09:58:21.807170 13773 ts_tablet_manager.cc:1428] T 963c1e91c6224e5dae4ecc9a6f48f1ac P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.023s user 0.006s sys 0.000s
I20260925 09:58:21.816666 13770 ts_tablet_manager.cc:1428] T 963c1e91c6224e5dae4ecc9a6f48f1ac P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.032s user 0.007s sys 0.000s
I20260925 09:58:21.817893 13770 tablet_bootstrap.cc:492] T e3027f1b971f483d91c9046163f62abe P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:21.825254 13772 tablet_bootstrap.cc:492] T 27b03ccddd364f2f96b9a303ae6fda91 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:21.825790 13772 ts_tablet_manager.cc:1397] T 27b03ccddd364f2f96b9a303ae6fda91 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.065s user 0.009s sys 0.001s
I20260925 09:58:21.828128 13772 raft_consensus.cc:348] T 27b03ccddd364f2f96b9a303ae6fda91 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.835562 13772 raft_consensus.cc:374] T 27b03ccddd364f2f96b9a303ae6fda91 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:21.836683 13772 raft_consensus.cc:729] T 27b03ccddd364f2f96b9a303ae6fda91 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:21.837487 13772 consensus_queue.cc:260] T 27b03ccddd364f2f96b9a303ae6fda91 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.843116 13772 ts_tablet_manager.cc:1428] T 27b03ccddd364f2f96b9a303ae6fda91 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.017s user 0.004s sys 0.000s
I20260925 09:58:21.844159 13770 tablet_bootstrap.cc:654] T e3027f1b971f483d91c9046163f62abe P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:21.856896 13786 rpc_server.cc:307] RPC server started. Bound to: 127.10.9.60:42083
I20260925 09:58:21.858855 13772 tablet_bootstrap.cc:492] T 963c1e91c6224e5dae4ecc9a6f48f1ac P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:21.860879 13786 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/master-2/data/info.pb
I20260925 09:58:21.854128 13073 catalog_manager.cc:5495] T 8313c1ae006341bc84438085dd38f8ad P c4d724d4b83e4d1e817a3a505dc24eb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2). New cstate: current_term: 1 leader_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: UNKNOWN } } }
I20260925 09:58:21.869127 13341 tablet_service.cc:1467] Processing CreateTablet for tablet 8115a86775884efb8b9f91646f8abc48 (DEFAULT_TABLE table=default.table_4d3f82d91a5749679343f60e3dbbc422 [id=dc9c88f862a64597aab300baf32f6f76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:21.869796 13484 tablet_service.cc:1467] Processing CreateTablet for tablet 8115a86775884efb8b9f91646f8abc48 (DEFAULT_TABLE table=default.table_4d3f82d91a5749679343f60e3dbbc422 [id=dc9c88f862a64597aab300baf32f6f76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:21.870373 13341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8115a86775884efb8b9f91646f8abc48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:21.870874 13484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8115a86775884efb8b9f91646f8abc48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:21.872176 10276 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskdR3P3Z/build/tsan/bin/kudu as pid 13786
I20260925 09:58:21.877722 13651 tablet_service.cc:1467] Processing CreateTablet for tablet 8115a86775884efb8b9f91646f8abc48 (DEFAULT_TABLE table=default.table_4d3f82d91a5749679343f60e3dbbc422 [id=dc9c88f862a64597aab300baf32f6f76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:21.878880 13651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8115a86775884efb8b9f91646f8abc48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:21.879732 13081 master_service.cc:946] ReplaceTablet: received request to replace tablet 1db61f5876974206be17c93462236bb9 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:21.891201 13770 tablet_bootstrap.cc:492] T e3027f1b971f483d91c9046163f62abe P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:21.891652 13770 ts_tablet_manager.cc:1397] T e3027f1b971f483d91c9046163f62abe P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.074s user 0.008s sys 0.002s
I20260925 09:58:21.893811 13770 raft_consensus.cc:348] T e3027f1b971f483d91c9046163f62abe P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.894439 13770 raft_consensus.cc:374] T e3027f1b971f483d91c9046163f62abe P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:21.894667 13770 raft_consensus.cc:729] T e3027f1b971f483d91c9046163f62abe P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:21.897571 13772 tablet_bootstrap.cc:654] T 963c1e91c6224e5dae4ecc9a6f48f1ac P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:21.904737 13671 raft_consensus.cc:1261] T a9d515adc84446e6b46b0332d3ea10db P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Refusing update from remote peer b3082cf15dc74c0480e5d33ecf592843: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:58:21.906266 14079 consensus_queue.cc:1035] T a9d515adc84446e6b46b0332d3ea10db P b3082cf15dc74c0480e5d33ecf592843 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:58:21.924866 13081 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet 1db61f5876974206be17c93462236bb9
I20260925 09:58:21.926265 13081 catalog_manager.cc:6565] ReplaceTablet: tablet 1db61f5876974206be17c93462236bb9 deleted and replaced by tablet 6edbb7ba7c86437c84adcde397b76222
I20260925 09:58:21.928109 13505 raft_consensus.cc:1261] T a9d515adc84446e6b46b0332d3ea10db P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Refusing update from remote peer b3082cf15dc74c0480e5d33ecf592843: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:58:21.928107 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet 1db61f5876974206be17c93462236bb9 with delete_type TABLET_DATA_DELETED (replaced by tablet 6edbb7ba7c86437c84adcde397b76222) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:21.928740 13777 tablet_replica.cc:330] T 1db61f5876974206be17c93462236bb9 P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:21.929423 13777 raft_consensus.cc:2229] T 1db61f5876974206be17c93462236bb9 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:21.929816 13777 raft_consensus.cc:2258] T 1db61f5876974206be17c93462236bb9 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:21.930162 13650 tablet_service.cc:1514] Processing DeleteTablet for tablet 1db61f5876974206be17c93462236bb9 with delete_type TABLET_DATA_DELETED (replaced by tablet 6edbb7ba7c86437c84adcde397b76222) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:21.931478 13780 tablet_replica.cc:330] T 1db61f5876974206be17c93462236bb9 P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:21.931382 13777 ts_tablet_manager.cc:1905] T 1db61f5876974206be17c93462236bb9 P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:21.931820 14079 consensus_queue.cc:1035] T a9d515adc84446e6b46b0332d3ea10db P b3082cf15dc74c0480e5d33ecf592843 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:21.932126 13780 raft_consensus.cc:2229] T 1db61f5876974206be17c93462236bb9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:58:21.932720 13780 pending_rounds.cc:62] T 1db61f5876974206be17c93462236bb9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Trying to abort 1 pending ops.
I20260925 09:58:21.932950 13780 pending_rounds.cc:68] T 1db61f5876974206be17c93462236bb9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192914355744768 op_type: NO_OP noop_request { }
I20260925 09:58:21.933321 13780 raft_consensus.cc:2866] T 1db61f5876974206be17c93462236bb9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:58:21.933646 13780 raft_consensus.cc:2258] T 1db61f5876974206be17c93462236bb9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:21.936239 13780 ts_tablet_manager.cc:1905] T 1db61f5876974206be17c93462236bb9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:21.937515 13485 tablet_service.cc:1514] Processing DeleteTablet for tablet 1db61f5876974206be17c93462236bb9 with delete_type TABLET_DATA_DELETED (replaced by tablet 6edbb7ba7c86437c84adcde397b76222) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:21.938093 13781 tablet_replica.cc:330] T 1db61f5876974206be17c93462236bb9 P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:21.938766 13781 raft_consensus.cc:2229] T 1db61f5876974206be17c93462236bb9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:21.939183 13781 raft_consensus.cc:2258] T 1db61f5876974206be17c93462236bb9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:21.941275 13781 ts_tablet_manager.cc:1905] T 1db61f5876974206be17c93462236bb9 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:21.939603 13075 catalog_manager.cc:5495] T 3cde6676790b4e3f89f05ce60796b36e P b3082cf15dc74c0480e5d33ecf592843 reported cstate change: term changed from 0 to 1, leader changed from <none> to b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b3082cf15dc74c0480e5d33ecf592843" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: UNKNOWN } } }
I20260925 09:58:21.956355 13081 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50556:
name: "default.Table_ca5cce29b409479f9191e1b06eb5bac0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:58:21.966728 13777 ts_tablet_manager.cc:1918] T 1db61f5876974206be17c93462236bb9 P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:21.865677 14094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.9.60:42083 every 8 connection(s)
I20260925 09:58:21.975629 13777 log.cc:1196] T 1db61f5876974206be17c93462236bb9 P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/1db61f5876974206be17c93462236bb9
I20260925 09:58:21.979238 13780 ts_tablet_manager.cc:1918] T 1db61f5876974206be17c93462236bb9 P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:21.979715 13780 log.cc:1196] T 1db61f5876974206be17c93462236bb9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/1db61f5876974206be17c93462236bb9
I20260925 09:58:21.980607 13780 ts_tablet_manager.cc:1939] T 1db61f5876974206be17c93462236bb9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:21.983484 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet 1db61f5876974206be17c93462236bb9 (table default.table_8f71132b5a004ad89115f7a2278b95d6 [id=0cdc8d8fef5445668c0b0107243e0ae9]) successfully deleted
I20260925 09:58:21.990181 13777 ts_tablet_manager.cc:1939] T 1db61f5876974206be17c93462236bb9 P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:21.895220 13770 consensus_queue.cc:260] T e3027f1b971f483d91c9046163f62abe P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:21.995865 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet 1db61f5876974206be17c93462236bb9 (table default.table_8f71132b5a004ad89115f7a2278b95d6 [id=0cdc8d8fef5445668c0b0107243e0ae9]) successfully deleted
I20260925 09:58:21.996409 13770 ts_tablet_manager.cc:1428] T e3027f1b971f483d91c9046163f62abe P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.104s user 0.005s sys 0.000s
I20260925 09:58:22.003835 13073 master_service.cc:946] ReplaceTablet: received request to replace tablet a9d515adc84446e6b46b0332d3ea10db from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:22.021229 14098 sys_catalog.cc:263] Verifying existing consensus state
I20260925 09:58:22.025658 13781 ts_tablet_manager.cc:1918] T 1db61f5876974206be17c93462236bb9 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:22.071563 13342 tablet_service.cc:1467] Processing CreateTablet for tablet 6edbb7ba7c86437c84adcde397b76222 (DEFAULT_TABLE table=default.table_8f71132b5a004ad89115f7a2278b95d6 [id=0cdc8d8fef5445668c0b0107243e0ae9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:22.072633 13342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6edbb7ba7c86437c84adcde397b76222. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:22.072816 13650 tablet_service.cc:1467] Processing CreateTablet for tablet 6edbb7ba7c86437c84adcde397b76222 (DEFAULT_TABLE table=default.table_8f71132b5a004ad89115f7a2278b95d6 [id=0cdc8d8fef5445668c0b0107243e0ae9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:22.073735 13650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6edbb7ba7c86437c84adcde397b76222. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:22.100950 13409 heartbeater.cc:347] Connected to a master server at 127.10.9.60:42083
W20260925 09:58:22.105521 13066 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 3735920fa0a64b588480af485d0fd6f0 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 3735920fa0a64b588480af485d0fd6f0 consensus state has no leader
I20260925 09:58:22.110069 13073 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet a9d515adc84446e6b46b0332d3ea10db
I20260925 09:58:22.110646 13649 tablet_service.cc:1514] Processing DeleteTablet for tablet a9d515adc84446e6b46b0332d3ea10db with delete_type TABLET_DATA_DELETED (replaced by tablet b7a703a4d9e04401b13610380acd2ac8) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:22.111354 13780 tablet_replica.cc:330] T a9d515adc84446e6b46b0332d3ea10db P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:22.112183 13073 catalog_manager.cc:6565] ReplaceTablet: tablet a9d515adc84446e6b46b0332d3ea10db deleted and replaced by tablet b7a703a4d9e04401b13610380acd2ac8
I20260925 09:58:22.112234 13780 raft_consensus.cc:2229] T a9d515adc84446e6b46b0332d3ea10db P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:22.112622 13780 pending_rounds.cc:62] T a9d515adc84446e6b46b0332d3ea10db P a06c3ef7a6a94f7cbcc03e7cd232f237: Trying to abort 2 pending ops.
I20260925 09:58:22.113406 13649 tablet_service.cc:1514] Processing DeleteTablet for tablet a9d515adc84446e6b46b0332d3ea10db with delete_type TABLET_DATA_DELETED (replaced by tablet b7a703a4d9e04401b13610380acd2ac8) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:22.112820 13780 pending_rounds.cc:68] T a9d515adc84446e6b46b0332d3ea10db P a06c3ef7a6a94f7cbcc03e7cd232f237: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7333192916584280064 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "a9d515adc84446e6b46b0332d3ea10db" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 1 new_table_name: "default.table_e59da12e7b95405787a2f9f73efdc76b" dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" new_extra_config { } }
I20260925 09:58:22.114223 13780 pending_rounds.cc:68] T a9d515adc84446e6b46b0332d3ea10db P a06c3ef7a6a94f7cbcc03e7cd232f237: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192915466883072 op_type: NO_OP noop_request { }
I20260925 09:58:22.115531 13074 catalog_manager.cc:3411] renamed table default.table_a29aa5b33fd644a2ab80242e289ed25d [id=8fa5a3419e394be2a52d1b13713a04d9] in HMS: new name default.table_81574dad60e9490cb79e268b12969a83
I20260925 09:58:22.114876 14098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761: Bootstrap starting.
I20260925 09:58:22.117681 13341 tablet_service.cc:1514] Processing DeleteTablet for tablet a9d515adc84446e6b46b0332d3ea10db with delete_type TABLET_DATA_DELETED (replaced by tablet b7a703a4d9e04401b13610380acd2ac8) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:22.118324 13777 tablet_replica.cc:330] T a9d515adc84446e6b46b0332d3ea10db P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:22.118990 13777 raft_consensus.cc:2229] T a9d515adc84446e6b46b0332d3ea10db P b3082cf15dc74c0480e5d33ecf592843 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:58:22.116043 13780 raft_consensus.cc:2866] T a9d515adc84446e6b46b0332d3ea10db P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:58:22.119531 13777 pending_rounds.cc:62] T a9d515adc84446e6b46b0332d3ea10db P b3082cf15dc74c0480e5d33ecf592843: Trying to abort 2 pending ops.
I20260925 09:58:22.119520 13780 raft_consensus.cc:2258] T a9d515adc84446e6b46b0332d3ea10db P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:22.119717 13777 pending_rounds.cc:68] T a9d515adc84446e6b46b0332d3ea10db P b3082cf15dc74c0480e5d33ecf592843: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7333192916584280064 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "a9d515adc84446e6b46b0332d3ea10db" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 1 new_table_name: "default.table_e59da12e7b95405787a2f9f73efdc76b" dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" new_extra_config { } }
W20260925 09:58:22.120743 13777 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:49260: Aborted: Op aborted
W20260925 09:58:22.127962 13066 catalog_manager.cc:4920] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): alter failed for tablet a9d515adc84446e6b46b0332d3ea10db (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]): Aborted: Op aborted
I20260925 09:58:22.128718 13780 ts_tablet_manager.cc:1905] T a9d515adc84446e6b46b0332d3ea10db P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:22.121538 13777 pending_rounds.cc:68] T a9d515adc84446e6b46b0332d3ea10db P b3082cf15dc74c0480e5d33ecf592843: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192915466883072 op_type: NO_OP noop_request { }
I20260925 09:58:22.131806 13777 raft_consensus.cc:2866] T a9d515adc84446e6b46b0332d3ea10db P b3082cf15dc74c0480e5d33ecf592843 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:58:22.132117 13777 raft_consensus.cc:2258] T a9d515adc84446e6b46b0332d3ea10db P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:22.134207 13777 ts_tablet_manager.cc:1905] T a9d515adc84446e6b46b0332d3ea10db P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:22.135857 14079 raft_consensus.cc:482] T 27b03ccddd364f2f96b9a303ae6fda91 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:22.136343 13269 catalog_manager.cc:2709] Deleting table default.table_3823faf045944cc1a828ce929b8a7275 [id=f6ae78b7a67640f5a6e71e04249b4262] in response to Hive Metastore notification log event 16
I20260925 09:58:22.136281 14079 raft_consensus.cc:504] T 27b03ccddd364f2f96b9a303ae6fda91 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:22.139588 14079 leader_election.cc:288] T 27b03ccddd364f2f96b9a303ae6fda91 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:22.140457 13409 heartbeater.cc:347] Connected to a master server at 127.10.9.60:42083
I20260925 09:58:22.140058 13670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27b03ccddd364f2f96b9a303ae6fda91" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:22.141444 13670 raft_consensus.cc:2454] T 27b03ccddd364f2f96b9a303ae6fda91 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 0.
I20260925 09:58:22.142472 13297 leader_election.cc:302] T 27b03ccddd364f2f96b9a303ae6fda91 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, b3082cf15dc74c0480e5d33ecf592843; no voters:
I20260925 09:58:22.143211 14079 raft_consensus.cc:2781] T 27b03ccddd364f2f96b9a303ae6fda91 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:22.143597 14079 raft_consensus.cc:482] T 27b03ccddd364f2f96b9a303ae6fda91 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:22.144250 14079 raft_consensus.cc:3037] T 27b03ccddd364f2f96b9a303ae6fda91 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:22.150110 14079 raft_consensus.cc:504] T 27b03ccddd364f2f96b9a303ae6fda91 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:22.163902 13780 ts_tablet_manager.cc:1918] T a9d515adc84446e6b46b0332d3ea10db P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:58:22.164233 13780 log.cc:1196] T a9d515adc84446e6b46b0332d3ea10db P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/a9d515adc84446e6b46b0332d3ea10db
I20260925 09:58:22.164848 13780 ts_tablet_manager.cc:1939] T a9d515adc84446e6b46b0332d3ea10db P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:22.168659 13649 tablet_service.cc:1467] Processing CreateTablet for tablet b7a703a4d9e04401b13610380acd2ac8 (DEFAULT_TABLE table=default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:22.170765 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet a9d515adc84446e6b46b0332d3ea10db (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]) successfully deleted
W20260925 09:58:22.171454 13068 catalog_manager.cc:4808] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): delete failed for tablet a9d515adc84446e6b46b0332d3ea10db because the tablet was not found. No further retry: Not found: Tablet not found: a9d515adc84446e6b46b0332d3ea10db
I20260925 09:58:22.172487 13649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7a703a4d9e04401b13610380acd2ac8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:22.181291 13073 catalog_manager.cc:5587] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731) does not have the latest schema for tablet e3027f1b971f483d91c9046163f62abe (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]). Expected version 1 got 0
I20260925 09:58:22.187592 13770 tablet_bootstrap.cc:492] T 8115a86775884efb8b9f91646f8abc48 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:22.192147 13770 tablet_bootstrap.cc:654] T 8115a86775884efb8b9f91646f8abc48 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:22.167510 13342 tablet_service.cc:1467] Processing CreateTablet for tablet b7a703a4d9e04401b13610380acd2ac8 (DEFAULT_TABLE table=default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:22.182385 13772 tablet_bootstrap.cc:492] T 963c1e91c6224e5dae4ecc9a6f48f1ac P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:22.199762 13772 ts_tablet_manager.cc:1397] T 963c1e91c6224e5dae4ecc9a6f48f1ac P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.341s user 0.018s sys 0.011s
I20260925 09:58:22.200073 13342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7a703a4d9e04401b13610380acd2ac8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:22.201869 13772 raft_consensus.cc:348] T 963c1e91c6224e5dae4ecc9a6f48f1ac P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:22.202504 13772 raft_consensus.cc:374] T 963c1e91c6224e5dae4ecc9a6f48f1ac P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:22.202713 13772 raft_consensus.cc:729] T 963c1e91c6224e5dae4ecc9a6f48f1ac P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:22.205698 13772 consensus_queue.cc:260] T 963c1e91c6224e5dae4ecc9a6f48f1ac P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:22.210345 13269 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet 27b03ccddd364f2f96b9a303ae6fda91
I20260925 09:58:22.212543 13341 tablet_service.cc:1514] Processing DeleteTablet for tablet 27b03ccddd364f2f96b9a303ae6fda91 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:58:22 UTC) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:22.214550 13772 ts_tablet_manager.cc:1428] T 963c1e91c6224e5dae4ecc9a6f48f1ac P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.015s user 0.003s sys 0.002s
I20260925 09:58:22.216662 13648 tablet_service.cc:1514] Processing DeleteTablet for tablet 27b03ccddd364f2f96b9a303ae6fda91 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:58:22 UTC) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:22.217017 13772 tablet_bootstrap.cc:492] T e3027f1b971f483d91c9046163f62abe P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:22.217414 13780 tablet_replica.cc:330] T 27b03ccddd364f2f96b9a303ae6fda91 P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:22.218044 13780 raft_consensus.cc:2229] T 27b03ccddd364f2f96b9a303ae6fda91 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:22.218456 13780 raft_consensus.cc:2258] T 27b03ccddd364f2f96b9a303ae6fda91 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:22.220084 13780 ts_tablet_manager.cc:1905] T 27b03ccddd364f2f96b9a303ae6fda91 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:22.221954 13772 tablet_bootstrap.cc:654] T e3027f1b971f483d91c9046163f62abe P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:22.067013 13485 tablet_service.cc:1467] Processing CreateTablet for tablet 6edbb7ba7c86437c84adcde397b76222 (DEFAULT_TABLE table=default.table_8f71132b5a004ad89115f7a2278b95d6 [id=0cdc8d8fef5445668c0b0107243e0ae9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:22.222246 13781 log.cc:1196] T 1db61f5876974206be17c93462236bb9 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/1db61f5876974206be17c93462236bb9
I20260925 09:58:22.222772 13485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6edbb7ba7c86437c84adcde397b76222. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:22.222883 13781 ts_tablet_manager.cc:1939] T 1db61f5876974206be17c93462236bb9 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:22.115021 13483 tablet_service.cc:1514] Processing DeleteTablet for tablet a9d515adc84446e6b46b0332d3ea10db with delete_type TABLET_DATA_DELETED (replaced by tablet b7a703a4d9e04401b13610380acd2ac8) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:22.117236 13482 tablet_service.cc:1514] Processing DeleteTablet for tablet a9d515adc84446e6b46b0332d3ea10db with delete_type TABLET_DATA_DELETED (replaced by tablet b7a703a4d9e04401b13610380acd2ac8) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:22.152988 13550 heartbeater.cc:347] Connected to a master server at 127.10.9.60:42083
I20260925 09:58:22.227648 14018 raft_consensus.cc:482] T e3027f1b971f483d91c9046163f62abe P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:22.228044 14018 raft_consensus.cc:504] T e3027f1b971f483d91c9046163f62abe P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:22.229846 14018 leader_election.cc:288] T e3027f1b971f483d91c9046163f62abe P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
I20260925 09:58:22.236524 13781 tablet_replica.cc:330] T a9d515adc84446e6b46b0332d3ea10db P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:22.236717 13503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3027f1b971f483d91c9046163f62abe" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
I20260925 09:58:22.237272 13781 raft_consensus.cc:2229] T a9d515adc84446e6b46b0332d3ea10db P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:22.237367 13503 raft_consensus.cc:2454] T e3027f1b971f483d91c9046163f62abe P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 0.
I20260925 09:58:22.237659 13781 pending_rounds.cc:62] T a9d515adc84446e6b46b0332d3ea10db P c4d724d4b83e4d1e817a3a505dc24eb9: Trying to abort 2 pending ops.
I20260925 09:58:22.237867 13781 pending_rounds.cc:68] T a9d515adc84446e6b46b0332d3ea10db P c4d724d4b83e4d1e817a3a505dc24eb9: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7333192916584280064 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "a9d515adc84446e6b46b0332d3ea10db" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 1 new_table_name: "default.table_e59da12e7b95405787a2f9f73efdc76b" dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" new_extra_config { } }
I20260925 09:58:22.239055 13781 pending_rounds.cc:68] T a9d515adc84446e6b46b0332d3ea10db P c4d724d4b83e4d1e817a3a505dc24eb9: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192915466883072 op_type: NO_OP noop_request { }
I20260925 09:58:22.241088 13606 leader_election.cc:302] T e3027f1b971f483d91c9046163f62abe P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:22.241997 14018 raft_consensus.cc:2781] T e3027f1b971f483d91c9046163f62abe P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:22.242347 14018 raft_consensus.cc:482] T e3027f1b971f483d91c9046163f62abe P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:22.242632 14018 raft_consensus.cc:3037] T e3027f1b971f483d91c9046163f62abe P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:22.173718 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27b03ccddd364f2f96b9a303ae6fda91" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
I20260925 09:58:22.178794 13481 tablet_service.cc:1467] Processing CreateTablet for tablet b7a703a4d9e04401b13610380acd2ac8 (DEFAULT_TABLE table=default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:22.217240 13504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27b03ccddd364f2f96b9a303ae6fda91" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9"
I20260925 09:58:22.253633 13777 ts_tablet_manager.cc:1918] T a9d515adc84446e6b46b0332d3ea10db P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:58:22.254096 13777 log.cc:1196] T a9d515adc84446e6b46b0332d3ea10db P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/a9d515adc84446e6b46b0332d3ea10db
I20260925 09:58:22.255946 14018 raft_consensus.cc:504] T e3027f1b971f483d91c9046163f62abe P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:22.256601 13770 tablet_bootstrap.cc:492] T 8115a86775884efb8b9f91646f8abc48 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:22.245309 13773 tablet_bootstrap.cc:492] T 8115a86775884efb8b9f91646f8abc48 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:22.256968 13770 ts_tablet_manager.cc:1397] T 8115a86775884efb8b9f91646f8abc48 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.070s user 0.012s sys 0.000s
I20260925 09:58:22.248023 13504 raft_consensus.cc:3037] T 27b03ccddd364f2f96b9a303ae6fda91 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:22.261865 13773 tablet_bootstrap.cc:654] T 8115a86775884efb8b9f91646f8abc48 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:22.259033 13770 raft_consensus.cc:348] T 8115a86775884efb8b9f91646f8abc48 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:22.262781 13770 raft_consensus.cc:374] T 8115a86775884efb8b9f91646f8abc48 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:22.263026 13770 raft_consensus.cc:729] T 8115a86775884efb8b9f91646f8abc48 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:22.263535 13504 raft_consensus.cc:2454] T 27b03ccddd364f2f96b9a303ae6fda91 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 1.
I20260925 09:58:22.239370 13781 raft_consensus.cc:2866] T a9d515adc84446e6b46b0332d3ea10db P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:58:22.265370 13781 raft_consensus.cc:2258] T a9d515adc84446e6b46b0332d3ea10db P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:22.266436 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet 1db61f5876974206be17c93462236bb9 (table default.table_8f71132b5a004ad89115f7a2278b95d6 [id=0cdc8d8fef5445668c0b0107243e0ae9]) successfully deleted
I20260925 09:58:22.266817 13484 tablet_service.cc:1514] Processing DeleteTablet for tablet 27b03ccddd364f2f96b9a303ae6fda91 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:58:22 UTC) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:22.269476 13481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7a703a4d9e04401b13610380acd2ac8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:22.271461 13781 ts_tablet_manager.cc:1905] T a9d515adc84446e6b46b0332d3ea10db P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:22.272706 14018 leader_election.cc:288] T e3027f1b971f483d91c9046163f62abe P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 election: Requested vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
I20260925 09:58:22.263618 13770 consensus_queue.cc:260] T 8115a86775884efb8b9f91646f8abc48 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:22.276888 13770 ts_tablet_manager.cc:1428] T 8115a86775884efb8b9f91646f8abc48 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.020s user 0.002s sys 0.003s
I20260925 09:58:22.277581 13770 tablet_bootstrap.cc:492] T b7a703a4d9e04401b13610380acd2ac8 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:22.290970 13777 ts_tablet_manager.cc:1939] T a9d515adc84446e6b46b0332d3ea10db P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:22.291812 13298 leader_election.cc:302] T 27b03ccddd364f2f96b9a303ae6fda91 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:22.292582 13777 tablet_replica.cc:330] T 27b03ccddd364f2f96b9a303ae6fda91 P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:22.293396 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet a9d515adc84446e6b46b0332d3ea10db (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]) successfully deleted
I20260925 09:58:22.300603 13073 master_service.cc:946] ReplaceTablet: received request to replace tablet 27b03ccddd364f2f96b9a303ae6fda91 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:22.263773 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3027f1b971f483d91c9046163f62abe" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9"
I20260925 09:58:22.285542 13773 tablet_bootstrap.cc:492] T 8115a86775884efb8b9f91646f8abc48 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:22.306155 13773 ts_tablet_manager.cc:1397] T 8115a86775884efb8b9f91646f8abc48 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.061s user 0.007s sys 0.003s
I20260925 09:58:22.306107 13505 raft_consensus.cc:3037] T e3027f1b971f483d91c9046163f62abe P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:22.308480 13773 raft_consensus.cc:348] T 8115a86775884efb8b9f91646f8abc48 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:22.309125 13773 raft_consensus.cc:374] T 8115a86775884efb8b9f91646f8abc48 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:22.309366 13773 raft_consensus.cc:729] T 8115a86775884efb8b9f91646f8abc48 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:22.310035 13773 consensus_queue.cc:260] T 8115a86775884efb8b9f91646f8abc48 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:22.314042 13780 ts_tablet_manager.cc:1918] T 27b03ccddd364f2f96b9a303ae6fda91 P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:22.314373 13780 log.cc:1196] T 27b03ccddd364f2f96b9a303ae6fda91 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/27b03ccddd364f2f96b9a303ae6fda91
I20260925 09:58:22.314953 13770 tablet_bootstrap.cc:654] T b7a703a4d9e04401b13610380acd2ac8 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:22.314980 13780 ts_tablet_manager.cc:1939] T 27b03ccddd364f2f96b9a303ae6fda91 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:22.323508 13550 heartbeater.cc:347] Connected to a master server at 127.10.9.60:42083
I20260925 09:58:22.325764 13716 heartbeater.cc:347] Connected to a master server at 127.10.9.60:42083
I20260925 09:58:22.327023 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet 27b03ccddd364f2f96b9a303ae6fda91 (table default.table_3823faf045944cc1a828ce929b8a7275 [id=f6ae78b7a67640f5a6e71e04249b4262]) successfully deleted
I20260925 09:58:22.292727 13971 raft_consensus.cc:2781] T 27b03ccddd364f2f96b9a303ae6fda91 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:22.344225 13773 ts_tablet_manager.cc:1428] T 8115a86775884efb8b9f91646f8abc48 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.038s user 0.005s sys 0.000s
I20260925 09:58:22.350072 13773 tablet_bootstrap.cc:492] T b7a703a4d9e04401b13610380acd2ac8 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:22.351270 13770 tablet_bootstrap.cc:492] T b7a703a4d9e04401b13610380acd2ac8 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:22.351722 13770 ts_tablet_manager.cc:1397] T b7a703a4d9e04401b13610380acd2ac8 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.074s user 0.009s sys 0.000s
I20260925 09:58:22.353777 13770 raft_consensus.cc:348] T b7a703a4d9e04401b13610380acd2ac8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:22.354355 13770 raft_consensus.cc:374] T b7a703a4d9e04401b13610380acd2ac8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:22.354588 13770 raft_consensus.cc:729] T b7a703a4d9e04401b13610380acd2ac8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:22.355103 13770 consensus_queue.cc:260] T b7a703a4d9e04401b13610380acd2ac8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:22.373103 13770 ts_tablet_manager.cc:1428] T b7a703a4d9e04401b13610380acd2ac8 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.021s user 0.005s sys 0.000s
I20260925 09:58:22.373864 13770 tablet_bootstrap.cc:492] T 6edbb7ba7c86437c84adcde397b76222 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:22.319329 14098 log.cc:824] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761: Log is configured to *not* fsync() on all Append() calls
I20260925 09:58:22.365700 13505 raft_consensus.cc:2454] T e3027f1b971f483d91c9046163f62abe P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 1.
W20260925 09:58:22.383776 13269 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 963c1e91c6224e5dae4ecc9a6f48f1ac consensus state has no leader
W20260925 09:58:22.385716 13079 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_2c3c283350d644f39f1903713ab06589 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:58:22.359632 14079 leader_election.cc:288] T 27b03ccddd364f2f96b9a303ae6fda91 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Requested vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:22.403049 13773 tablet_bootstrap.cc:654] T b7a703a4d9e04401b13610380acd2ac8 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:22.405550 13770 tablet_bootstrap.cc:654] T 6edbb7ba7c86437c84adcde397b76222 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:22.406078 13606 leader_election.cc:302] T e3027f1b971f483d91c9046163f62abe P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:22.406824 14018 raft_consensus.cc:2781] T e3027f1b971f483d91c9046163f62abe P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election won for term 1
W20260925 09:58:22.412017 13297 leader_election.cc:341] T 27b03ccddd364f2f96b9a303ae6fda91 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Not found: Tablet not found: 27b03ccddd364f2f96b9a303ae6fda91
I20260925 09:58:22.293238 13362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3027f1b971f483d91c9046163f62abe" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843"
I20260925 09:58:22.293365 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3027f1b971f483d91c9046163f62abe" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
W20260925 09:58:22.415463 13603 leader_election.cc:341] T e3027f1b971f483d91c9046163f62abe P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:58:22.416782 13603 leader_election.cc:341] T e3027f1b971f483d91c9046163f62abe P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:22.418591 13971 raft_consensus.cc:686] T 27b03ccddd364f2f96b9a303ae6fda91 P b3082cf15dc74c0480e5d33ecf592843 [term 1 LEADER]: Becoming Leader. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Running, Role: LEADER
I20260925 09:58:22.419368 13971 consensus_queue.cc:237] T 27b03ccddd364f2f96b9a303ae6fda91 P b3082cf15dc74c0480e5d33ecf592843 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:22.420877 14018 raft_consensus.cc:686] T e3027f1b971f483d91c9046163f62abe P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 LEADER]: Becoming Leader. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Running, Role: LEADER
I20260925 09:58:22.421628 14018 consensus_queue.cc:237] T e3027f1b971f483d91c9046163f62abe P a06c3ef7a6a94f7cbcc03e7cd232f237 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:22.425750 13772 tablet_bootstrap.cc:492] T e3027f1b971f483d91c9046163f62abe P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:22.426133 13772 ts_tablet_manager.cc:1397] T e3027f1b971f483d91c9046163f62abe P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.209s user 0.004s sys 0.006s
I20260925 09:58:22.426203 13773 tablet_bootstrap.cc:492] T b7a703a4d9e04401b13610380acd2ac8 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:22.426559 13773 ts_tablet_manager.cc:1397] T b7a703a4d9e04401b13610380acd2ac8 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.077s user 0.004s sys 0.004s
I20260925 09:58:22.425880 13073 master_service.cc:946] ReplaceTablet: received request to replace tablet 6edbb7ba7c86437c84adcde397b76222 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:22.428130 13772 raft_consensus.cc:348] T e3027f1b971f483d91c9046163f62abe P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:22.428761 13772 raft_consensus.cc:374] T e3027f1b971f483d91c9046163f62abe P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:22.429028 13772 raft_consensus.cc:729] T e3027f1b971f483d91c9046163f62abe P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:22.434413 13777 raft_consensus.cc:2229] T 27b03ccddd364f2f96b9a303ae6fda91 P b3082cf15dc74c0480e5d33ecf592843 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:58:22.435505 13777 pending_rounds.cc:62] T 27b03ccddd364f2f96b9a303ae6fda91 P b3082cf15dc74c0480e5d33ecf592843: Trying to abort 1 pending ops.
I20260925 09:58:22.435708 13777 pending_rounds.cc:68] T 27b03ccddd364f2f96b9a303ae6fda91 P b3082cf15dc74c0480e5d33ecf592843: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192918733426688 op_type: NO_OP noop_request { }
I20260925 09:58:22.436040 13777 raft_consensus.cc:2866] T 27b03ccddd364f2f96b9a303ae6fda91 P b3082cf15dc74c0480e5d33ecf592843 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:58:22.436307 13777 raft_consensus.cc:2258] T 27b03ccddd364f2f96b9a303ae6fda91 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:22.442608 13772 consensus_queue.cc:260] T e3027f1b971f483d91c9046163f62abe P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:22.451143 13777 ts_tablet_manager.cc:1905] T 27b03ccddd364f2f96b9a303ae6fda91 P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:22.456085 14047 raft_consensus.cc:482] T 8115a86775884efb8b9f91646f8abc48 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:22.456312 13076 catalog_manager.cc:2685] deleted HMS catalog entry for table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]
I20260925 09:58:22.456527 14047 raft_consensus.cc:504] T 8115a86775884efb8b9f91646f8abc48 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:22.456961 13772 ts_tablet_manager.cc:1428] T e3027f1b971f483d91c9046163f62abe P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.031s user 0.007s sys 0.000s
I20260925 09:58:22.458377 14047 leader_election.cc:288] T 8115a86775884efb8b9f91646f8abc48 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:22.458666 13772 tablet_bootstrap.cc:492] T 8115a86775884efb8b9f91646f8abc48 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:22.459978 13362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8115a86775884efb8b9f91646f8abc48" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:22.460772 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8115a86775884efb8b9f91646f8abc48" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
W20260925 09:58:22.461342 13438 leader_election.cc:341] T 8115a86775884efb8b9f91646f8abc48 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:22.461369 13671 raft_consensus.cc:2454] T 8115a86775884efb8b9f91646f8abc48 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 0.
I20260925 09:58:22.462522 13440 leader_election.cc:302] T 8115a86775884efb8b9f91646f8abc48 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9; no voters: b3082cf15dc74c0480e5d33ecf592843
I20260925 09:58:22.466667 14047 raft_consensus.cc:2781] T 8115a86775884efb8b9f91646f8abc48 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:22.466993 14047 raft_consensus.cc:482] T 8115a86775884efb8b9f91646f8abc48 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:22.467288 14047 raft_consensus.cc:3037] T 8115a86775884efb8b9f91646f8abc48 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:22.472939 13772 tablet_bootstrap.cc:654] T 8115a86775884efb8b9f91646f8abc48 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:22.477563 13075 catalog_manager.cc:5587] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731) does not have the latest schema for tablet b7a703a4d9e04401b13610380acd2ac8 (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]). Expected version 1 got 0
W20260925 09:58:22.480309 13075 catalog_manager.cc:5687] Failed to send AlterTable RPC for tablet b7a703a4d9e04401b13610380acd2ac8 (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet b7a703a4d9e04401b13610380acd2ac8 consensus state has no leader
I20260925 09:58:22.485313 13781 ts_tablet_manager.cc:1918] T a9d515adc84446e6b46b0332d3ea10db P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:58:22.492802 13075 catalog_manager.cc:5587] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161) does not have the latest schema for tablet 963c1e91c6224e5dae4ecc9a6f48f1ac (table default.table_81574dad60e9490cb79e268b12969a83 [id=8fa5a3419e394be2a52d1b13713a04d9]). Expected version 1 got 0
I20260925 09:58:22.494513 13772 tablet_bootstrap.cc:492] T 8115a86775884efb8b9f91646f8abc48 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:22.495380 13772 ts_tablet_manager.cc:1397] T 8115a86775884efb8b9f91646f8abc48 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.038s user 0.009s sys 0.002s
I20260925 09:58:22.499886 13772 raft_consensus.cc:348] T 8115a86775884efb8b9f91646f8abc48 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:22.500507 13772 raft_consensus.cc:374] T 8115a86775884efb8b9f91646f8abc48 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:22.500744 13772 raft_consensus.cc:729] T 8115a86775884efb8b9f91646f8abc48 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:22.501551 13772 consensus_queue.cc:260] T 8115a86775884efb8b9f91646f8abc48 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:22.504344 13772 ts_tablet_manager.cc:1428] T 8115a86775884efb8b9f91646f8abc48 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20260925 09:58:22.505138 13772 tablet_bootstrap.cc:492] T 6edbb7ba7c86437c84adcde397b76222 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:22.448599 13773 raft_consensus.cc:348] T b7a703a4d9e04401b13610380acd2ac8 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:22.506613 13773 raft_consensus.cc:374] T b7a703a4d9e04401b13610380acd2ac8 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:22.506937 13773 raft_consensus.cc:729] T b7a703a4d9e04401b13610380acd2ac8 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:22.507653 13773 consensus_queue.cc:260] T b7a703a4d9e04401b13610380acd2ac8 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:22.475075 14047 raft_consensus.cc:504] T 8115a86775884efb8b9f91646f8abc48 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:22.511561 13362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8115a86775884efb8b9f91646f8abc48" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843"
I20260925 09:58:22.512100 13362 raft_consensus.cc:3037] T 8115a86775884efb8b9f91646f8abc48 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:22.518484 13362 raft_consensus.cc:2454] T 8115a86775884efb8b9f91646f8abc48 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:22.519701 13438 leader_election.cc:302] T 8115a86775884efb8b9f91646f8abc48 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:22.521663 13772 tablet_bootstrap.cc:654] T 6edbb7ba7c86437c84adcde397b76222 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:22.534228 14047 leader_election.cc:288] T 8115a86775884efb8b9f91646f8abc48 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Requested vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:22.534471 13073 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet 6edbb7ba7c86437c84adcde397b76222
I20260925 09:58:22.534795 14047 raft_consensus.cc:482] T 963c1e91c6224e5dae4ecc9a6f48f1ac P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:22.535276 14047 raft_consensus.cc:504] T 963c1e91c6224e5dae4ecc9a6f48f1ac P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:22.538292 13772 tablet_bootstrap.cc:492] T 6edbb7ba7c86437c84adcde397b76222 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:22.538338 13770 tablet_bootstrap.cc:492] T 6edbb7ba7c86437c84adcde397b76222 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:22.538663 13772 ts_tablet_manager.cc:1397] T 6edbb7ba7c86437c84adcde397b76222 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.034s user 0.005s sys 0.004s
I20260925 09:58:22.538738 13770 ts_tablet_manager.cc:1397] T 6edbb7ba7c86437c84adcde397b76222 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.165s user 0.009s sys 0.008s
I20260925 09:58:22.540794 13770 raft_consensus.cc:348] T 6edbb7ba7c86437c84adcde397b76222 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:22.540791 13772 raft_consensus.cc:348] T 6edbb7ba7c86437c84adcde397b76222 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:22.541419 13770 raft_consensus.cc:374] T 6edbb7ba7c86437c84adcde397b76222 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:22.541409 13772 raft_consensus.cc:374] T 6edbb7ba7c86437c84adcde397b76222 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:22.541653 13770 raft_consensus.cc:729] T 6edbb7ba7c86437c84adcde397b76222 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:22.541653 13772 raft_consensus.cc:729] T 6edbb7ba7c86437c84adcde397b76222 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:22.542239 13770 consensus_queue.cc:260] T 6edbb7ba7c86437c84adcde397b76222 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:22.542470 13772 consensus_queue.cc:260] T 6edbb7ba7c86437c84adcde397b76222 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:22.544433 13770 ts_tablet_manager.cc:1428] T 6edbb7ba7c86437c84adcde397b76222 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20260925 09:58:22.545701 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet 6edbb7ba7c86437c84adcde397b76222 with delete_type TABLET_DATA_DELETED (replaced by tablet caa7f8c507de4d1b9bcc78a39e8482f9) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:22.546415 13971 raft_consensus.cc:482] T 8313c1ae006341bc84438085dd38f8ad P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:22.546795 13971 raft_consensus.cc:504] T 8313c1ae006341bc84438085dd38f8ad P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:22.548521 13971 leader_election.cc:288] T 8313c1ae006341bc84438085dd38f8ad P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:22.551965 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8313c1ae006341bc84438085dd38f8ad" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:22.552702 13671 raft_consensus.cc:2379] T 8313c1ae006341bc84438085dd38f8ad P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3082cf15dc74c0480e5d33ecf592843 in current term 1: Already voted for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in this term.
I20260925 09:58:22.560443 13650 tablet_service.cc:1514] Processing DeleteTablet for tablet 6edbb7ba7c86437c84adcde397b76222 with delete_type TABLET_DATA_DELETED (replaced by tablet caa7f8c507de4d1b9bcc78a39e8482f9) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:22.561097 13780 tablet_replica.cc:330] T 6edbb7ba7c86437c84adcde397b76222 P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:22.561736 13780 raft_consensus.cc:2229] T 6edbb7ba7c86437c84adcde397b76222 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:22.562170 13780 raft_consensus.cc:2258] T 6edbb7ba7c86437c84adcde397b76222 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:22.563875 13780 ts_tablet_manager.cc:1905] T 6edbb7ba7c86437c84adcde397b76222 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:22.566849 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8115a86775884efb8b9f91646f8abc48" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
I20260925 09:58:22.567333 13671 raft_consensus.cc:3037] T 8115a86775884efb8b9f91646f8abc48 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:22.575191 13772 ts_tablet_manager.cc:1428] T 6edbb7ba7c86437c84adcde397b76222 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.036s user 0.005s sys 0.001s
I20260925 09:58:22.576691 13671 raft_consensus.cc:2454] T 8115a86775884efb8b9f91646f8abc48 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:22.541683 13781 log.cc:1196] T a9d515adc84446e6b46b0332d3ea10db P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/a9d515adc84446e6b46b0332d3ea10db
I20260925 09:58:22.550467 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8313c1ae006341bc84438085dd38f8ad" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
I20260925 09:58:22.581596 13078 catalog_manager.cc:2148] created HMS catalog entry for table [id=7f3f3dd4c3d74f21878caad38c9c25b9]
I20260925 09:58:22.582456 13781 ts_tablet_manager.cc:1939] T a9d515adc84446e6b46b0332d3ea10db P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:22.583523 13298 leader_election.cc:302] T 8313c1ae006341bc84438085dd38f8ad P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:58:22.585331 13781 tablet_replica.cc:330] T 27b03ccddd364f2f96b9a303ae6fda91 P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:22.585954 13781 raft_consensus.cc:2229] T 27b03ccddd364f2f96b9a303ae6fda91 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:22.586323 13781 raft_consensus.cc:2258] T 27b03ccddd364f2f96b9a303ae6fda91 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:22.588428 13971 raft_consensus.cc:3037] T 8313c1ae006341bc84438085dd38f8ad P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:22.592298 13773 ts_tablet_manager.cc:1428] T b7a703a4d9e04401b13610380acd2ac8 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.166s user 0.011s sys 0.000s
I20260925 09:58:22.593302 14047 leader_election.cc:288] T 963c1e91c6224e5dae4ecc9a6f48f1ac P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:22.600519 13777 ts_tablet_manager.cc:1918] T 27b03ccddd364f2f96b9a303ae6fda91 P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:22.605197 13971 raft_consensus.cc:2726] T 8313c1ae006341bc84438085dd38f8ad P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:58:22.604933 13777 log.cc:1196] T 27b03ccddd364f2f96b9a303ae6fda91 P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/27b03ccddd364f2f96b9a303ae6fda91
I20260925 09:58:22.604885 13781 ts_tablet_manager.cc:1905] T 27b03ccddd364f2f96b9a303ae6fda91 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:22.610041 13078 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:50556 (request call id 59) took 3232 ms. Trace:
I20260925 09:58:22.610374 13078 rpcz_store.cc:276] 0925 09:58:19.377718 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:19.380117 (+ 2399us) service_pool.cc:227] Handling call
0925 09:58:20.693245 (+1313128us) catalog_manager.cc:2075] Acquired catalog manager lock
0925 09:58:20.695083 (+ 1838us) catalog_manager.cc:2121] Created new table and tablet info
0925 09:58:22.581461 (+1886378us) catalog_manager.cc:2147] Created new table in HMS catalog
0925 09:58:22.609270 (+ 27809us) catalog_manager.cc:2181] Wrote table and tablets to system table
0925 09:58:22.609610 (+ 340us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0925 09:58:22.609976 (+ 366us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:58:22.584526 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:58:22.584848 (+ 322us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:58:22.584871 (+ 23us) write_op.cc:435] Acquired schema lock
0925 09:58:22.584884 (+ 13us) tablet.cc:660] Decoding operations
0925 09:58:22.585141 (+ 257us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:58:22.585178 (+ 37us) write_op.cc:641] Partition lock acquired for write op
0925 09:58:22.585198 (+ 20us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:58:22.585392 (+ 194us) tablet.cc:699] Row locks acquired
0925 09:58:22.585405 (+ 13us) write_op.cc:260] PREPARE: finished
0925 09:58:22.585536 (+ 131us) write_op.cc:270] Start()
0925 09:58:22.585613 (+ 77us) write_op.cc:276] Timestamp: P: 1790330302585470 usec, L: 0
0925 09:58:22.585630 (+ 17us) op_driver.cc:331] REPLICATION: starting
0925 09:58:22.586078 (+ 448us) log.cc:842] Serialized 433 byte log entry
0925 09:58:22.607559 (+ 21481us) op_driver.cc:444] REPLICATION: finished
0925 09:58:22.607866 (+ 307us) write_op.cc:301] APPLY: starting
0925 09:58:22.607921 (+ 55us) tablet.cc:1365] starting BulkCheckPresence
0925 09:58:22.608021 (+ 100us) tablet.cc:1368] finished BulkCheckPresence
0925 09:58:22.608105 (+ 84us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:58:22.608491 (+ 386us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:58:22.608627 (+ 136us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 09:58:22.608668 (+ 41us) write_op.cc:312] APPLY: finished
0925 09:58:22.608839 (+ 171us) log.cc:842] Serialized 29 byte log entry
0925 09:58:22.608973 (+ 134us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:22.609044 (+ 71us) write_op.cc:454] Released schema lock
0925 09:58:22.609072 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0925 09:58:22.609240 (+ 168us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:22.609262 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":1642623,"Hive Metastore.run_cpu_time_us":605,"Hive Metastore.run_wall_time_us":125128,"child_traces":[["op",{"apply.queue_time_us":148,"apply.run_cpu_time_us":1554,"apply.run_wall_time_us":1554,"num_ops":2,"prepare.queue_time_us":1314,"prepare.run_cpu_time_us":1795,"prepare.run_wall_time_us":1796,"raft.queue_time_us":5712,"raft.run_cpu_time_us":642,"raft.run_wall_time_us":642,"replication_time_us":21852}]]}
I20260925 09:58:22.614990 13777 ts_tablet_manager.cc:1939] T 27b03ccddd364f2f96b9a303ae6fda91 P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:22.616110 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "963c1e91c6224e5dae4ecc9a6f48f1ac" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:22.617535 13671 raft_consensus.cc:2454] T 963c1e91c6224e5dae4ecc9a6f48f1ac P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 0.
I20260925 09:58:22.618506 13440 leader_election.cc:302] T 963c1e91c6224e5dae4ecc9a6f48f1ac P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:22.619374 14047 raft_consensus.cc:2781] T 963c1e91c6224e5dae4ecc9a6f48f1ac P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:22.619753 14047 raft_consensus.cc:482] T 963c1e91c6224e5dae4ecc9a6f48f1ac P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:22.620170 14047 raft_consensus.cc:3037] T 963c1e91c6224e5dae4ecc9a6f48f1ac P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:22.623436 13971 consensus_queue.cc:1035] T 3cde6676790b4e3f89f05ce60796b36e P b3082cf15dc74c0480e5d33ecf592843 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:22.561167 13073 catalog_manager.cc:6565] ReplaceTablet: tablet 6edbb7ba7c86437c84adcde397b76222 deleted and replaced by tablet caa7f8c507de4d1b9bcc78a39e8482f9
I20260925 09:58:22.628407 13780 ts_tablet_manager.cc:1918] T 6edbb7ba7c86437c84adcde397b76222 P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:22.632612 13716 heartbeater.cc:347] Connected to a master server at 127.10.9.60:42083
I20260925 09:58:22.633965 13773 tablet_bootstrap.cc:492] T 6edbb7ba7c86437c84adcde397b76222 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:22.640529 13773 tablet_bootstrap.cc:654] T 6edbb7ba7c86437c84adcde397b76222 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:22.631067 13780 log.cc:1196] T 6edbb7ba7c86437c84adcde397b76222 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/6edbb7ba7c86437c84adcde397b76222
I20260925 09:58:22.645125 13780 ts_tablet_manager.cc:1939] T 6edbb7ba7c86437c84adcde397b76222 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:22.565502 14127 raft_consensus.cc:2781] T 8115a86775884efb8b9f91646f8abc48 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election won for term 1
W20260925 09:58:22.575591 13438 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.1:42161, user_credentials={real_user=slave}} blocked reactor thread for 56012.6us
I20260925 09:58:22.647740 13485 tablet_service.cc:1514] Processing DeleteTablet for tablet 6edbb7ba7c86437c84adcde397b76222 with delete_type TABLET_DATA_DELETED (replaced by tablet caa7f8c507de4d1b9bcc78a39e8482f9) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:22.648051 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet 6edbb7ba7c86437c84adcde397b76222 (table default.table_8f71132b5a004ad89115f7a2278b95d6 [id=0cdc8d8fef5445668c0b0107243e0ae9]) successfully deleted
I20260925 09:58:22.649416 13362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "963c1e91c6224e5dae4ecc9a6f48f1ac" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:22.650050 13362 raft_consensus.cc:2454] T 963c1e91c6224e5dae4ecc9a6f48f1ac P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 0.
I20260925 09:58:22.651069 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet a9d515adc84446e6b46b0332d3ea10db (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]) successfully deleted
I20260925 09:58:22.653199 14123 consensus_queue.cc:1035] T 8313c1ae006341bc84438085dd38f8ad P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260925 09:58:22.651718 13069 catalog_manager.cc:4808] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): delete failed for tablet a9d515adc84446e6b46b0332d3ea10db because the tablet was not found. No further retry: Not found: Tablet not found: a9d515adc84446e6b46b0332d3ea10db
I20260925 09:58:22.668042 14127 raft_consensus.cc:686] T 8115a86775884efb8b9f91646f8abc48 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: Becoming Leader. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Running, Role: LEADER
I20260925 09:58:22.668967 14127 consensus_queue.cc:237] T 8115a86775884efb8b9f91646f8abc48 P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:22.676136 13772 tablet_bootstrap.cc:492] T b7a703a4d9e04401b13610380acd2ac8 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:22.677774 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet 27b03ccddd364f2f96b9a303ae6fda91 (table default.table_3823faf045944cc1a828ce929b8a7275 [id=f6ae78b7a67640f5a6e71e04249b4262]) successfully deleted
I20260925 09:58:22.678759 13777 tablet_replica.cc:330] T 6edbb7ba7c86437c84adcde397b76222 P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:22.679571 13777 raft_consensus.cc:2229] T 6edbb7ba7c86437c84adcde397b76222 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:22.680047 13777 raft_consensus.cc:2258] T 6edbb7ba7c86437c84adcde397b76222 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:22.680815 13772 tablet_bootstrap.cc:654] T b7a703a4d9e04401b13610380acd2ac8 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:22.682183 13777 ts_tablet_manager.cc:1905] T 6edbb7ba7c86437c84adcde397b76222 P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:22.648878 14047 raft_consensus.cc:504] T 963c1e91c6224e5dae4ecc9a6f48f1ac P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:22.695232 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "963c1e91c6224e5dae4ecc9a6f48f1ac" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843"
I20260925 09:58:22.695535 14047 leader_election.cc:288] T 963c1e91c6224e5dae4ecc9a6f48f1ac P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Requested vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:22.695904 13363 raft_consensus.cc:3037] T 963c1e91c6224e5dae4ecc9a6f48f1ac P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:22.696710 13670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "963c1e91c6224e5dae4ecc9a6f48f1ac" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
I20260925 09:58:22.697297 13670 raft_consensus.cc:3037] T 963c1e91c6224e5dae4ecc9a6f48f1ac P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:22.701745 13363 raft_consensus.cc:2454] T 963c1e91c6224e5dae4ecc9a6f48f1ac P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:22.703961 13438 leader_election.cc:302] T 963c1e91c6224e5dae4ecc9a6f48f1ac P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:22.704135 13670 raft_consensus.cc:2454] T 963c1e91c6224e5dae4ecc9a6f48f1ac P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:22.705607 14047 raft_consensus.cc:2781] T 963c1e91c6224e5dae4ecc9a6f48f1ac P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:22.713045 13772 tablet_bootstrap.cc:492] T b7a703a4d9e04401b13610380acd2ac8 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:22.713485 13772 ts_tablet_manager.cc:1397] T b7a703a4d9e04401b13610380acd2ac8 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.038s user 0.010s sys 0.005s
I20260925 09:58:22.715548 14047 raft_consensus.cc:686] T 963c1e91c6224e5dae4ecc9a6f48f1ac P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: Becoming Leader. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Running, Role: LEADER
I20260925 09:58:22.719754 13772 raft_consensus.cc:348] T b7a703a4d9e04401b13610380acd2ac8 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:22.717694 14047 consensus_queue.cc:237] T 963c1e91c6224e5dae4ecc9a6f48f1ac P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:22.727491 13772 raft_consensus.cc:374] T b7a703a4d9e04401b13610380acd2ac8 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:22.728672 13772 raft_consensus.cc:729] T b7a703a4d9e04401b13610380acd2ac8 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:22.732488 13772 consensus_queue.cc:260] T b7a703a4d9e04401b13610380acd2ac8 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:22.734290 14123 consensus_queue.cc:1035] T 8313c1ae006341bc84438085dd38f8ad P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:58:22.730351 14079 consensus_queue.cc:1035] T 3cde6676790b4e3f89f05ce60796b36e P b3082cf15dc74c0480e5d33ecf592843 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:58:22.741776 13079 catalog_manager.cc:2148] created HMS catalog entry for table [id=fec42fd086714e079c23b924ca453b65]
I20260925 09:58:22.742599 13772 ts_tablet_manager.cc:1428] T b7a703a4d9e04401b13610380acd2ac8 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.029s user 0.005s sys 0.001s
I20260925 09:58:22.757053 13079 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:50556 (request call id 68) took 2071 ms. Trace:
I20260925 09:58:22.757397 13079 rpcz_store.cc:276] 0925 09:58:20.685261 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:20.686243 (+ 982us) service_pool.cc:227] Handling call
0925 09:58:22.386139 (+1699896us) catalog_manager.cc:2075] Acquired catalog manager lock
0925 09:58:22.389553 (+ 3414us) catalog_manager.cc:2121] Created new table and tablet info
0925 09:58:22.741737 (+352184us) catalog_manager.cc:2147] Created new table in HMS catalog
0925 09:58:22.756397 (+ 14660us) catalog_manager.cc:2181] Wrote table and tablets to system table
0925 09:58:22.756721 (+ 324us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0925 09:58:22.756997 (+ 276us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:58:22.743619 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:58:22.743974 (+ 355us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:58:22.744011 (+ 37us) write_op.cc:435] Acquired schema lock
0925 09:58:22.744025 (+ 14us) tablet.cc:660] Decoding operations
0925 09:58:22.744272 (+ 247us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:58:22.744308 (+ 36us) write_op.cc:641] Partition lock acquired for write op
0925 09:58:22.744327 (+ 19us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:58:22.744520 (+ 193us) tablet.cc:699] Row locks acquired
0925 09:58:22.744532 (+ 12us) write_op.cc:260] PREPARE: finished
0925 09:58:22.744631 (+ 99us) write_op.cc:270] Start()
0925 09:58:22.744705 (+ 74us) write_op.cc:276] Timestamp: P: 1790330302744553 usec, L: 0
0925 09:58:22.744719 (+ 14us) op_driver.cc:331] REPLICATION: starting
0925 09:58:22.745191 (+ 472us) log.cc:842] Serialized 433 byte log entry
0925 09:58:22.753775 (+ 8584us) op_driver.cc:444] REPLICATION: finished
0925 09:58:22.754088 (+ 313us) write_op.cc:301] APPLY: starting
0925 09:58:22.754136 (+ 48us) tablet.cc:1365] starting BulkCheckPresence
0925 09:58:22.754236 (+ 100us) tablet.cc:1368] finished BulkCheckPresence
0925 09:58:22.754345 (+ 109us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:58:22.754769 (+ 424us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:58:22.754906 (+ 137us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 09:58:22.754943 (+ 37us) write_op.cc:312] APPLY: finished
0925 09:58:22.755116 (+ 173us) log.cc:842] Serialized 29 byte log entry
0925 09:58:22.755243 (+ 127us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:22.755314 (+ 71us) write_op.cc:454] Released schema lock
0925 09:58:22.755343 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0925 09:58:22.755547 (+ 204us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:22.755574 (+ 27us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":190845,"Hive Metastore.run_cpu_time_us":593,"Hive Metastore.run_wall_time_us":160494,"child_traces":[["op",{"apply.queue_time_us":176,"apply.run_cpu_time_us":1637,"apply.run_wall_time_us":1636,"num_ops":2,"prepare.queue_time_us":191,"prepare.run_cpu_time_us":1841,"prepare.run_wall_time_us":1841,"raft.queue_time_us":1148,"raft.run_cpu_time_us":608,"raft.run_wall_time_us":608,"replication_time_us":8985}]]}
I20260925 09:58:22.768325 13079 master_service.cc:946] ReplaceTablet: received request to replace tablet 27b03ccddd364f2f96b9a303ae6fda91 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:22.775890 13269 catalog_manager.cc:2709] Deleting table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e] in response to Hive Metastore notification log event 19
I20260925 09:58:22.820274 13079 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50556:
name: "default.Table_47c0ac37d06f4f45a701b0910a42c579"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:58:22.826606 13075 catalog_manager.cc:5587] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327) does not have the latest schema for tablet b7a703a4d9e04401b13610380acd2ac8 (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]). Expected version 1 got 0
I20260925 09:58:22.831877 13078 catalog_manager.cc:5495] T e3027f1b971f483d91c9046163f62abe P a06c3ef7a6a94f7cbcc03e7cd232f237 reported cstate change: term changed from 0 to 1, leader changed from <none> to a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3). New cstate: current_term: 1 leader_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: HEALTHY } } }
I20260925 09:58:22.833452 13078 catalog_manager.cc:5587] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731) does not have the latest schema for tablet e3027f1b971f483d91c9046163f62abe (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]). Expected version 1 got 0
I20260925 09:58:22.843109 13773 tablet_bootstrap.cc:492] T 6edbb7ba7c86437c84adcde397b76222 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:22.844702 13075 master_service.cc:946] ReplaceTablet: received request to replace tablet af6ace3011c84603b32d0a409d5b8c1c from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:22.843664 13773 ts_tablet_manager.cc:1397] T 6edbb7ba7c86437c84adcde397b76222 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.210s user 0.016s sys 0.000s
I20260925 09:58:22.848266 13342 tablet_service.cc:1467] Processing CreateTablet for tablet caa7f8c507de4d1b9bcc78a39e8482f9 (DEFAULT_TABLE table=default.table_8f71132b5a004ad89115f7a2278b95d6 [id=0cdc8d8fef5445668c0b0107243e0ae9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:22.849229 13342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet caa7f8c507de4d1b9bcc78a39e8482f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:22.849608 14127 raft_consensus.cc:482] T b7a703a4d9e04401b13610380acd2ac8 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:22.849992 14127 raft_consensus.cc:504] T b7a703a4d9e04401b13610380acd2ac8 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:22.853936 13073 catalog_manager.cc:5587] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161) does not have the latest schema for tablet e3027f1b971f483d91c9046163f62abe (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]). Expected version 1 got 0
I20260925 09:58:22.854524 14127 leader_election.cc:288] T b7a703a4d9e04401b13610380acd2ac8 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:22.854832 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7a703a4d9e04401b13610380acd2ac8" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:22.855472 13363 raft_consensus.cc:2454] T b7a703a4d9e04401b13610380acd2ac8 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 0.
I20260925 09:58:22.856348 13669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7a703a4d9e04401b13610380acd2ac8" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:22.856812 13438 leader_election.cc:302] T b7a703a4d9e04401b13610380acd2ac8 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:22.857061 13669 raft_consensus.cc:2454] T b7a703a4d9e04401b13610380acd2ac8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 0.
I20260925 09:58:22.859615 14123 raft_consensus.cc:2781] T b7a703a4d9e04401b13610380acd2ac8 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:22.859934 14123 raft_consensus.cc:482] T b7a703a4d9e04401b13610380acd2ac8 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:22.860212 14123 raft_consensus.cc:3037] T b7a703a4d9e04401b13610380acd2ac8 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:22.866575 13781 ts_tablet_manager.cc:1918] T 27b03ccddd364f2f96b9a303ae6fda91 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:22.866998 13781 log.cc:1196] T 27b03ccddd364f2f96b9a303ae6fda91 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/27b03ccddd364f2f96b9a303ae6fda91
I20260925 09:58:22.867815 13781 ts_tablet_manager.cc:1939] T 27b03ccddd364f2f96b9a303ae6fda91 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:22.869309 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet 27b03ccddd364f2f96b9a303ae6fda91 (table default.table_3823faf045944cc1a828ce929b8a7275 [id=f6ae78b7a67640f5a6e71e04249b4262]) successfully deleted
W20260925 09:58:22.870956 13069 catalog_manager.cc:4821] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): delete failed for tablet 6edbb7ba7c86437c84adcde397b76222 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6edbb7ba7c86437c84adcde397b76222 already in progress: creating tablet
I20260925 09:58:22.871584 13773 raft_consensus.cc:348] T 6edbb7ba7c86437c84adcde397b76222 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:22.886739 13772 tablet_bootstrap.cc:492] T caa7f8c507de4d1b9bcc78a39e8482f9 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:22.889786 13650 tablet_service.cc:1467] Processing CreateTablet for tablet caa7f8c507de4d1b9bcc78a39e8482f9 (DEFAULT_TABLE table=default.table_8f71132b5a004ad89115f7a2278b95d6 [id=0cdc8d8fef5445668c0b0107243e0ae9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:22.890790 13650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet caa7f8c507de4d1b9bcc78a39e8482f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:22.891872 13772 tablet_bootstrap.cc:654] T caa7f8c507de4d1b9bcc78a39e8482f9 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:22.896880 13773 raft_consensus.cc:374] T 6edbb7ba7c86437c84adcde397b76222 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:22.897187 13773 raft_consensus.cc:729] T 6edbb7ba7c86437c84adcde397b76222 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:22.867025 14123 raft_consensus.cc:504] T b7a703a4d9e04401b13610380acd2ac8 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:22.891808 13485 tablet_service.cc:1467] Processing CreateTablet for tablet caa7f8c507de4d1b9bcc78a39e8482f9 (DEFAULT_TABLE table=default.table_8f71132b5a004ad89115f7a2278b95d6 [id=0cdc8d8fef5445668c0b0107243e0ae9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:22.900153 13485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet caa7f8c507de4d1b9bcc78a39e8482f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:22.903287 13773 consensus_queue.cc:260] T 6edbb7ba7c86437c84adcde397b76222 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:22.905716 13777 ts_tablet_manager.cc:1918] T 6edbb7ba7c86437c84adcde397b76222 P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:22.906234 14098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260925 09:58:22.908216 13777 log.cc:1196] T 6edbb7ba7c86437c84adcde397b76222 P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/6edbb7ba7c86437c84adcde397b76222
I20260925 09:58:22.912683 13777 ts_tablet_manager.cc:1939] T 6edbb7ba7c86437c84adcde397b76222 P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:22.915356 13773 ts_tablet_manager.cc:1428] T 6edbb7ba7c86437c84adcde397b76222 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.068s user 0.005s sys 0.000s
I20260925 09:58:22.916711 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet 6edbb7ba7c86437c84adcde397b76222 (table default.table_8f71132b5a004ad89115f7a2278b95d6 [id=0cdc8d8fef5445668c0b0107243e0ae9]) successfully deleted
I20260925 09:58:22.924793 14123 leader_election.cc:288] T b7a703a4d9e04401b13610380acd2ac8 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Requested vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:22.925781 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7a703a4d9e04401b13610380acd2ac8" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843"
I20260925 09:58:22.926292 13363 raft_consensus.cc:3037] T b7a703a4d9e04401b13610380acd2ac8 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:22.930886 13669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7a703a4d9e04401b13610380acd2ac8" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
I20260925 09:58:22.931653 13669 raft_consensus.cc:3037] T b7a703a4d9e04401b13610380acd2ac8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:22.933199 13363 raft_consensus.cc:2454] T b7a703a4d9e04401b13610380acd2ac8 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:22.934240 13438 leader_election.cc:302] T b7a703a4d9e04401b13610380acd2ac8 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:22.935827 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet 6edbb7ba7c86437c84adcde397b76222 with delete_type TABLET_DATA_DELETED (replaced by tablet caa7f8c507de4d1b9bcc78a39e8482f9) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:22.936553 14123 raft_consensus.cc:2781] T b7a703a4d9e04401b13610380acd2ac8 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:22.937693 14123 raft_consensus.cc:686] T b7a703a4d9e04401b13610380acd2ac8 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: Becoming Leader. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Running, Role: LEADER
I20260925 09:58:22.939206 14123 consensus_queue.cc:237] T b7a703a4d9e04401b13610380acd2ac8 P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
W20260925 09:58:22.942423 13066 catalog_manager.cc:4808] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): delete failed for tablet 6edbb7ba7c86437c84adcde397b76222 because the tablet was not found. No further retry: Not found: Tablet not found: 6edbb7ba7c86437c84adcde397b76222
I20260925 09:58:22.943177 13269 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet e3027f1b971f483d91c9046163f62abe
I20260925 09:58:22.943320 13770 tablet_bootstrap.cc:492] T caa7f8c507de4d1b9bcc78a39e8482f9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:22.945427 13481 tablet_service.cc:1514] Processing DeleteTablet for tablet e3027f1b971f483d91c9046163f62abe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:58:22 UTC) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:22.946038 13649 tablet_service.cc:1514] Processing DeleteTablet for tablet e3027f1b971f483d91c9046163f62abe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:58:22 UTC) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:22.946681 13780 tablet_replica.cc:330] T e3027f1b971f483d91c9046163f62abe P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:22.947304 13780 raft_consensus.cc:2229] T e3027f1b971f483d91c9046163f62abe P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:58:22.947912 13780 pending_rounds.cc:62] T e3027f1b971f483d91c9046163f62abe P a06c3ef7a6a94f7cbcc03e7cd232f237: Trying to abort 1 pending ops.
I20260925 09:58:22.948098 13780 pending_rounds.cc:68] T e3027f1b971f483d91c9046163f62abe P a06c3ef7a6a94f7cbcc03e7cd232f237: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192918727225344 op_type: NO_OP noop_request { }
I20260925 09:58:22.948410 13780 raft_consensus.cc:2866] T e3027f1b971f483d91c9046163f62abe P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:58:22.948673 13780 raft_consensus.cc:2258] T e3027f1b971f483d91c9046163f62abe P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:22.950650 13781 tablet_replica.cc:330] T e3027f1b971f483d91c9046163f62abe P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:22.950819 13780 ts_tablet_manager.cc:1905] T e3027f1b971f483d91c9046163f62abe P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:22.951306 13781 raft_consensus.cc:2229] T e3027f1b971f483d91c9046163f62abe P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:22.951735 13781 raft_consensus.cc:2258] T e3027f1b971f483d91c9046163f62abe P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:22.953469 13781 ts_tablet_manager.cc:1905] T e3027f1b971f483d91c9046163f62abe P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:22.948688 13770 tablet_bootstrap.cc:654] T caa7f8c507de4d1b9bcc78a39e8482f9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:22.958768 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet e3027f1b971f483d91c9046163f62abe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:58:22 UTC) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:22.959164 13075 catalog_manager.cc:5495] T 8115a86775884efb8b9f91646f8abc48 P c4d724d4b83e4d1e817a3a505dc24eb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2). New cstate: current_term: 1 leader_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: UNKNOWN } } }
I20260925 09:58:22.960922 13075 catalog_manager.cc:5495] T 963c1e91c6224e5dae4ecc9a6f48f1ac P c4d724d4b83e4d1e817a3a505dc24eb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2). New cstate: current_term: 1 leader_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: UNKNOWN } } }
I20260925 09:58:22.962265 13075 catalog_manager.cc:5587] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327) does not have the latest schema for tablet 963c1e91c6224e5dae4ecc9a6f48f1ac (table default.table_81574dad60e9490cb79e268b12969a83 [id=8fa5a3419e394be2a52d1b13713a04d9]). Expected version 1 got 0
W20260925 09:58:22.966647 13080 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_41304460d5a345c88866a7429d816aca in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:58:22.970475 13082 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:50556 (request call id 55) took 3889 ms. Trace:
I20260925 09:58:22.970818 13082 rpcz_store.cc:276] 0925 09:58:19.080769 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:19.080879 (+ 110us) service_pool.cc:227] Handling call
0925 09:58:20.426651 (+1345772us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:58:22.970348 (+2543697us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":345063,"Hive Metastore.run_cpu_time_us":503,"Hive Metastore.run_wall_time_us":461524}
I20260925 09:58:22.972419 13082 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50556:
name: "default.Table_5f3286f12ae947ef8c1dfa71d8dd21d2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 09:58:22.975692 13081 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_ca5cce29b409479f9191e1b06eb5bac0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:58:22.980068 13777 tablet_replica.cc:330] T e3027f1b971f483d91c9046163f62abe P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:22.980737 13777 raft_consensus.cc:2229] T e3027f1b971f483d91c9046163f62abe P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:22.981141 13777 raft_consensus.cc:2258] T e3027f1b971f483d91c9046163f62abe P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:22.983134 13777 ts_tablet_manager.cc:1905] T e3027f1b971f483d91c9046163f62abe P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:22.984760 13074 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:50556 (request call id 54) took 3904 ms. Trace:
I20260925 09:58:22.985116 13074 rpcz_store.cc:276] 0925 09:58:19.080412 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:19.080912 (+ 500us) service_pool.cc:227] Handling call
0925 09:58:20.426380 (+1345468us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:58:22.980017 (+2553637us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:58:22.982494 (+ 2477us) catalog_manager.cc:3616] Apply alter schema
0925 09:58:22.984689 (+ 2195us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":1180458,"Hive Metastore.run_cpu_time_us":2054,"Hive Metastore.run_wall_time_us":458219,"spinlock_wait_cycles":196864}
I20260925 09:58:22.986522 13074 master_service.cc:946] ReplaceTablet: received request to replace tablet e75bfd761e314d60b6638eb5174b0227 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:22.992980 13770 tablet_bootstrap.cc:492] T caa7f8c507de4d1b9bcc78a39e8482f9 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:22.993458 13770 ts_tablet_manager.cc:1397] T caa7f8c507de4d1b9bcc78a39e8482f9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.050s user 0.011s sys 0.000s
I20260925 09:58:22.995643 13770 raft_consensus.cc:348] T caa7f8c507de4d1b9bcc78a39e8482f9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:22.996637 13770 raft_consensus.cc:374] T caa7f8c507de4d1b9bcc78a39e8482f9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:22.998946 13772 tablet_bootstrap.cc:492] T caa7f8c507de4d1b9bcc78a39e8482f9 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:22.999020 13770 raft_consensus.cc:729] T caa7f8c507de4d1b9bcc78a39e8482f9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:22.999430 13772 ts_tablet_manager.cc:1397] T caa7f8c507de4d1b9bcc78a39e8482f9 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.113s user 0.012s sys 0.006s
I20260925 09:58:23.001662 13772 raft_consensus.cc:348] T caa7f8c507de4d1b9bcc78a39e8482f9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:23.002391 13772 raft_consensus.cc:374] T caa7f8c507de4d1b9bcc78a39e8482f9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:23.002712 13772 raft_consensus.cc:729] T caa7f8c507de4d1b9bcc78a39e8482f9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:23.003382 13772 consensus_queue.cc:260] T caa7f8c507de4d1b9bcc78a39e8482f9 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:23.003929 13770 consensus_queue.cc:260] T caa7f8c507de4d1b9bcc78a39e8482f9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:23.004983 13074 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet e75bfd761e314d60b6638eb5174b0227
I20260925 09:58:23.006644 14098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761: Bootstrap complete.
I20260925 09:58:23.011029 13481 tablet_service.cc:1514] Processing DeleteTablet for tablet e75bfd761e314d60b6638eb5174b0227 with delete_type TABLET_DATA_DELETED (replaced by tablet cedc5b0609e14c409542a83efc45f3f0) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:23.018692 13074 catalog_manager.cc:6565] ReplaceTablet: tablet e75bfd761e314d60b6638eb5174b0227 deleted and replaced by tablet cedc5b0609e14c409542a83efc45f3f0
I20260925 09:58:23.026114 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet e75bfd761e314d60b6638eb5174b0227 with delete_type TABLET_DATA_DELETED (replaced by tablet cedc5b0609e14c409542a83efc45f3f0) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:23.009383 13650 tablet_service.cc:1514] Processing DeleteTablet for tablet e75bfd761e314d60b6638eb5174b0227 with delete_type TABLET_DATA_DELETED (replaced by tablet cedc5b0609e14c409542a83efc45f3f0) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:23.034216 13773 tablet_bootstrap.cc:492] T caa7f8c507de4d1b9bcc78a39e8482f9 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:23.035684 13770 ts_tablet_manager.cc:1428] T caa7f8c507de4d1b9bcc78a39e8482f9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.042s user 0.005s sys 0.000s
I20260925 09:58:23.037624 13772 ts_tablet_manager.cc:1428] T caa7f8c507de4d1b9bcc78a39e8482f9 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.038s user 0.005s sys 0.000s
I20260925 09:58:23.040406 13773 tablet_bootstrap.cc:654] T caa7f8c507de4d1b9bcc78a39e8482f9 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:23.018034 13669 raft_consensus.cc:2454] T b7a703a4d9e04401b13610380acd2ac8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:23.025731 13780 ts_tablet_manager.cc:1918] T e3027f1b971f483d91c9046163f62abe P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:23.053083 13780 log.cc:1196] T e3027f1b971f483d91c9046163f62abe P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/e3027f1b971f483d91c9046163f62abe
I20260925 09:58:23.053879 13780 ts_tablet_manager.cc:1939] T e3027f1b971f483d91c9046163f62abe P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:23.053915 14018 raft_consensus.cc:482] T caa7f8c507de4d1b9bcc78a39e8482f9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:23.054381 14018 raft_consensus.cc:504] T caa7f8c507de4d1b9bcc78a39e8482f9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:23.056244 14018 leader_election.cc:288] T caa7f8c507de4d1b9bcc78a39e8482f9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
I20260925 09:58:23.057596 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caa7f8c507de4d1b9bcc78a39e8482f9" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
I20260925 09:58:23.058744 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caa7f8c507de4d1b9bcc78a39e8482f9" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:23.059536 13363 raft_consensus.cc:2454] T caa7f8c507de4d1b9bcc78a39e8482f9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 0.
I20260925 09:58:23.060976 13603 leader_election.cc:302] T caa7f8c507de4d1b9bcc78a39e8482f9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, b3082cf15dc74c0480e5d33ecf592843; no voters:
I20260925 09:58:23.061700 14018 raft_consensus.cc:2781] T caa7f8c507de4d1b9bcc78a39e8482f9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:23.062005 14018 raft_consensus.cc:482] T caa7f8c507de4d1b9bcc78a39e8482f9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:23.062330 14018 raft_consensus.cc:3037] T caa7f8c507de4d1b9bcc78a39e8482f9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:58:23.063655 13606 leader_election.cc:341] T caa7f8c507de4d1b9bcc78a39e8482f9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:23.073627 13074 master_service.cc:946] ReplaceTablet: received request to replace tablet e3027f1b971f483d91c9046163f62abe from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:23.076395 13780 tablet_replica.cc:330] T e75bfd761e314d60b6638eb5174b0227 P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:23.076292 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet e3027f1b971f483d91c9046163f62abe (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]) successfully deleted
I20260925 09:58:23.077169 13780 raft_consensus.cc:2229] T e75bfd761e314d60b6638eb5174b0227 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:23.077633 13780 raft_consensus.cc:2258] T e75bfd761e314d60b6638eb5174b0227 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:23.078768 13781 ts_tablet_manager.cc:1918] T e3027f1b971f483d91c9046163f62abe P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:23.079140 13781 log.cc:1196] T e3027f1b971f483d91c9046163f62abe P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/e3027f1b971f483d91c9046163f62abe
I20260925 09:58:23.079917 14018 raft_consensus.cc:504] T caa7f8c507de4d1b9bcc78a39e8482f9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:23.085048 13650 tablet_service.cc:1514] Processing DeleteTablet for tablet e3027f1b971f483d91c9046163f62abe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:58:22 UTC) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:23.087646 14018 leader_election.cc:288] T caa7f8c507de4d1b9bcc78a39e8482f9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 election: Requested vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
I20260925 09:58:23.088403 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caa7f8c507de4d1b9bcc78a39e8482f9" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843"
I20260925 09:58:23.088708 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caa7f8c507de4d1b9bcc78a39e8482f9" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9"
I20260925 09:58:23.088965 13363 raft_consensus.cc:3037] T caa7f8c507de4d1b9bcc78a39e8482f9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:58:23.089941 13606 leader_election.cc:341] T caa7f8c507de4d1b9bcc78a39e8482f9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:23.096026 13363 raft_consensus.cc:2454] T caa7f8c507de4d1b9bcc78a39e8482f9 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 1.
I20260925 09:58:23.087168 13073 catalog_manager.cc:5587] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161) does not have the latest schema for tablet b7a703a4d9e04401b13610380acd2ac8 (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]). Expected version 1 got 0
I20260925 09:58:23.098922 13780 ts_tablet_manager.cc:1905] T e75bfd761e314d60b6638eb5174b0227 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:23.099984 13781 ts_tablet_manager.cc:1939] T e3027f1b971f483d91c9046163f62abe P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:23.109715 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet e3027f1b971f483d91c9046163f62abe (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]) successfully deleted
I20260925 09:58:23.113018 13781 tablet_replica.cc:330] T e75bfd761e314d60b6638eb5174b0227 P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:23.114979 13781 raft_consensus.cc:2229] T e75bfd761e314d60b6638eb5174b0227 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:58:23.115597 13773 tablet_bootstrap.cc:492] T caa7f8c507de4d1b9bcc78a39e8482f9 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:23.124049 13781 raft_consensus.cc:2258] T e75bfd761e314d60b6638eb5174b0227 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:23.124377 13773 ts_tablet_manager.cc:1397] T caa7f8c507de4d1b9bcc78a39e8482f9 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.090s user 0.011s sys 0.010s
I20260925 09:58:23.126435 13773 raft_consensus.cc:348] T caa7f8c507de4d1b9bcc78a39e8482f9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:23.126726 13481 tablet_service.cc:1467] Processing CreateTablet for tablet dedf3e0a273d43ba84a861b36d0707ef (DEFAULT_TABLE table=default.table_f83185097e7343848b564d3d5881c9ab [id=7f3f3dd4c3d74f21878caad38c9c25b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:23.127074 13773 raft_consensus.cc:374] T caa7f8c507de4d1b9bcc78a39e8482f9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:23.127300 13773 raft_consensus.cc:729] T caa7f8c507de4d1b9bcc78a39e8482f9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:23.127781 13481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dedf3e0a273d43ba84a861b36d0707ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:23.128171 13773 consensus_queue.cc:260] T caa7f8c507de4d1b9bcc78a39e8482f9 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:23.129947 13773 ts_tablet_manager.cc:1428] T caa7f8c507de4d1b9bcc78a39e8482f9 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260925 09:58:23.133134 13603 leader_election.cc:302] T caa7f8c507de4d1b9bcc78a39e8482f9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, b3082cf15dc74c0480e5d33ecf592843; no voters: c4d724d4b83e4d1e817a3a505dc24eb9
W20260925 09:58:23.133823 13066 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 3735920fa0a64b588480af485d0fd6f0 (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 3735920fa0a64b588480af485d0fd6f0 consensus state has no leader
I20260925 09:58:23.134246 14018 raft_consensus.cc:2781] T caa7f8c507de4d1b9bcc78a39e8482f9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:23.145682 13650 tablet_service.cc:1467] Processing CreateTablet for tablet dedf3e0a273d43ba84a861b36d0707ef (DEFAULT_TABLE table=default.table_f83185097e7343848b564d3d5881c9ab [id=7f3f3dd4c3d74f21878caad38c9c25b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:23.146689 13650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dedf3e0a273d43ba84a861b36d0707ef. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:58:23.157135 13066 catalog_manager.cc:4914] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): alter failed for tablet a9d515adc84446e6b46b0332d3ea10db (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]),no further retry: Not found: Tablet not found: a9d515adc84446e6b46b0332d3ea10db
I20260925 09:58:23.156781 13075 catalog_manager.cc:5495] T b7a703a4d9e04401b13610380acd2ac8 P c4d724d4b83e4d1e817a3a505dc24eb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2). New cstate: current_term: 1 leader_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: UNKNOWN } } }
I20260925 09:58:23.158262 13075 catalog_manager.cc:5587] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327) does not have the latest schema for tablet b7a703a4d9e04401b13610380acd2ac8 (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]). Expected version 1 got 0
I20260925 09:58:23.160233 13342 tablet_service.cc:1467] Processing CreateTablet for tablet dedf3e0a273d43ba84a861b36d0707ef (DEFAULT_TABLE table=default.table_f83185097e7343848b564d3d5881c9ab [id=7f3f3dd4c3d74f21878caad38c9c25b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:23.167565 13342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dedf3e0a273d43ba84a861b36d0707ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:23.170344 13781 ts_tablet_manager.cc:1905] T e75bfd761e314d60b6638eb5174b0227 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:23.174320 13074 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50556:
table { table_name: "default.Table_81574dad60e9490cb79e268b12969a83" } modify_external_catalogs: true
I20260925 09:58:23.175069 13074 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50556:
table { table_name: "default.Table_81574dad60e9490cb79e268b12969a83" } modify_external_catalogs: true
I20260925 09:58:23.182905 13669 raft_consensus.cc:1261] T 963c1e91c6224e5dae4ecc9a6f48f1ac P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Refusing update from remote peer c4d724d4b83e4d1e817a3a505dc24eb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:58:23.184401 14124 consensus_queue.cc:1035] T 963c1e91c6224e5dae4ecc9a6f48f1ac P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:58:23.191282 14018 raft_consensus.cc:686] T caa7f8c507de4d1b9bcc78a39e8482f9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 LEADER]: Becoming Leader. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Running, Role: LEADER
I20260925 09:58:23.192071 14018 consensus_queue.cc:237] T caa7f8c507de4d1b9bcc78a39e8482f9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:23.196079 13671 raft_consensus.cc:1261] T b7a703a4d9e04401b13610380acd2ac8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Refusing update from remote peer c4d724d4b83e4d1e817a3a505dc24eb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:58:23.196244 13363 raft_consensus.cc:1261] T 963c1e91c6224e5dae4ecc9a6f48f1ac P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Refusing update from remote peer c4d724d4b83e4d1e817a3a505dc24eb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:58:23.197575 14124 consensus_queue.cc:1035] T b7a703a4d9e04401b13610380acd2ac8 P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:23.201272 14124 consensus_queue.cc:1035] T 963c1e91c6224e5dae4ecc9a6f48f1ac P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:23.202612 13485 tablet_service.cc:1467] Processing CreateTablet for tablet 368f01226d6a4974acdd79006703a54c (DEFAULT_TABLE table=default.table_2c3c283350d644f39f1903713ab06589 [id=fec42fd086714e079c23b924ca453b65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:23.208142 13409 heartbeater.cc:347] Connected to a master server at 127.10.9.60:42083
I20260925 09:58:23.214891 13363 raft_consensus.cc:1261] T b7a703a4d9e04401b13610380acd2ac8 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Refusing update from remote peer c4d724d4b83e4d1e817a3a505dc24eb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:58:23.212636 14098 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 4 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:23.219924 13078 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:50556:
table { table_name: "default.Table_4d3f82d91a5749679343f60e3dbbc422" } new_table_name: "default.Table_1310f782dc864b068b28842f01c96b9c" modify_external_catalogs: true
I20260925 09:58:23.223768 13649 tablet_service.cc:1467] Processing CreateTablet for tablet 368f01226d6a4974acdd79006703a54c (DEFAULT_TABLE table=default.table_2c3c283350d644f39f1903713ab06589 [id=fec42fd086714e079c23b924ca453b65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:23.224706 13649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 368f01226d6a4974acdd79006703a54c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:23.231309 14098 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61e29afab0cf46819cbffc91248bf761, State: Initialized, Role: FOLLOWER
I20260925 09:58:23.233053 14098 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:23.236202 13780 ts_tablet_manager.cc:1918] T e75bfd761e314d60b6638eb5174b0227 P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:58:23.236589 13780 log.cc:1196] T e75bfd761e314d60b6638eb5174b0227 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/e75bfd761e314d60b6638eb5174b0227
I20260925 09:58:23.237517 13780 ts_tablet_manager.cc:1939] T e75bfd761e314d60b6638eb5174b0227 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:23.167562 13777 ts_tablet_manager.cc:1918] T e3027f1b971f483d91c9046163f62abe P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:23.240974 13777 log.cc:1196] T e3027f1b971f483d91c9046163f62abe P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/e3027f1b971f483d91c9046163f62abe
I20260925 09:58:23.241668 13777 ts_tablet_manager.cc:1939] T e3027f1b971f483d91c9046163f62abe P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:23.207481 13484 tablet_service.cc:1514] Processing DeleteTablet for tablet 6edbb7ba7c86437c84adcde397b76222 with delete_type TABLET_DATA_DELETED (replaced by tablet caa7f8c507de4d1b9bcc78a39e8482f9) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:23.210992 13773 tablet_bootstrap.cc:492] T dedf3e0a273d43ba84a861b36d0707ef P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:23.243134 13485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 368f01226d6a4974acdd79006703a54c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:23.240674 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet e75bfd761e314d60b6638eb5174b0227 (table default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]) successfully deleted
W20260925 09:58:23.244589 13068 catalog_manager.cc:4808] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): delete failed for tablet e3027f1b971f483d91c9046163f62abe because the tablet was not found. No further retry: Not found: Tablet not found: e3027f1b971f483d91c9046163f62abe
I20260925 09:58:23.248311 14150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 4 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } } }
I20260925 09:58:23.259379 14150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:58:23.262372 14098 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 09:58:23.241730 13971 raft_consensus.cc:482] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:23.270730 13971 raft_consensus.cc:504] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:23.271492 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet e3027f1b971f483d91c9046163f62abe (table default.table_fb4b236f93c04bb08319a6915fa8796c [id=2f774783b25a4b0e88449cd7cedaed2e]) successfully deleted
I20260925 09:58:23.272457 13777 tablet_replica.cc:330] T e75bfd761e314d60b6638eb5174b0227 P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:23.272591 13971 leader_election.cc:288] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:23.260300 13341 tablet_service.cc:1467] Processing CreateTablet for tablet 368f01226d6a4974acdd79006703a54c (DEFAULT_TABLE table=default.table_2c3c283350d644f39f1903713ab06589 [id=fec42fd086714e079c23b924ca453b65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:23.273314 13777 raft_consensus.cc:2229] T e75bfd761e314d60b6638eb5174b0227 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:23.273581 13341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 368f01226d6a4974acdd79006703a54c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:23.273769 13777 raft_consensus.cc:2258] T e75bfd761e314d60b6638eb5174b0227 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:23.246783 13773 tablet_bootstrap.cc:654] T dedf3e0a273d43ba84a861b36d0707ef P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
W20260925 09:58:23.277658 13297 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
I20260925 09:58:23.240252 14124 consensus_queue.cc:1035] T b7a703a4d9e04401b13610380acd2ac8 P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:23.246990 13781 ts_tablet_manager.cc:1918] T e75bfd761e314d60b6638eb5174b0227 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:58:23.278589 13781 log.cc:1196] T e75bfd761e314d60b6638eb5174b0227 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/e75bfd761e314d60b6638eb5174b0227
I20260925 09:58:23.279278 13781 ts_tablet_manager.cc:1939] T e75bfd761e314d60b6638eb5174b0227 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
W20260925 09:58:23.280236 13298 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
I20260925 09:58:23.280577 13298 leader_election.cc:302] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:58:23.281663 13781 tablet_replica.cc:330] T 6edbb7ba7c86437c84adcde397b76222 P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:23.282298 13781 raft_consensus.cc:2229] T 6edbb7ba7c86437c84adcde397b76222 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:23.282701 13781 raft_consensus.cc:2258] T 6edbb7ba7c86437c84adcde397b76222 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:23.283535 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet e75bfd761e314d60b6638eb5174b0227 (table default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]) successfully deleted
I20260925 09:58:23.286676 13971 raft_consensus.cc:2726] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:58:23.287242 13781 ts_tablet_manager.cc:1905] T 6edbb7ba7c86437c84adcde397b76222 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:23.298938 13773 tablet_bootstrap.cc:492] T dedf3e0a273d43ba84a861b36d0707ef P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:23.299333 13773 ts_tablet_manager.cc:1397] T dedf3e0a273d43ba84a861b36d0707ef P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.089s user 0.012s sys 0.001s
I20260925 09:58:23.301504 13773 raft_consensus.cc:348] T dedf3e0a273d43ba84a861b36d0707ef P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:23.302158 13773 raft_consensus.cc:374] T dedf3e0a273d43ba84a861b36d0707ef P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:23.302430 13773 raft_consensus.cc:729] T dedf3e0a273d43ba84a861b36d0707ef P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:23.303047 13773 consensus_queue.cc:260] T dedf3e0a273d43ba84a861b36d0707ef P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:23.314620 13770 tablet_bootstrap.cc:492] T 368f01226d6a4974acdd79006703a54c P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:23.317996 13773 ts_tablet_manager.cc:1428] T dedf3e0a273d43ba84a861b36d0707ef P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.018s user 0.005s sys 0.000s
I20260925 09:58:23.322849 13777 ts_tablet_manager.cc:1905] T e75bfd761e314d60b6638eb5174b0227 P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:23.339852 14038 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 4 FOLLOWER]: Deduplicated request from leader. Original: 2.5->[4.6-4.129] Dedup: 4.6->[4.7-4.129]
I20260925 09:58:23.348579 14155 tablet.cc:1722] T 963c1e91c6224e5dae4ecc9a6f48f1ac P c4d724d4b83e4d1e817a3a505dc24eb9: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:58:23.350878 13770 tablet_bootstrap.cc:654] T 368f01226d6a4974acdd79006703a54c P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:23.357155 14132 mvcc.cc:205] Tried to move back new op lower bound from 7333192921753554944 to 7333192920513626112. Current Snapshot: MvccSnapshot[applied={T|T < 7333192921753554944}]
I20260925 09:58:23.357761 13075 catalog_manager.cc:5587] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327) does not have the latest schema for tablet 963c1e91c6224e5dae4ecc9a6f48f1ac (table default.table_81574dad60e9490cb79e268b12969a83 [id=8fa5a3419e394be2a52d1b13713a04d9]). Expected version 1 got 0
I20260925 09:58:23.378707 13772 tablet_bootstrap.cc:492] T dedf3e0a273d43ba84a861b36d0707ef P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:23.378685 13073 catalog_manager.cc:5495] T caa7f8c507de4d1b9bcc78a39e8482f9 P a06c3ef7a6a94f7cbcc03e7cd232f237 reported cstate change: term changed from 0 to 1, leader changed from <none> to a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3). New cstate: current_term: 1 leader_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: HEALTHY } } }
I20260925 09:58:23.383639 13772 tablet_bootstrap.cc:654] T dedf3e0a273d43ba84a861b36d0707ef P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:23.394474 14165 tablet.cc:1722] T 963c1e91c6224e5dae4ecc9a6f48f1ac P a06c3ef7a6a94f7cbcc03e7cd232f237: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:58:23.405328 13773 tablet_bootstrap.cc:492] T 368f01226d6a4974acdd79006703a54c P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:23.422501 13773 tablet_bootstrap.cc:654] T 368f01226d6a4974acdd79006703a54c P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:23.426249 13777 ts_tablet_manager.cc:1918] T e75bfd761e314d60b6638eb5174b0227 P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:58:23.426777 13777 log.cc:1196] T e75bfd761e314d60b6638eb5174b0227 P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/e75bfd761e314d60b6638eb5174b0227
I20260925 09:58:23.429345 14170 tablet.cc:1722] T b7a703a4d9e04401b13610380acd2ac8 P c4d724d4b83e4d1e817a3a505dc24eb9: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:58:23.431841 14171 tablet.cc:1722] T 963c1e91c6224e5dae4ecc9a6f48f1ac P b3082cf15dc74c0480e5d33ecf592843: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:58:23.437487 13550 heartbeater.cc:347] Connected to a master server at 127.10.9.60:42083
I20260925 09:58:23.462005 13075 master_service.cc:946] ReplaceTablet: received request to replace tablet e75bfd761e314d60b6638eb5174b0227 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:23.465935 13075 catalog_manager.cc:5587] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161) does not have the latest schema for tablet 963c1e91c6224e5dae4ecc9a6f48f1ac (table default.table_81574dad60e9490cb79e268b12969a83 [id=8fa5a3419e394be2a52d1b13713a04d9]). Expected version 1 got 0
I20260925 09:58:23.437911 13777 ts_tablet_manager.cc:1939] T e75bfd761e314d60b6638eb5174b0227 P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:23.471644 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet e75bfd761e314d60b6638eb5174b0227 (table default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]) successfully deleted
I20260925 09:58:23.472640 13773 tablet_bootstrap.cc:492] T 368f01226d6a4974acdd79006703a54c P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:23.473132 13773 ts_tablet_manager.cc:1397] T 368f01226d6a4974acdd79006703a54c P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.068s user 0.008s sys 0.004s
I20260925 09:58:23.477161 14162 tablet.cc:1722] T b7a703a4d9e04401b13610380acd2ac8 P b3082cf15dc74c0480e5d33ecf592843: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:58:23.478236 13773 raft_consensus.cc:348] T 368f01226d6a4974acdd79006703a54c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:23.478832 13773 raft_consensus.cc:374] T 368f01226d6a4974acdd79006703a54c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:23.479102 13773 raft_consensus.cc:729] T 368f01226d6a4974acdd79006703a54c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:23.486793 13772 tablet_bootstrap.cc:492] T dedf3e0a273d43ba84a861b36d0707ef P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:23.487179 13772 ts_tablet_manager.cc:1397] T dedf3e0a273d43ba84a861b36d0707ef P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.109s user 0.011s sys 0.000s
I20260925 09:58:23.489621 13772 raft_consensus.cc:348] T dedf3e0a273d43ba84a861b36d0707ef P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:23.490214 13772 raft_consensus.cc:374] T dedf3e0a273d43ba84a861b36d0707ef P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:23.490451 13772 raft_consensus.cc:729] T dedf3e0a273d43ba84a861b36d0707ef P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:23.491092 13772 consensus_queue.cc:260] T dedf3e0a273d43ba84a861b36d0707ef P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:23.493176 13073 master_service.cc:946] ReplaceTablet: received request to replace tablet 963c1e91c6224e5dae4ecc9a6f48f1ac from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:23.497391 13772 ts_tablet_manager.cc:1428] T dedf3e0a273d43ba84a861b36d0707ef P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.010s user 0.004s sys 0.000s
I20260925 09:58:23.507218 13772 tablet_bootstrap.cc:492] T 368f01226d6a4974acdd79006703a54c P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:23.512580 13073 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet 963c1e91c6224e5dae4ecc9a6f48f1ac
I20260925 09:58:23.514442 13485 tablet_service.cc:1514] Processing DeleteTablet for tablet 963c1e91c6224e5dae4ecc9a6f48f1ac with delete_type TABLET_DATA_DELETED (replaced by tablet 0749453435d54676803789c380179af4) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:23.515036 13341 tablet_service.cc:1514] Processing DeleteTablet for tablet 963c1e91c6224e5dae4ecc9a6f48f1ac with delete_type TABLET_DATA_DELETED (replaced by tablet 0749453435d54676803789c380179af4) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:23.515679 13777 tablet_replica.cc:330] T 963c1e91c6224e5dae4ecc9a6f48f1ac P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:23.516283 13777 raft_consensus.cc:2229] T 963c1e91c6224e5dae4ecc9a6f48f1ac P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:23.516710 13777 raft_consensus.cc:2258] T 963c1e91c6224e5dae4ecc9a6f48f1ac P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:23.507901 13773 consensus_queue.cc:260] T 368f01226d6a4974acdd79006703a54c P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:23.519627 13773 ts_tablet_manager.cc:1428] T 368f01226d6a4974acdd79006703a54c P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.046s user 0.003s sys 0.004s
I20260925 09:58:23.533367 13971 raft_consensus.cc:482] T dedf3e0a273d43ba84a861b36d0707ef P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:23.533849 13971 raft_consensus.cc:504] T dedf3e0a273d43ba84a861b36d0707ef P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:23.535770 13971 leader_election.cc:288] T dedf3e0a273d43ba84a861b36d0707ef P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:23.537257 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dedf3e0a273d43ba84a861b36d0707ef" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
I20260925 09:58:23.540297 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dedf3e0a273d43ba84a861b36d0707ef" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:23.544950 13073 catalog_manager.cc:6565] ReplaceTablet: tablet 963c1e91c6224e5dae4ecc9a6f48f1ac deleted and replaced by tablet 0749453435d54676803789c380179af4
I20260925 09:58:23.544849 14125 consensus_queue.cc:1035] T 8115a86775884efb8b9f91646f8abc48 P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:58:23.545737 13650 tablet_service.cc:1514] Processing DeleteTablet for tablet 963c1e91c6224e5dae4ecc9a6f48f1ac with delete_type TABLET_DATA_DELETED (replaced by tablet 0749453435d54676803789c380179af4) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:23.546452 13780 tablet_replica.cc:330] T 963c1e91c6224e5dae4ecc9a6f48f1ac P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
W20260925 09:58:23.546712 13297 leader_election.cc:341] T dedf3e0a273d43ba84a861b36d0707ef P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:23.547122 13780 raft_consensus.cc:2229] T 963c1e91c6224e5dae4ecc9a6f48f1ac P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:23.547606 13780 raft_consensus.cc:2258] T 963c1e91c6224e5dae4ecc9a6f48f1ac P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:23.557207 13772 tablet_bootstrap.cc:654] T 368f01226d6a4974acdd79006703a54c P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:23.559118 13077 catalog_manager.cc:3411] renamed table default.table_eadc20bcf90e4a72b8f26e3a4798e259 [id=f196d9d3750b4857905c1332d3ac763e] in HMS: new name default.table_d4409ab5331140e9a4bb995d57be4ec4
I20260925 09:58:23.570766 14124 consensus_queue.cc:1035] T 8115a86775884efb8b9f91646f8abc48 P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:23.588694 13770 tablet_bootstrap.cc:492] T 368f01226d6a4974acdd79006703a54c P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:23.589180 13770 ts_tablet_manager.cc:1397] T 368f01226d6a4974acdd79006703a54c P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.275s user 0.012s sys 0.008s
I20260925 09:58:23.591331 13770 raft_consensus.cc:348] T 368f01226d6a4974acdd79006703a54c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:23.592072 13770 raft_consensus.cc:374] T 368f01226d6a4974acdd79006703a54c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:23.592368 13770 raft_consensus.cc:729] T 368f01226d6a4974acdd79006703a54c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:23.593096 13770 consensus_queue.cc:260] T 368f01226d6a4974acdd79006703a54c P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:23.595412 13770 ts_tablet_manager.cc:1428] T 368f01226d6a4974acdd79006703a54c P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260925 09:58:23.593117 14177 tablet.cc:1722] T b7a703a4d9e04401b13610380acd2ac8 P a06c3ef7a6a94f7cbcc03e7cd232f237: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:58:23.599812 13505 raft_consensus.cc:2454] T dedf3e0a273d43ba84a861b36d0707ef P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 0.
I20260925 09:58:23.601532 13298 leader_election.cc:302] T dedf3e0a273d43ba84a861b36d0707ef P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237
I20260925 09:58:23.602566 13971 raft_consensus.cc:2781] T dedf3e0a273d43ba84a861b36d0707ef P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:23.602870 13971 raft_consensus.cc:482] T dedf3e0a273d43ba84a861b36d0707ef P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:23.603159 13971 raft_consensus.cc:3037] T dedf3e0a273d43ba84a861b36d0707ef P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:23.467229 13781 ts_tablet_manager.cc:1918] T 6edbb7ba7c86437c84adcde397b76222 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:23.606582 13781 log.cc:1196] T 6edbb7ba7c86437c84adcde397b76222 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/6edbb7ba7c86437c84adcde397b76222
I20260925 09:58:23.608647 13075 catalog_manager.cc:6040] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7] alter complete (version 1)
I20260925 09:58:23.614485 13971 raft_consensus.cc:504] T dedf3e0a273d43ba84a861b36d0707ef P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:23.617142 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dedf3e0a273d43ba84a861b36d0707ef" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9"
I20260925 09:58:23.617261 13073 catalog_manager.cc:5587] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731) does not have the latest schema for tablet b7a703a4d9e04401b13610380acd2ac8 (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]). Expected version 1 got 0
I20260925 09:58:23.617672 13505 raft_consensus.cc:3037] T dedf3e0a273d43ba84a861b36d0707ef P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:23.625041 13505 raft_consensus.cc:2454] T dedf3e0a273d43ba84a861b36d0707ef P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 1.
I20260925 09:58:23.626055 13298 leader_election.cc:302] T dedf3e0a273d43ba84a861b36d0707ef P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:23.631570 13080 catalog_manager.cc:2148] created HMS catalog entry for table [id=77d343c035fa4fd2b3faadcfc4fec52f]
I20260925 09:58:23.631660 13669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dedf3e0a273d43ba84a861b36d0707ef" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
I20260925 09:58:23.631877 13971 leader_election.cc:288] T dedf3e0a273d43ba84a861b36d0707ef P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Requested vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:23.632375 13971 raft_consensus.cc:2781] T dedf3e0a273d43ba84a861b36d0707ef P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:23.632764 13971 raft_consensus.cc:686] T dedf3e0a273d43ba84a861b36d0707ef P b3082cf15dc74c0480e5d33ecf592843 [term 1 LEADER]: Becoming Leader. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Running, Role: LEADER
W20260925 09:58:23.633013 13297 leader_election.cc:341] T dedf3e0a273d43ba84a861b36d0707ef P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:23.633505 13971 consensus_queue.cc:237] T dedf3e0a273d43ba84a861b36d0707ef P b3082cf15dc74c0480e5d33ecf592843 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:23.650638 13080 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:50556 (request call id 73) took 2496 ms. Trace:
I20260925 09:58:23.650978 13080 rpcz_store.cc:276] 0925 09:58:21.154087 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:21.154270 (+ 183us) service_pool.cc:227] Handling call
0925 09:58:22.967087 (+1812817us) catalog_manager.cc:2075] Acquired catalog manager lock
0925 09:58:22.968893 (+ 1806us) catalog_manager.cc:2121] Created new table and tablet info
0925 09:58:23.631529 (+662636us) catalog_manager.cc:2147] Created new table in HMS catalog
0925 09:58:23.649935 (+ 18406us) catalog_manager.cc:2181] Wrote table and tablets to system table
0925 09:58:23.650291 (+ 356us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0925 09:58:23.650575 (+ 284us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:58:23.633238 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:58:23.633592 (+ 354us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:58:23.633616 (+ 24us) write_op.cc:435] Acquired schema lock
0925 09:58:23.633628 (+ 12us) tablet.cc:660] Decoding operations
0925 09:58:23.633901 (+ 273us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:58:23.633938 (+ 37us) write_op.cc:641] Partition lock acquired for write op
0925 09:58:23.633958 (+ 20us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:58:23.634157 (+ 199us) tablet.cc:699] Row locks acquired
0925 09:58:23.634170 (+ 13us) write_op.cc:260] PREPARE: finished
0925 09:58:23.634285 (+ 115us) write_op.cc:270] Start()
0925 09:58:23.634372 (+ 87us) write_op.cc:276] Timestamp: P: 1790330303634221 usec, L: 0
0925 09:58:23.634391 (+ 19us) op_driver.cc:331] REPLICATION: starting
0925 09:58:23.634927 (+ 536us) log.cc:842] Serialized 433 byte log entry
0925 09:58:23.641880 (+ 6953us) op_driver.cc:444] REPLICATION: finished
0925 09:58:23.648136 (+ 6256us) write_op.cc:301] APPLY: starting
0925 09:58:23.648188 (+ 52us) tablet.cc:1365] starting BulkCheckPresence
0925 09:58:23.648286 (+ 98us) tablet.cc:1368] finished BulkCheckPresence
0925 09:58:23.648369 (+ 83us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:58:23.648707 (+ 338us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:58:23.648837 (+ 130us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 09:58:23.648876 (+ 39us) write_op.cc:312] APPLY: finished
0925 09:58:23.649053 (+ 177us) log.cc:842] Serialized 29 byte log entry
0925 09:58:23.649624 (+ 571us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:23.649700 (+ 76us) write_op.cc:454] Released schema lock
0925 09:58:23.649738 (+ 38us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"Hive Metastore.queue_time_us":588081,"Hive Metastore.run_cpu_time_us":613,"Hive Metastore.run_wall_time_us":73612,"child_traces":[["op",{"apply.queue_time_us":6089,"num_ops":2,"prepare.queue_time_us":131,"prepare.run_cpu_time_us":1973,"prepare.run_wall_time_us":1973,"raft.queue_time_us":172,"raft.run_cpu_time_us":604,"raft.run_wall_time_us":604,"replication_time_us":7409}]]}
I20260925 09:58:23.656630 13770 tablet_bootstrap.cc:492] T dedf3e0a273d43ba84a861b36d0707ef P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:23.666005 13770 tablet_bootstrap.cc:654] T dedf3e0a273d43ba84a861b36d0707ef P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:23.669566 13772 tablet_bootstrap.cc:492] T 368f01226d6a4974acdd79006703a54c P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:23.669904 13777 ts_tablet_manager.cc:1905] T 963c1e91c6224e5dae4ecc9a6f48f1ac P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:23.670003 13772 ts_tablet_manager.cc:1397] T 368f01226d6a4974acdd79006703a54c P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.163s user 0.010s sys 0.006s
I20260925 09:58:23.672348 13772 raft_consensus.cc:348] T 368f01226d6a4974acdd79006703a54c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:23.673040 13772 raft_consensus.cc:374] T 368f01226d6a4974acdd79006703a54c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:23.673264 13772 raft_consensus.cc:729] T 368f01226d6a4974acdd79006703a54c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:23.607193 13781 ts_tablet_manager.cc:1939] T 6edbb7ba7c86437c84adcde397b76222 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:23.673859 13772 consensus_queue.cc:260] T 368f01226d6a4974acdd79006703a54c P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:23.675801 13772 ts_tablet_manager.cc:1428] T 368f01226d6a4974acdd79006703a54c P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260925 09:58:23.677043 13781 tablet_replica.cc:330] T 963c1e91c6224e5dae4ecc9a6f48f1ac P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:23.678409 13781 raft_consensus.cc:2229] T 963c1e91c6224e5dae4ecc9a6f48f1ac P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:58:23.676321 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet 6edbb7ba7c86437c84adcde397b76222 (table default.table_8f71132b5a004ad89115f7a2278b95d6 [id=0cdc8d8fef5445668c0b0107243e0ae9]) successfully deleted
I20260925 09:58:23.696249 13780 ts_tablet_manager.cc:1905] T 963c1e91c6224e5dae4ecc9a6f48f1ac P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:23.698357 13716 heartbeater.cc:347] Connected to a master server at 127.10.9.60:42083
I20260925 09:58:23.705435 13781 raft_consensus.cc:2258] T 963c1e91c6224e5dae4ecc9a6f48f1ac P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:23.718230 13485 tablet_service.cc:1467] Processing CreateTablet for tablet 0749453435d54676803789c380179af4 (DEFAULT_TABLE table=default.table_81574dad60e9490cb79e268b12969a83 [id=8fa5a3419e394be2a52d1b13713a04d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:23.719214 13485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0749453435d54676803789c380179af4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:23.724006 13341 tablet_service.cc:1467] Processing CreateTablet for tablet 0749453435d54676803789c380179af4 (DEFAULT_TABLE table=default.table_81574dad60e9490cb79e268b12969a83 [id=8fa5a3419e394be2a52d1b13713a04d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:23.724318 13780 ts_tablet_manager.cc:1918] T 963c1e91c6224e5dae4ecc9a6f48f1ac P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:58:23.724648 13780 log.cc:1196] T 963c1e91c6224e5dae4ecc9a6f48f1ac P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/963c1e91c6224e5dae4ecc9a6f48f1ac
I20260925 09:58:23.724923 13341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0749453435d54676803789c380179af4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:23.725286 13780 ts_tablet_manager.cc:1939] T 963c1e91c6224e5dae4ecc9a6f48f1ac P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:23.726814 13777 ts_tablet_manager.cc:1918] T 963c1e91c6224e5dae4ecc9a6f48f1ac P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:58:23.727133 13777 log.cc:1196] T 963c1e91c6224e5dae4ecc9a6f48f1ac P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/963c1e91c6224e5dae4ecc9a6f48f1ac
I20260925 09:58:23.727825 13777 ts_tablet_manager.cc:1939] T 963c1e91c6224e5dae4ecc9a6f48f1ac P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:23.733222 13080 catalog_manager.cc:5495] T dedf3e0a273d43ba84a861b36d0707ef P b3082cf15dc74c0480e5d33ecf592843 reported cstate change: term changed from 0 to 1, leader changed from <none> to b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b3082cf15dc74c0480e5d33ecf592843" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: HEALTHY } } }
I20260925 09:58:23.744998 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet 963c1e91c6224e5dae4ecc9a6f48f1ac (table default.table_81574dad60e9490cb79e268b12969a83 [id=8fa5a3419e394be2a52d1b13713a04d9]) successfully deleted
I20260925 09:58:23.753446 13073 master_service.cc:946] ReplaceTablet: received request to replace tablet 963c1e91c6224e5dae4ecc9a6f48f1ac from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:23.757289 13080 master_service.cc:946] ReplaceTablet: received request to replace tablet b7a703a4d9e04401b13610380acd2ac8 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:23.762606 13650 tablet_service.cc:1467] Processing CreateTablet for tablet 0749453435d54676803789c380179af4 (DEFAULT_TABLE table=default.table_81574dad60e9490cb79e268b12969a83 [id=8fa5a3419e394be2a52d1b13713a04d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:23.764199 13342 tablet_service.cc:1467] Processing CreateTablet for tablet cedc5b0609e14c409542a83efc45f3f0 (DEFAULT_TABLE table=default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:23.765069 13342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cedc5b0609e14c409542a83efc45f3f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:23.772495 13770 tablet_bootstrap.cc:492] T dedf3e0a273d43ba84a861b36d0707ef P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:23.772917 13770 ts_tablet_manager.cc:1397] T dedf3e0a273d43ba84a861b36d0707ef P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.160s user 0.007s sys 0.003s
I20260925 09:58:23.780612 13781 ts_tablet_manager.cc:1905] T 963c1e91c6224e5dae4ecc9a6f48f1ac P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:23.785880 13081 catalog_manager.cc:2148] created HMS catalog entry for table [id=db33e4ea5b1d4f0abedf9b9dde8614e2]
I20260925 09:58:23.795799 13650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0749453435d54676803789c380179af4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:23.800326 13484 tablet_service.cc:1467] Processing CreateTablet for tablet cedc5b0609e14c409542a83efc45f3f0 (DEFAULT_TABLE table=default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:23.801280 13484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cedc5b0609e14c409542a83efc45f3f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:23.802523 13772 tablet_bootstrap.cc:492] T cedc5b0609e14c409542a83efc45f3f0 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:23.803586 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet 963c1e91c6224e5dae4ecc9a6f48f1ac (table default.table_81574dad60e9490cb79e268b12969a83 [id=8fa5a3419e394be2a52d1b13713a04d9]) successfully deleted
I20260925 09:58:23.807710 13772 tablet_bootstrap.cc:654] T cedc5b0609e14c409542a83efc45f3f0 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:23.819316 13649 tablet_service.cc:1514] Processing DeleteTablet for tablet 963c1e91c6224e5dae4ecc9a6f48f1ac with delete_type TABLET_DATA_DELETED (replaced by tablet 0749453435d54676803789c380179af4) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:23.819423 13481 tablet_service.cc:1514] Processing DeleteTablet for tablet b7a703a4d9e04401b13610380acd2ac8 with delete_type TABLET_DATA_DELETED (replaced by tablet a3562966c8c34046af271a7c617cdb6b) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:23.822623 13651 tablet_service.cc:1514] Processing DeleteTablet for tablet b7a703a4d9e04401b13610380acd2ac8 with delete_type TABLET_DATA_DELETED (replaced by tablet a3562966c8c34046af271a7c617cdb6b) from {username='slave'} at 127.0.0.1:54548
W20260925 09:58:23.823830 13068 catalog_manager.cc:4808] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): delete failed for tablet 963c1e91c6224e5dae4ecc9a6f48f1ac because the tablet was not found. No further retry: Not found: Tablet not found: 963c1e91c6224e5dae4ecc9a6f48f1ac
I20260925 09:58:23.824327 13780 tablet_replica.cc:330] T b7a703a4d9e04401b13610380acd2ac8 P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:23.824966 13780 raft_consensus.cc:2229] T b7a703a4d9e04401b13610380acd2ac8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:23.825402 13780 raft_consensus.cc:2258] T b7a703a4d9e04401b13610380acd2ac8 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:23.825536 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet b7a703a4d9e04401b13610380acd2ac8 with delete_type TABLET_DATA_DELETED (replaced by tablet a3562966c8c34046af271a7c617cdb6b) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:23.826745 13777 tablet_replica.cc:330] T b7a703a4d9e04401b13610380acd2ac8 P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:23.827687 13777 raft_consensus.cc:2229] T b7a703a4d9e04401b13610380acd2ac8 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:23.828276 13777 raft_consensus.cc:2258] T b7a703a4d9e04401b13610380acd2ac8 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:23.830760 13777 ts_tablet_manager.cc:1905] T b7a703a4d9e04401b13610380acd2ac8 P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:23.835083 13081 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:50556 (request call id 78) took 1879 ms. Trace:
I20260925 09:58:23.835441 13081 rpcz_store.cc:276] 0925 09:58:21.955501 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:21.955734 (+ 233us) service_pool.cc:227] Handling call
0925 09:58:22.976140 (+1020406us) catalog_manager.cc:2075] Acquired catalog manager lock
0925 09:58:22.977868 (+ 1728us) catalog_manager.cc:2121] Created new table and tablet info
0925 09:58:23.785836 (+807968us) catalog_manager.cc:2147] Created new table in HMS catalog
0925 09:58:23.834417 (+ 48581us) catalog_manager.cc:2181] Wrote table and tablets to system table
0925 09:58:23.834754 (+ 337us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0925 09:58:23.835024 (+ 270us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:58:23.787622 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:58:23.788110 (+ 488us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:58:23.788132 (+ 22us) write_op.cc:435] Acquired schema lock
0925 09:58:23.788143 (+ 11us) tablet.cc:660] Decoding operations
0925 09:58:23.788467 (+ 324us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:58:23.788504 (+ 37us) write_op.cc:641] Partition lock acquired for write op
0925 09:58:23.788529 (+ 25us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:58:23.788717 (+ 188us) tablet.cc:699] Row locks acquired
0925 09:58:23.788730 (+ 13us) write_op.cc:260] PREPARE: finished
0925 09:58:23.788834 (+ 104us) write_op.cc:270] Start()
0925 09:58:23.788919 (+ 85us) write_op.cc:276] Timestamp: P: 1790330303788765 usec, L: 0
0925 09:58:23.788938 (+ 19us) op_driver.cc:331] REPLICATION: starting
0925 09:58:23.799719 (+ 10781us) log.cc:842] Serialized 433 byte log entry
0925 09:58:23.832640 (+ 32921us) op_driver.cc:444] REPLICATION: finished
0925 09:58:23.832950 (+ 310us) write_op.cc:301] APPLY: starting
0925 09:58:23.832996 (+ 46us) tablet.cc:1365] starting BulkCheckPresence
0925 09:58:23.833080 (+ 84us) tablet.cc:1368] finished BulkCheckPresence
0925 09:58:23.833154 (+ 74us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:58:23.833678 (+ 524us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:58:23.833788 (+ 110us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 09:58:23.833820 (+ 32us) write_op.cc:312] APPLY: finished
0925 09:58:23.834035 (+ 215us) log.cc:842] Serialized 29 byte log entry
0925 09:58:23.834156 (+ 121us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:23.834214 (+ 58us) write_op.cc:454] Released schema lock
0925 09:58:23.834239 (+ 25us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"Hive Metastore.queue_time_us":652826,"Hive Metastore.run_cpu_time_us":652,"Hive Metastore.run_wall_time_us":154528,"child_traces":[["op",{"apply.queue_time_us":169,"num_ops":2,"prepare.queue_time_us":210,"prepare.run_cpu_time_us":1920,"prepare.run_wall_time_us":13250,"raft.queue_time_us":2808,"raft.run_cpu_time_us":588,"raft.run_wall_time_us":588,"replication_time_us":43616}]]}
I20260925 09:58:23.839020 13649 tablet_service.cc:1467] Processing CreateTablet for tablet cedc5b0609e14c409542a83efc45f3f0 (DEFAULT_TABLE table=default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:23.840161 13649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cedc5b0609e14c409542a83efc45f3f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:23.844209 13780 ts_tablet_manager.cc:1905] T b7a703a4d9e04401b13610380acd2ac8 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:23.859025 13073 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50556:
name: "default.Table_afc7cdc28ab84dff9a8782192aa34919"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:58:23.856676 13770 raft_consensus.cc:348] T dedf3e0a273d43ba84a861b36d0707ef P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:23.865638 13080 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet b7a703a4d9e04401b13610380acd2ac8
I20260925 09:58:23.866063 13770 raft_consensus.cc:374] T dedf3e0a273d43ba84a861b36d0707ef P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:23.867468 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet b7a703a4d9e04401b13610380acd2ac8 with delete_type TABLET_DATA_DELETED (replaced by tablet a3562966c8c34046af271a7c617cdb6b) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:23.869565 13080 catalog_manager.cc:6565] ReplaceTablet: tablet b7a703a4d9e04401b13610380acd2ac8 deleted and replaced by tablet a3562966c8c34046af271a7c617cdb6b
I20260925 09:58:23.870469 13481 tablet_service.cc:1514] Processing DeleteTablet for tablet b7a703a4d9e04401b13610380acd2ac8 with delete_type TABLET_DATA_DELETED (replaced by tablet a3562966c8c34046af271a7c617cdb6b) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:23.872921 13777 ts_tablet_manager.cc:1918] T b7a703a4d9e04401b13610380acd2ac8 P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:58:23.873397 13777 log.cc:1196] T b7a703a4d9e04401b13610380acd2ac8 P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/b7a703a4d9e04401b13610380acd2ac8
I20260925 09:58:23.874105 13777 ts_tablet_manager.cc:1939] T b7a703a4d9e04401b13610380acd2ac8 P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:23.876639 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet b7a703a4d9e04401b13610380acd2ac8 (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]) successfully deleted
W20260925 09:58:23.877527 13066 catalog_manager.cc:4808] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): delete failed for tablet b7a703a4d9e04401b13610380acd2ac8 because the tablet was not found. No further retry: Not found: Tablet not found: b7a703a4d9e04401b13610380acd2ac8
I20260925 09:58:23.880756 13773 tablet_bootstrap.cc:492] T 0749453435d54676803789c380179af4 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:23.866693 13780 ts_tablet_manager.cc:1918] T b7a703a4d9e04401b13610380acd2ac8 P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:58:23.871461 13650 tablet_service.cc:1514] Processing DeleteTablet for tablet b7a703a4d9e04401b13610380acd2ac8 with delete_type TABLET_DATA_DELETED (replaced by tablet a3562966c8c34046af271a7c617cdb6b) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:23.883803 13780 log.cc:1196] T b7a703a4d9e04401b13610380acd2ac8 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/b7a703a4d9e04401b13610380acd2ac8
I20260925 09:58:23.884562 13780 ts_tablet_manager.cc:1939] T b7a703a4d9e04401b13610380acd2ac8 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:23.885792 13773 tablet_bootstrap.cc:654] T 0749453435d54676803789c380179af4 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:23.887044 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet b7a703a4d9e04401b13610380acd2ac8 (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]) successfully deleted
I20260925 09:58:23.887334 13770 raft_consensus.cc:729] T dedf3e0a273d43ba84a861b36d0707ef P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
W20260925 09:58:23.888113 13068 catalog_manager.cc:4808] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): delete failed for tablet b7a703a4d9e04401b13610380acd2ac8 because the tablet was not found. No further retry: Not found: Tablet not found: b7a703a4d9e04401b13610380acd2ac8
I20260925 09:58:23.892442 13772 tablet_bootstrap.cc:492] T cedc5b0609e14c409542a83efc45f3f0 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:23.892822 13772 ts_tablet_manager.cc:1397] T cedc5b0609e14c409542a83efc45f3f0 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.090s user 0.011s sys 0.009s
I20260925 09:58:23.894825 13772 raft_consensus.cc:348] T cedc5b0609e14c409542a83efc45f3f0 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:23.895462 13772 raft_consensus.cc:374] T cedc5b0609e14c409542a83efc45f3f0 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:23.895731 13772 raft_consensus.cc:729] T cedc5b0609e14c409542a83efc45f3f0 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:23.896337 13772 consensus_queue.cc:260] T cedc5b0609e14c409542a83efc45f3f0 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:23.904728 13773 tablet_bootstrap.cc:492] T 0749453435d54676803789c380179af4 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:23.905118 13773 ts_tablet_manager.cc:1397] T 0749453435d54676803789c380179af4 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.025s user 0.004s sys 0.006s
I20260925 09:58:23.905825 13770 consensus_queue.cc:260] T dedf3e0a273d43ba84a861b36d0707ef P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:23.910476 13773 raft_consensus.cc:348] T 0749453435d54676803789c380179af4 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:23.911094 13773 raft_consensus.cc:374] T 0749453435d54676803789c380179af4 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:23.911322 13773 raft_consensus.cc:729] T 0749453435d54676803789c380179af4 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:23.912063 13773 consensus_queue.cc:260] T 0749453435d54676803789c380179af4 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:23.919059 14177 tablet.cc:1722] T 3cde6676790b4e3f89f05ce60796b36e P a06c3ef7a6a94f7cbcc03e7cd232f237: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:58:23.924325 13773 ts_tablet_manager.cc:1428] T 0749453435d54676803789c380179af4 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.019s user 0.004s sys 0.001s
I20260925 09:58:23.925127 13773 tablet_bootstrap.cc:492] T cedc5b0609e14c409542a83efc45f3f0 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:23.929175 13770 ts_tablet_manager.cc:1428] T dedf3e0a273d43ba84a861b36d0707ef P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.156s user 0.005s sys 0.001s
I20260925 09:58:23.930903 14162 tablet.cc:1722] T 3cde6676790b4e3f89f05ce60796b36e P b3082cf15dc74c0480e5d33ecf592843: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260925 09:58:23.936561 13079 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_47c0ac37d06f4f45a701b0910a42c579 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260925 09:58:23.940892 13082 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_5f3286f12ae947ef8c1dfa71d8dd21d2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:58:23.943614 13076 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:50556 (request call id 60) took 4546 ms. Trace:
I20260925 09:58:23.943925 13076 rpcz_store.cc:276] 0925 09:58:19.397164 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:19.410449 (+ 13285us) service_pool.cc:227] Handling call
0925 09:58:20.695492 (+1285043us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:58:23.943534 (+3248042us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":1433441,"Hive Metastore.run_cpu_time_us":366,"Hive Metastore.run_wall_time_us":325840}
I20260925 09:58:23.951539 13409 heartbeater.cc:347] Connected to a master server at 127.10.9.60:42083
I20260925 09:58:23.952831 13772 ts_tablet_manager.cc:1428] T cedc5b0609e14c409542a83efc45f3f0 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.060s user 0.006s sys 0.000s
I20260925 09:58:23.959892 13772 tablet_bootstrap.cc:492] T 0749453435d54676803789c380179af4 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:23.970233 13773 tablet_bootstrap.cc:654] T cedc5b0609e14c409542a83efc45f3f0 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:23.973047 13075 catalog_manager.cc:5587] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327) does not have the latest schema for tablet 0749453435d54676803789c380179af4 (table default.table_81574dad60e9490cb79e268b12969a83 [id=8fa5a3419e394be2a52d1b13713a04d9]). Expected version 1 got 0
W20260925 09:58:23.974382 13075 catalog_manager.cc:5687] Failed to send AlterTable RPC for tablet 0749453435d54676803789c380179af4 (table default.table_81574dad60e9490cb79e268b12969a83 [id=8fa5a3419e394be2a52d1b13713a04d9], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 0749453435d54676803789c380179af4 consensus state has no leader
I20260925 09:58:23.975585 13770 tablet_bootstrap.cc:492] T cedc5b0609e14c409542a83efc45f3f0 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:23.976003 13080 catalog_manager.cc:5587] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161) does not have the latest schema for tablet cedc5b0609e14c409542a83efc45f3f0 (table default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]). Expected version 1 got 0
I20260925 09:58:23.976590 13080 catalog_manager.cc:5587] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161) does not have the latest schema for tablet 3cde6676790b4e3f89f05ce60796b36e (table default.table_d4409ab5331140e9a4bb995d57be4ec4 [id=f196d9d3750b4857905c1332d3ac763e]). Expected version 1 got 0
I20260925 09:58:23.979265 14125 raft_consensus.cc:482] T 368f01226d6a4974acdd79006703a54c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:23.979699 14125 raft_consensus.cc:504] T 368f01226d6a4974acdd79006703a54c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:23.981518 14125 leader_election.cc:288] T 368f01226d6a4974acdd79006703a54c P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731), b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161)
I20260925 09:58:23.986300 13341 tablet_service.cc:1467] Processing CreateTablet for tablet d2065bf05eb2489bb09736b2333bf215 (DEFAULT_TABLE table=default.table_41304460d5a345c88866a7429d816aca [id=77d343c035fa4fd2b3faadcfc4fec52f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:23.987286 13341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2065bf05eb2489bb09736b2333bf215. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:23.989481 13772 tablet_bootstrap.cc:654] T 0749453435d54676803789c380179af4 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:23.993013 13649 tablet_service.cc:1467] Processing CreateTablet for tablet e3247dedd2b0444e858de21f6c9aa3a5 (DEFAULT_TABLE table=default.table_ca5cce29b409479f9191e1b06eb5bac0 [id=db33e4ea5b1d4f0abedf9b9dde8614e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:23.993516 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "368f01226d6a4974acdd79006703a54c" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:23.993923 13649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3247dedd2b0444e858de21f6c9aa3a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:23.994092 13671 raft_consensus.cc:2454] T 368f01226d6a4974acdd79006703a54c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 0.
I20260925 09:58:23.995262 13650 tablet_service.cc:1467] Processing CreateTablet for tablet d2065bf05eb2489bb09736b2333bf215 (DEFAULT_TABLE table=default.table_41304460d5a345c88866a7429d816aca [id=77d343c035fa4fd2b3faadcfc4fec52f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:23.996117 13650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2065bf05eb2489bb09736b2333bf215. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:23.999740 13362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "368f01226d6a4974acdd79006703a54c" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:24.000360 13362 raft_consensus.cc:2454] T 368f01226d6a4974acdd79006703a54c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 0.
I20260925 09:58:24.000921 14170 tablet.cc:1722] T 3cde6676790b4e3f89f05ce60796b36e P c4d724d4b83e4d1e817a3a505dc24eb9: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:58:24.006285 13438 leader_election.cc:302] T 368f01226d6a4974acdd79006703a54c P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:24.006965 14124 raft_consensus.cc:482] T caa7f8c507de4d1b9bcc78a39e8482f9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:24.007342 14124 raft_consensus.cc:504] T caa7f8c507de4d1b9bcc78a39e8482f9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:24.009285 14124 leader_election.cc:288] T caa7f8c507de4d1b9bcc78a39e8482f9 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:24.009720 14124 raft_consensus.cc:2781] T 368f01226d6a4974acdd79006703a54c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:24.009981 14124 raft_consensus.cc:482] T 368f01226d6a4974acdd79006703a54c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:24.010281 14124 raft_consensus.cc:3037] T 368f01226d6a4974acdd79006703a54c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:24.015543 13075 master_service.cc:946] ReplaceTablet: received request to replace tablet 3cde6676790b4e3f89f05ce60796b36e from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:24.016882 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caa7f8c507de4d1b9bcc78a39e8482f9" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:24.019363 13484 tablet_service.cc:1467] Processing CreateTablet for tablet d2065bf05eb2489bb09736b2333bf215 (DEFAULT_TABLE table=default.table_41304460d5a345c88866a7429d816aca [id=77d343c035fa4fd2b3faadcfc4fec52f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:24.020320 13484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2065bf05eb2489bb09736b2333bf215. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:24.025143 13343 tablet_service.cc:1467] Processing CreateTablet for tablet e3247dedd2b0444e858de21f6c9aa3a5 (DEFAULT_TABLE table=default.table_ca5cce29b409479f9191e1b06eb5bac0 [id=db33e4ea5b1d4f0abedf9b9dde8614e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:24.025674 13342 tablet_service.cc:1467] Processing CreateTablet for tablet a3562966c8c34046af271a7c617cdb6b (DEFAULT_TABLE table=default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:24.026036 13343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3247dedd2b0444e858de21f6c9aa3a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:24.026535 13342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3562966c8c34046af271a7c617cdb6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:24.019363 13485 tablet_service.cc:1467] Processing CreateTablet for tablet e3247dedd2b0444e858de21f6c9aa3a5 (DEFAULT_TABLE table=default.table_ca5cce29b409479f9191e1b06eb5bac0 [id=db33e4ea5b1d4f0abedf9b9dde8614e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:24.028294 13485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3247dedd2b0444e858de21f6c9aa3a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:24.032572 13362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caa7f8c507de4d1b9bcc78a39e8482f9" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:24.033351 13362 raft_consensus.cc:2379] T caa7f8c507de4d1b9bcc78a39e8482f9 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c4d724d4b83e4d1e817a3a505dc24eb9 in current term 1: Already voted for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in this term.
W20260925 09:58:24.034147 13080 catalog_manager.cc:5687] Failed to send AlterTable RPC for tablet cedc5b0609e14c409542a83efc45f3f0 (table default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet cedc5b0609e14c409542a83efc45f3f0 consensus state has no leader
I20260925 09:58:24.042279 13076 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50556:
name: "default.Table_76fed9ed8d204d969883e54e5c7330c7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:58:24.043903 13080 catalog_manager.cc:5587] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327) does not have the latest schema for tablet 0749453435d54676803789c380179af4 (table default.table_81574dad60e9490cb79e268b12969a83 [id=8fa5a3419e394be2a52d1b13713a04d9]). Expected version 1 got 0
I20260925 09:58:24.047840 13649 tablet_service.cc:1467] Processing CreateTablet for tablet a3562966c8c34046af271a7c617cdb6b (DEFAULT_TABLE table=default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:24.048883 13649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3562966c8c34046af271a7c617cdb6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:24.050773 13438 leader_election.cc:302] T caa7f8c507de4d1b9bcc78a39e8482f9 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4d724d4b83e4d1e817a3a505dc24eb9; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, b3082cf15dc74c0480e5d33ecf592843
I20260925 09:58:24.051975 13481 tablet_service.cc:1467] Processing CreateTablet for tablet a3562966c8c34046af271a7c617cdb6b (DEFAULT_TABLE table=default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:24.054728 13481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3562966c8c34046af271a7c617cdb6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:24.059722 14125 raft_consensus.cc:3037] T caa7f8c507de4d1b9bcc78a39e8482f9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:24.097846 13075 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet 3cde6676790b4e3f89f05ce60796b36e
I20260925 09:58:24.100585 13075 catalog_manager.cc:6565] ReplaceTablet: tablet 3cde6676790b4e3f89f05ce60796b36e deleted and replaced by tablet 62d6d3750baa41c193c39ca39c4486a1
I20260925 09:58:24.100656 13649 tablet_service.cc:1514] Processing DeleteTablet for tablet 3cde6676790b4e3f89f05ce60796b36e with delete_type TABLET_DATA_DELETED (replaced by tablet 62d6d3750baa41c193c39ca39c4486a1) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:24.101161 13340 tablet_service.cc:1514] Processing DeleteTablet for tablet 3cde6676790b4e3f89f05ce60796b36e with delete_type TABLET_DATA_DELETED (replaced by tablet 62d6d3750baa41c193c39ca39c4486a1) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:24.101456 13780 tablet_replica.cc:330] T 3cde6676790b4e3f89f05ce60796b36e P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:24.101886 13777 tablet_replica.cc:330] T 3cde6676790b4e3f89f05ce60796b36e P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:24.102135 13075 master_service.cc:946] ReplaceTablet: received request to replace tablet 8115a86775884efb8b9f91646f8abc48 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:24.102169 13780 raft_consensus.cc:2229] T 3cde6676790b4e3f89f05ce60796b36e P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:24.102623 13780 raft_consensus.cc:2258] T 3cde6676790b4e3f89f05ce60796b36e P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:24.102593 13777 raft_consensus.cc:2229] T 3cde6676790b4e3f89f05ce60796b36e P b3082cf15dc74c0480e5d33ecf592843 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:58:24.103492 13777 raft_consensus.cc:2258] T 3cde6676790b4e3f89f05ce60796b36e P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:24.104702 13482 tablet_service.cc:1514] Processing DeleteTablet for tablet 3cde6676790b4e3f89f05ce60796b36e with delete_type TABLET_DATA_DELETED (replaced by tablet 62d6d3750baa41c193c39ca39c4486a1) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:24.104851 13780 ts_tablet_manager.cc:1905] T 3cde6676790b4e3f89f05ce60796b36e P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:24.105885 13777 ts_tablet_manager.cc:1905] T 3cde6676790b4e3f89f05ce60796b36e P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:24.117308 13772 tablet_bootstrap.cc:492] T 0749453435d54676803789c380179af4 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:24.117780 13772 ts_tablet_manager.cc:1397] T 0749453435d54676803789c380179af4 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.158s user 0.015s sys 0.005s
I20260925 09:58:24.119899 13780 ts_tablet_manager.cc:1918] T 3cde6676790b4e3f89f05ce60796b36e P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:58:24.123107 13780 log.cc:1196] T 3cde6676790b4e3f89f05ce60796b36e P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/3cde6676790b4e3f89f05ce60796b36e
I20260925 09:58:23.987289 13770 tablet_bootstrap.cc:654] T cedc5b0609e14c409542a83efc45f3f0 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:24.120008 13772 raft_consensus.cc:348] T 0749453435d54676803789c380179af4 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:24.127732 13772 raft_consensus.cc:374] T 0749453435d54676803789c380179af4 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:24.128083 13773 tablet_bootstrap.cc:492] T cedc5b0609e14c409542a83efc45f3f0 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:24.128356 13772 raft_consensus.cc:729] T 0749453435d54676803789c380179af4 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:24.128516 13773 ts_tablet_manager.cc:1397] T cedc5b0609e14c409542a83efc45f3f0 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.204s user 0.010s sys 0.029s
I20260925 09:58:24.130928 13773 raft_consensus.cc:348] T cedc5b0609e14c409542a83efc45f3f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:24.131625 13780 ts_tablet_manager.cc:1939] T 3cde6676790b4e3f89f05ce60796b36e P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:24.133985 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet 3cde6676790b4e3f89f05ce60796b36e (table default.table_d4409ab5331140e9a4bb995d57be4ec4 [id=f196d9d3750b4857905c1332d3ac763e]) successfully deleted
I20260925 09:58:24.130213 13772 consensus_queue.cc:260] T 0749453435d54676803789c380179af4 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:24.134564 13773 raft_consensus.cc:374] T cedc5b0609e14c409542a83efc45f3f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:24.134831 13773 raft_consensus.cc:729] T cedc5b0609e14c409542a83efc45f3f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:24.135509 13773 consensus_queue.cc:260] T cedc5b0609e14c409542a83efc45f3f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:24.141990 13781 ts_tablet_manager.cc:1918] T 963c1e91c6224e5dae4ecc9a6f48f1ac P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:58:24.144711 13781 log.cc:1196] T 963c1e91c6224e5dae4ecc9a6f48f1ac P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/963c1e91c6224e5dae4ecc9a6f48f1ac
I20260925 09:58:24.144881 13773 ts_tablet_manager.cc:1428] T cedc5b0609e14c409542a83efc45f3f0 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.016s user 0.007s sys 0.004s
I20260925 09:58:24.145320 13075 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet 8115a86775884efb8b9f91646f8abc48
I20260925 09:58:24.145457 13781 ts_tablet_manager.cc:1939] T 963c1e91c6224e5dae4ecc9a6f48f1ac P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:24.147737 13075 catalog_manager.cc:6565] ReplaceTablet: tablet 8115a86775884efb8b9f91646f8abc48 deleted and replaced by tablet e505eea45c414e82a69d7d965de6c814
I20260925 09:58:24.151737 13650 tablet_service.cc:1514] Processing DeleteTablet for tablet 8115a86775884efb8b9f91646f8abc48 with delete_type TABLET_DATA_DELETED (replaced by tablet e505eea45c414e82a69d7d965de6c814) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:24.152424 13780 tablet_replica.cc:330] T 8115a86775884efb8b9f91646f8abc48 P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:24.153067 13780 raft_consensus.cc:2229] T 8115a86775884efb8b9f91646f8abc48 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:24.153558 13780 raft_consensus.cc:2258] T 8115a86775884efb8b9f91646f8abc48 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:24.155807 13777 ts_tablet_manager.cc:1918] T 3cde6676790b4e3f89f05ce60796b36e P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:58:24.155915 13780 ts_tablet_manager.cc:1905] T 8115a86775884efb8b9f91646f8abc48 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:24.156086 14194 raft_consensus.cc:482] T cedc5b0609e14c409542a83efc45f3f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:24.156643 14194 raft_consensus.cc:504] T cedc5b0609e14c409542a83efc45f3f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:24.158396 13777 log.cc:1196] T 3cde6676790b4e3f89f05ce60796b36e P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/3cde6676790b4e3f89f05ce60796b36e
I20260925 09:58:24.161060 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cedc5b0609e14c409542a83efc45f3f0" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:24.156401 13772 ts_tablet_manager.cc:1428] T 0749453435d54676803789c380179af4 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.038s user 0.005s sys 0.000s
I20260925 09:58:24.159335 13362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cedc5b0609e14c409542a83efc45f3f0" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:24.161828 13362 raft_consensus.cc:2454] T cedc5b0609e14c409542a83efc45f3f0 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 0.
W20260925 09:58:24.162351 13440 leader_election.cc:341] T cedc5b0609e14c409542a83efc45f3f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:24.163462 13438 leader_election.cc:302] T cedc5b0609e14c409542a83efc45f3f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237
I20260925 09:58:24.163589 13341 tablet_service.cc:1514] Processing DeleteTablet for tablet 8115a86775884efb8b9f91646f8abc48 with delete_type TABLET_DATA_DELETED (replaced by tablet e505eea45c414e82a69d7d965de6c814) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:24.167606 14194 leader_election.cc:288] T cedc5b0609e14c409542a83efc45f3f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:24.168411 14194 raft_consensus.cc:2781] T cedc5b0609e14c409542a83efc45f3f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:24.169248 14194 raft_consensus.cc:482] T cedc5b0609e14c409542a83efc45f3f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:24.167176 14124 raft_consensus.cc:504] T 368f01226d6a4974acdd79006703a54c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:24.170845 13781 tablet_replica.cc:330] T b7a703a4d9e04401b13610380acd2ac8 P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:24.170064 14194 raft_consensus.cc:3037] T cedc5b0609e14c409542a83efc45f3f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:24.176016 13781 raft_consensus.cc:2229] T b7a703a4d9e04401b13610380acd2ac8 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:58:24.177511 13781 raft_consensus.cc:2258] T b7a703a4d9e04401b13610380acd2ac8 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:24.178900 13772 tablet_bootstrap.cc:492] T d2065bf05eb2489bb09736b2333bf215 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:24.131325 14125 raft_consensus.cc:2726] T caa7f8c507de4d1b9bcc78a39e8482f9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:58:24.161384 13777 ts_tablet_manager.cc:1939] T 3cde6676790b4e3f89f05ce60796b36e P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:24.186441 13777 tablet_replica.cc:330] T 8115a86775884efb8b9f91646f8abc48 P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:24.186403 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet 3cde6676790b4e3f89f05ce60796b36e (table default.table_d4409ab5331140e9a4bb995d57be4ec4 [id=f196d9d3750b4857905c1332d3ac763e]) successfully deleted
I20260925 09:58:24.187903 13080 catalog_manager.cc:5587] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161) does not have the latest schema for tablet cedc5b0609e14c409542a83efc45f3f0 (table default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]). Expected version 1 got 0
I20260925 09:58:24.188529 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet 963c1e91c6224e5dae4ecc9a6f48f1ac (table default.table_81574dad60e9490cb79e268b12969a83 [id=8fa5a3419e394be2a52d1b13713a04d9]) successfully deleted
I20260925 09:58:24.189158 13482 tablet_service.cc:1514] Processing DeleteTablet for tablet 8115a86775884efb8b9f91646f8abc48 with delete_type TABLET_DATA_DELETED (replaced by tablet e505eea45c414e82a69d7d965de6c814) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:24.191648 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "368f01226d6a4974acdd79006703a54c" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
I20260925 09:58:24.192083 13777 raft_consensus.cc:2229] T 8115a86775884efb8b9f91646f8abc48 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:24.192152 13671 raft_consensus.cc:3037] T 368f01226d6a4974acdd79006703a54c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:24.191229 13081 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50556:
table { table_name: "default.Table_2c3c283350d644f39f1903713ab06589" } modify_external_catalogs: true
I20260925 09:58:24.192510 13777 raft_consensus.cc:2258] T 8115a86775884efb8b9f91646f8abc48 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:24.193311 13781 ts_tablet_manager.cc:1905] T b7a703a4d9e04401b13610380acd2ac8 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 09:58:24.193733 13067 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet cedc5b0609e14c409542a83efc45f3f0 (table default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet cedc5b0609e14c409542a83efc45f3f0 consensus state has no leader
I20260925 09:58:24.192636 13081 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50556:
table { table_name: "default.Table_2c3c283350d644f39f1903713ab06589" } modify_external_catalogs: true
I20260925 09:58:24.195317 13772 tablet_bootstrap.cc:654] T d2065bf05eb2489bb09736b2333bf215 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:24.196709 13650 tablet_service.cc:1514] Processing DeleteTablet for tablet 3cde6676790b4e3f89f05ce60796b36e with delete_type TABLET_DATA_DELETED (replaced by tablet 62d6d3750baa41c193c39ca39c4486a1) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:24.197135 13079 catalog_manager.cc:2148] created HMS catalog entry for table [id=714d733484104c508e7ced7cb14148bf]
I20260925 09:58:24.202165 13482 tablet_service.cc:1467] Processing CreateTablet for tablet 62d6d3750baa41c193c39ca39c4486a1 (DEFAULT_TABLE table=default.table_d4409ab5331140e9a4bb995d57be4ec4 [id=f196d9d3750b4857905c1332d3ac763e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:24.202360 13080 catalog_manager.cc:5587] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161) does not have the latest schema for tablet 0749453435d54676803789c380179af4 (table default.table_81574dad60e9490cb79e268b12969a83 [id=8fa5a3419e394be2a52d1b13713a04d9]). Expected version 1 got 0
I20260925 09:58:24.203300 13482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62d6d3750baa41c193c39ca39c4486a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:24.203896 13650 tablet_service.cc:1467] Processing CreateTablet for tablet 62d6d3750baa41c193c39ca39c4486a1 (DEFAULT_TABLE table=default.table_d4409ab5331140e9a4bb995d57be4ec4 [id=f196d9d3750b4857905c1332d3ac763e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:24.204051 13343 tablet_service.cc:1467] Processing CreateTablet for tablet 62d6d3750baa41c193c39ca39c4486a1 (DEFAULT_TABLE table=default.table_d4409ab5331140e9a4bb995d57be4ec4 [id=f196d9d3750b4857905c1332d3ac763e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:24.204821 13650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62d6d3750baa41c193c39ca39c4486a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:24.204994 13343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62d6d3750baa41c193c39ca39c4486a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:24.207568 13341 tablet_service.cc:1514] Processing DeleteTablet for tablet 3cde6676790b4e3f89f05ce60796b36e with delete_type TABLET_DATA_DELETED (replaced by tablet 62d6d3750baa41c193c39ca39c4486a1) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:24.210028 13671 raft_consensus.cc:2454] T 368f01226d6a4974acdd79006703a54c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:24.210686 14018 consensus_queue.cc:1035] T caa7f8c507de4d1b9bcc78a39e8482f9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:24.211130 13440 leader_election.cc:302] T 368f01226d6a4974acdd79006703a54c P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:24.216459 14196 raft_consensus.cc:2781] T 368f01226d6a4974acdd79006703a54c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:24.185953 14194 raft_consensus.cc:504] T cedc5b0609e14c409542a83efc45f3f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:24.227973 13770 tablet_bootstrap.cc:492] T cedc5b0609e14c409542a83efc45f3f0 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:24.228585 13770 ts_tablet_manager.cc:1397] T cedc5b0609e14c409542a83efc45f3f0 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.253s user 0.009s sys 0.000s
I20260925 09:58:24.233428 13770 raft_consensus.cc:348] T cedc5b0609e14c409542a83efc45f3f0 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:24.234270 13362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cedc5b0609e14c409542a83efc45f3f0" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843"
I20260925 09:58:24.234817 13362 raft_consensus.cc:3037] T cedc5b0609e14c409542a83efc45f3f0 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:24.236471 13773 tablet_bootstrap.cc:492] T d2065bf05eb2489bb09736b2333bf215 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:24.241408 13773 tablet_bootstrap.cc:654] T d2065bf05eb2489bb09736b2333bf215 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:24.241492 13772 tablet_bootstrap.cc:492] T d2065bf05eb2489bb09736b2333bf215 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:24.241559 13362 raft_consensus.cc:2454] T cedc5b0609e14c409542a83efc45f3f0 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:24.242756 14124 leader_election.cc:288] T 368f01226d6a4974acdd79006703a54c P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Requested vote from peers a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731), b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161)
I20260925 09:58:24.245610 13777 ts_tablet_manager.cc:1905] T 8115a86775884efb8b9f91646f8abc48 P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:24.242093 13772 ts_tablet_manager.cc:1397] T d2065bf05eb2489bb09736b2333bf215 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.063s user 0.010s sys 0.000s
I20260925 09:58:24.243343 13361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "368f01226d6a4974acdd79006703a54c" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843"
I20260925 09:58:24.246407 13361 raft_consensus.cc:3037] T 368f01226d6a4974acdd79006703a54c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:24.246304 13438 leader_election.cc:302] T cedc5b0609e14c409542a83efc45f3f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:24.247251 13770 raft_consensus.cc:374] T cedc5b0609e14c409542a83efc45f3f0 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:24.251257 13770 raft_consensus.cc:729] T cedc5b0609e14c409542a83efc45f3f0 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:24.251279 14194 leader_election.cc:288] T cedc5b0609e14c409542a83efc45f3f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Requested vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:24.234522 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cedc5b0609e14c409542a83efc45f3f0" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
I20260925 09:58:24.251868 14194 raft_consensus.cc:2781] T cedc5b0609e14c409542a83efc45f3f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:24.252388 13770 consensus_queue.cc:260] T cedc5b0609e14c409542a83efc45f3f0 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:24.254212 13361 raft_consensus.cc:2454] T 368f01226d6a4974acdd79006703a54c P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:24.255622 13671 raft_consensus.cc:3037] T cedc5b0609e14c409542a83efc45f3f0 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:24.259105 13772 raft_consensus.cc:348] T d2065bf05eb2489bb09736b2333bf215 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:24.259811 13772 raft_consensus.cc:374] T d2065bf05eb2489bb09736b2333bf215 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:24.260040 13772 raft_consensus.cc:729] T d2065bf05eb2489bb09736b2333bf215 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:24.260555 13772 consensus_queue.cc:260] T d2065bf05eb2489bb09736b2333bf215 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:24.263660 14180 raft_consensus.cc:482] T 0749453435d54676803789c380179af4 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:24.264055 14180 raft_consensus.cc:504] T 0749453435d54676803789c380179af4 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:24.265882 14180 leader_election.cc:288] T 0749453435d54676803789c380179af4 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:24.268464 13669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0749453435d54676803789c380179af4" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:24.272187 13504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0749453435d54676803789c380179af4" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
W20260925 09:58:24.272255 13297 leader_election.cc:341] T 0749453435d54676803789c380179af4 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:24.272789 13504 raft_consensus.cc:2454] T 0749453435d54676803789c380179af4 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 0.
I20260925 09:58:24.274817 14196 raft_consensus.cc:686] T 368f01226d6a4974acdd79006703a54c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: Becoming Leader. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Running, Role: LEADER
I20260925 09:58:24.276314 13298 leader_election.cc:302] T 0749453435d54676803789c380179af4 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237
I20260925 09:58:24.277271 14180 raft_consensus.cc:2781] T 0749453435d54676803789c380179af4 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:24.277575 14180 raft_consensus.cc:482] T 0749453435d54676803789c380179af4 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:24.277868 14180 raft_consensus.cc:3037] T 0749453435d54676803789c380179af4 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:24.278213 14194 raft_consensus.cc:686] T cedc5b0609e14c409542a83efc45f3f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: Becoming Leader. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Running, Role: LEADER
I20260925 09:58:24.283912 13079 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:50556 (request call id 83) took 1464 ms. Trace:
I20260925 09:58:24.285161 14180 raft_consensus.cc:504] T 0749453435d54676803789c380179af4 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:24.286268 13780 ts_tablet_manager.cc:1918] T 8115a86775884efb8b9f91646f8abc48 P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:24.286602 13780 log.cc:1196] T 8115a86775884efb8b9f91646f8abc48 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/8115a86775884efb8b9f91646f8abc48
I20260925 09:58:24.284219 13079 rpcz_store.cc:276] 0925 09:58:22.819514 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:22.819682 (+ 168us) service_pool.cc:227] Handling call
0925 09:58:23.937025 (+1117343us) catalog_manager.cc:2075] Acquired catalog manager lock
0925 09:58:23.938595 (+ 1570us) catalog_manager.cc:2121] Created new table and tablet info
0925 09:58:24.197098 (+258503us) catalog_manager.cc:2147] Created new table in HMS catalog
0925 09:58:24.266516 (+ 69418us) catalog_manager.cc:2181] Wrote table and tablets to system table
0925 09:58:24.283605 (+ 17089us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0925 09:58:24.283858 (+ 253us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:58:24.198606 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:58:24.198979 (+ 373us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:58:24.199002 (+ 23us) write_op.cc:435] Acquired schema lock
0925 09:58:24.199014 (+ 12us) tablet.cc:660] Decoding operations
0925 09:58:24.199258 (+ 244us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:58:24.199296 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0925 09:58:24.199316 (+ 20us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:58:24.199521 (+ 205us) tablet.cc:699] Row locks acquired
0925 09:58:24.199534 (+ 13us) write_op.cc:260] PREPARE: finished
0925 09:58:24.199639 (+ 105us) write_op.cc:270] Start()
0925 09:58:24.199714 (+ 75us) write_op.cc:276] Timestamp: P: 1790330304199546 usec, L: 0
0925 09:58:24.199730 (+ 16us) op_driver.cc:331] REPLICATION: starting
0925 09:58:24.200167 (+ 437us) log.cc:842] Serialized 433 byte log entry
0925 09:58:24.263405 (+ 63238us) op_driver.cc:444] REPLICATION: finished
0925 09:58:24.263875 (+ 470us) write_op.cc:301] APPLY: starting
0925 09:58:24.263925 (+ 50us) tablet.cc:1365] starting BulkCheckPresence
0925 09:58:24.264018 (+ 93us) tablet.cc:1368] finished BulkCheckPresence
0925 09:58:24.264101 (+ 83us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:58:24.264442 (+ 341us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:58:24.264565 (+ 123us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 09:58:24.264602 (+ 37us) write_op.cc:312] APPLY: finished
0925 09:58:24.264755 (+ 153us) log.cc:842] Serialized 29 byte log entry
0925 09:58:24.264880 (+ 125us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:24.264956 (+ 76us) write_op.cc:454] Released schema lock
0925 09:58:24.264983 (+ 27us) write_op.cc:341] FINISH: Updating metrics
0925 09:58:24.265133 (+ 150us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:24.265152 (+ 19us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":71,"Hive Metastore.run_cpu_time_us":622,"Hive Metastore.run_wall_time_us":256323,"child_traces":[["op",{"apply.queue_time_us":304,"apply.run_cpu_time_us":1420,"apply.run_wall_time_us":1420,"num_ops":2,"prepare.queue_time_us":126,"prepare.run_cpu_time_us":1759,"prepare.run_wall_time_us":1759,"replication_time_us":63562}]]}
I20260925 09:58:24.288326 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0749453435d54676803789c380179af4" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9"
I20260925 09:58:24.288877 13669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0749453435d54676803789c380179af4" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
W20260925 09:58:24.290122 13297 leader_election.cc:341] T 0749453435d54676803789c380179af4 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:24.293468 13505 raft_consensus.cc:3037] T 0749453435d54676803789c380179af4 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:24.298410 14180 leader_election.cc:288] T 0749453435d54676803789c380179af4 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Requested vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:24.281383 13773 tablet_bootstrap.cc:492] T d2065bf05eb2489bb09736b2333bf215 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:24.299031 13773 ts_tablet_manager.cc:1397] T d2065bf05eb2489bb09736b2333bf215 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.063s user 0.011s sys 0.001s
I20260925 09:58:24.279776 14196 consensus_queue.cc:237] T 368f01226d6a4974acdd79006703a54c P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:24.293956 14194 consensus_queue.cc:237] T cedc5b0609e14c409542a83efc45f3f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:24.307622 13770 ts_tablet_manager.cc:1428] T cedc5b0609e14c409542a83efc45f3f0 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.073s user 0.006s sys 0.000s
I20260925 09:58:24.307332 14174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } } }
I20260925 09:58:24.308141 14174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:58:24.312018 14150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7d0c1c2f1ed4e7a9ed54e73869d53b0. Latest consensus state: current_term: 4 leader_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } } }
I20260925 09:58:24.312667 14150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:58:24.312979 13770 tablet_bootstrap.cc:492] T 0749453435d54676803789c380179af4 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:24.313191 14184 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b7d0c1c2f1ed4e7a9ed54e73869d53b0)
I20260925 09:58:24.313710 14184 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:24.317817 13770 tablet_bootstrap.cc:654] T 0749453435d54676803789c380179af4 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:24.320127 13671 raft_consensus.cc:2454] T cedc5b0609e14c409542a83efc45f3f0 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:24.321125 13079 master_service.cc:946] ReplaceTablet: received request to replace tablet d2065bf05eb2489bb09736b2333bf215 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:24.325353 13777 ts_tablet_manager.cc:1918] T 8115a86775884efb8b9f91646f8abc48 P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:24.325755 13777 log.cc:1196] T 8115a86775884efb8b9f91646f8abc48 P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/8115a86775884efb8b9f91646f8abc48
I20260925 09:58:24.326762 14184 leader_election.cc:288] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 70718e3f6f9f489089c0d8354380ed6c (127.10.9.62:43437), b7d0c1c2f1ed4e7a9ed54e73869d53b0 (127.10.9.61:40425)
I20260925 09:58:24.322157 13773 raft_consensus.cc:348] T d2065bf05eb2489bb09736b2333bf215 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:24.340873 13773 raft_consensus.cc:374] T d2065bf05eb2489bb09736b2333bf215 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:24.341125 13773 raft_consensus.cc:729] T d2065bf05eb2489bb09736b2333bf215 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:24.345456 13772 ts_tablet_manager.cc:1428] T d2065bf05eb2489bb09736b2333bf215 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.094s user 0.006s sys 0.000s
I20260925 09:58:24.355043 13777 ts_tablet_manager.cc:1939] T 8115a86775884efb8b9f91646f8abc48 P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:24.360175 13772 tablet_bootstrap.cc:492] T e3247dedd2b0444e858de21f6c9aa3a5 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:24.366281 13079 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet d2065bf05eb2489bb09736b2333bf215
I20260925 09:58:24.368507 13079 catalog_manager.cc:6565] ReplaceTablet: tablet d2065bf05eb2489bb09736b2333bf215 deleted and replaced by tablet ace8260d4ac94cafbb9147ebeb01ed85
I20260925 09:58:24.368088 13773 consensus_queue.cc:260] T d2065bf05eb2489bb09736b2333bf215 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:24.368731 13482 tablet_service.cc:1514] Processing DeleteTablet for tablet d2065bf05eb2489bb09736b2333bf215 with delete_type TABLET_DATA_DELETED (replaced by tablet ace8260d4ac94cafbb9147ebeb01ed85) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:24.370146 13773 ts_tablet_manager.cc:1428] T d2065bf05eb2489bb09736b2333bf215 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.067s user 0.005s sys 0.000s
I20260925 09:58:24.370985 13773 tablet_bootstrap.cc:492] T e3247dedd2b0444e858de21f6c9aa3a5 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:24.371759 13649 tablet_service.cc:1514] Processing DeleteTablet for tablet d2065bf05eb2489bb09736b2333bf215 with delete_type TABLET_DATA_DELETED (replaced by tablet ace8260d4ac94cafbb9147ebeb01ed85) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:24.373008 14018 consensus_queue.cc:1035] T caa7f8c507de4d1b9bcc78a39e8482f9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:24.373829 13079 master_service.cc:946] ReplaceTablet: received request to replace tablet caa7f8c507de4d1b9bcc78a39e8482f9 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:24.374536 13505 raft_consensus.cc:2454] T 0749453435d54676803789c380179af4 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 1.
I20260925 09:58:24.375011 14180 raft_consensus.cc:482] T d2065bf05eb2489bb09736b2333bf215 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:24.375510 14180 raft_consensus.cc:504] T d2065bf05eb2489bb09736b2333bf215 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:24.379529 13504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2065bf05eb2489bb09736b2333bf215" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
I20260925 09:58:24.379786 13298 leader_election.cc:302] T 0749453435d54676803789c380179af4 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237
I20260925 09:58:24.380129 13504 raft_consensus.cc:2454] T d2065bf05eb2489bb09736b2333bf215 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 0.
I20260925 09:58:24.380686 13971 raft_consensus.cc:2781] T 0749453435d54676803789c380179af4 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:24.381114 13971 raft_consensus.cc:686] T 0749453435d54676803789c380179af4 P b3082cf15dc74c0480e5d33ecf592843 [term 1 LEADER]: Becoming Leader. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Running, Role: LEADER
I20260925 09:58:24.382182 14180 leader_election.cc:288] T d2065bf05eb2489bb09736b2333bf215 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:24.381834 13971 consensus_queue.cc:237] T 0749453435d54676803789c380179af4 P b3082cf15dc74c0480e5d33ecf592843 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:24.385411 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2065bf05eb2489bb09736b2333bf215" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:24.385690 13298 leader_election.cc:302] T d2065bf05eb2489bb09736b2333bf215 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:24.386612 14179 raft_consensus.cc:2781] T d2065bf05eb2489bb09736b2333bf215 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260925 09:58:24.386745 13297 leader_election.cc:341] T d2065bf05eb2489bb09736b2333bf215 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:24.387001 14179 raft_consensus.cc:482] T d2065bf05eb2489bb09736b2333bf215 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:24.387301 14179 raft_consensus.cc:3037] T d2065bf05eb2489bb09736b2333bf215 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:24.387617 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet 8115a86775884efb8b9f91646f8abc48 (table default.table_4d3f82d91a5749679343f60e3dbbc422 [id=dc9c88f862a64597aab300baf32f6f76]) successfully deleted
I20260925 09:58:24.388176 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet d2065bf05eb2489bb09736b2333bf215 with delete_type TABLET_DATA_DELETED (replaced by tablet ace8260d4ac94cafbb9147ebeb01ed85) from {username='slave'} at 127.0.0.1:49260
W20260925 09:58:24.388314 13066 catalog_manager.cc:4808] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): delete failed for tablet 3cde6676790b4e3f89f05ce60796b36e because the tablet was not found. No further retry: Not found: Tablet not found: 3cde6676790b4e3f89f05ce60796b36e
I20260925 09:58:24.389544 13777 tablet_replica.cc:330] T d2065bf05eb2489bb09736b2333bf215 P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:24.394109 13781 ts_tablet_manager.cc:1918] T b7a703a4d9e04401b13610380acd2ac8 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:58:24.397414 14179 raft_consensus.cc:504] T d2065bf05eb2489bb09736b2333bf215 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:24.401073 13777 raft_consensus.cc:2229] T d2065bf05eb2489bb09736b2333bf215 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:24.403637 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2065bf05eb2489bb09736b2333bf215" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
W20260925 09:58:24.404944 13297 leader_election.cc:341] T d2065bf05eb2489bb09736b2333bf215 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:24.406127 13777 raft_consensus.cc:2258] T d2065bf05eb2489bb09736b2333bf215 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:24.406467 14179 leader_election.cc:288] T d2065bf05eb2489bb09736b2333bf215 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Requested vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:24.412552 13777 ts_tablet_manager.cc:1905] T d2065bf05eb2489bb09736b2333bf215 P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:24.417836 13770 tablet_bootstrap.cc:492] T 0749453435d54676803789c380179af4 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:24.418284 13770 ts_tablet_manager.cc:1397] T 0749453435d54676803789c380179af4 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.106s user 0.007s sys 0.001s
I20260925 09:58:24.287264 13780 ts_tablet_manager.cc:1939] T 8115a86775884efb8b9f91646f8abc48 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:24.400290 13504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2065bf05eb2489bb09736b2333bf215" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9"
I20260925 09:58:24.421384 13504 raft_consensus.cc:3037] T d2065bf05eb2489bb09736b2333bf215 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:24.424517 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet 8115a86775884efb8b9f91646f8abc48 (table default.table_4d3f82d91a5749679343f60e3dbbc422 [id=dc9c88f862a64597aab300baf32f6f76]) successfully deleted
W20260925 09:58:24.425223 13068 catalog_manager.cc:4808] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): delete failed for tablet 3cde6676790b4e3f89f05ce60796b36e because the tablet was not found. No further retry: Not found: Tablet not found: 3cde6676790b4e3f89f05ce60796b36e
W20260925 09:58:24.425861 13068 catalog_manager.cc:4821] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): delete failed for tablet d2065bf05eb2489bb09736b2333bf215 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d2065bf05eb2489bb09736b2333bf215 already in progress: creating tablet
I20260925 09:58:24.428786 13504 raft_consensus.cc:2454] T d2065bf05eb2489bb09736b2333bf215 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 1.
I20260925 09:58:24.429776 13298 leader_election.cc:302] T d2065bf05eb2489bb09736b2333bf215 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237
W20260925 09:58:24.430327 13298 raft_consensus.cc:2670] T d2065bf05eb2489bb09736b2333bf215 P b3082cf15dc74c0480e5d33ecf592843: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260925 09:58:24.431990 13781 log.cc:1196] T b7a703a4d9e04401b13610380acd2ac8 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/b7a703a4d9e04401b13610380acd2ac8
I20260925 09:58:24.432901 13781 ts_tablet_manager.cc:1939] T b7a703a4d9e04401b13610380acd2ac8 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:24.432940 13772 tablet_bootstrap.cc:654] T e3247dedd2b0444e858de21f6c9aa3a5 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:24.440148 13079 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet caa7f8c507de4d1b9bcc78a39e8482f9
I20260925 09:58:24.440564 13770 raft_consensus.cc:348] T 0749453435d54676803789c380179af4 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:24.441983 13343 tablet_service.cc:1514] Processing DeleteTablet for tablet caa7f8c507de4d1b9bcc78a39e8482f9 with delete_type TABLET_DATA_DELETED (replaced by tablet 48ffc9cb12e346e7ae20c7f916ac33d4) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:24.443332 13770 raft_consensus.cc:374] T 0749453435d54676803789c380179af4 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:24.444243 14038 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37588 (request call id 0) took 2122 ms. Trace:
I20260925 09:58:24.444710 13773 tablet_bootstrap.cc:654] T e3247dedd2b0444e858de21f6c9aa3a5 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:24.444554 13770 raft_consensus.cc:729] T 0749453435d54676803789c380179af4 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:24.448165 13781 tablet_replica.cc:330] T 3cde6676790b4e3f89f05ce60796b36e P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:24.449375 13770 consensus_queue.cc:260] T 0749453435d54676803789c380179af4 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:24.467108 13017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "61e29afab0cf46819cbffc91248bf761" candidate_term: 5 candidate_status { last_received { term: 4 index: 129 } } ignore_live_leader: false dest_uuid: "70718e3f6f9f489089c0d8354380ed6c" is_pre_election: true
I20260925 09:58:24.455489 13781 raft_consensus.cc:2229] T 3cde6676790b4e3f89f05ce60796b36e P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:24.473136 13781 raft_consensus.cc:2258] T 3cde6676790b4e3f89f05ce60796b36e P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:24.479521 13770 ts_tablet_manager.cc:1428] T 0749453435d54676803789c380179af4 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.058s user 0.009s sys 0.000s
I20260925 09:58:24.442690 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet b7a703a4d9e04401b13610380acd2ac8 (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]) successfully deleted
W20260925 09:58:24.480662 13069 catalog_manager.cc:4808] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): delete failed for tablet b7a703a4d9e04401b13610380acd2ac8 because the tablet was not found. No further retry: Not found: Tablet not found: b7a703a4d9e04401b13610380acd2ac8
I20260925 09:58:24.484213 13482 tablet_service.cc:1514] Processing DeleteTablet for tablet caa7f8c507de4d1b9bcc78a39e8482f9 with delete_type TABLET_DATA_DELETED (replaced by tablet 48ffc9cb12e346e7ae20c7f916ac33d4) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:24.485996 13079 catalog_manager.cc:6565] ReplaceTablet: tablet caa7f8c507de4d1b9bcc78a39e8482f9 deleted and replaced by tablet 48ffc9cb12e346e7ae20c7f916ac33d4
I20260925 09:58:24.486832 13649 tablet_service.cc:1514] Processing DeleteTablet for tablet caa7f8c507de4d1b9bcc78a39e8482f9 with delete_type TABLET_DATA_DELETED (replaced by tablet 48ffc9cb12e346e7ae20c7f916ac33d4) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:24.495069 13770 tablet_bootstrap.cc:492] T e3247dedd2b0444e858de21f6c9aa3a5 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:24.478042 13781 ts_tablet_manager.cc:1905] T 3cde6676790b4e3f89f05ce60796b36e P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:24.501466 14221 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 09:58:24.499686 13780 tablet_replica.cc:330] T caa7f8c507de4d1b9bcc78a39e8482f9 P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:24.508121 13780 raft_consensus.cc:2229] T caa7f8c507de4d1b9bcc78a39e8482f9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:58:24.509101 14202 raft_consensus.cc:482] T dedf3e0a273d43ba84a861b36d0707ef P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:24.509523 14202 raft_consensus.cc:504] T dedf3e0a273d43ba84a861b36d0707ef P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:24.512681 13361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dedf3e0a273d43ba84a861b36d0707ef" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:24.513624 13504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dedf3e0a273d43ba84a861b36d0707ef" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
I20260925 09:58:24.514380 13504 raft_consensus.cc:2379] T dedf3e0a273d43ba84a861b36d0707ef P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in current term 1: Already voted for candidate b3082cf15dc74c0480e5d33ecf592843 in this term.
I20260925 09:58:24.511358 14202 leader_election.cc:288] T dedf3e0a273d43ba84a861b36d0707ef P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161)
I20260925 09:58:24.515596 13606 leader_election.cc:302] T dedf3e0a273d43ba84a861b36d0707ef P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237; no voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:58:24.517140 14202 raft_consensus.cc:3037] T dedf3e0a273d43ba84a861b36d0707ef P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:24.515275 13777 ts_tablet_manager.cc:1918] T d2065bf05eb2489bb09736b2333bf215 P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:24.519362 13777 log.cc:1196] T d2065bf05eb2489bb09736b2333bf215 P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/d2065bf05eb2489bb09736b2333bf215
I20260925 09:58:24.519876 13082 catalog_manager.cc:2148] created HMS catalog entry for table [id=07c6108c574e407f827bc4ab4ffdf345]
I20260925 09:58:24.523365 13777 ts_tablet_manager.cc:1939] T d2065bf05eb2489bb09736b2333bf215 P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:24.524178 14202 raft_consensus.cc:2726] T dedf3e0a273d43ba84a861b36d0707ef P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:58:24.525965 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet d2065bf05eb2489bb09736b2333bf215 (table default.table_41304460d5a345c88866a7429d816aca [id=77d343c035fa4fd2b3faadcfc4fec52f]) successfully deleted
I20260925 09:58:24.526473 13780 raft_consensus.cc:2258] T caa7f8c507de4d1b9bcc78a39e8482f9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:24.528645 13773 tablet_bootstrap.cc:492] T e3247dedd2b0444e858de21f6c9aa3a5 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:24.529055 13773 ts_tablet_manager.cc:1397] T e3247dedd2b0444e858de21f6c9aa3a5 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.158s user 0.009s sys 0.000s
I20260925 09:58:24.529269 13777 tablet_replica.cc:330] T caa7f8c507de4d1b9bcc78a39e8482f9 P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:24.530169 13777 raft_consensus.cc:2229] T caa7f8c507de4d1b9bcc78a39e8482f9 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:24.532646 13780 ts_tablet_manager.cc:1905] T caa7f8c507de4d1b9bcc78a39e8482f9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:24.531267 13773 raft_consensus.cc:348] T e3247dedd2b0444e858de21f6c9aa3a5 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:24.533474 13773 raft_consensus.cc:374] T e3247dedd2b0444e858de21f6c9aa3a5 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:24.531186 13777 raft_consensus.cc:2258] T caa7f8c507de4d1b9bcc78a39e8482f9 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:24.537802 13777 ts_tablet_manager.cc:1905] T caa7f8c507de4d1b9bcc78a39e8482f9 P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:24.538048 13773 raft_consensus.cc:729] T e3247dedd2b0444e858de21f6c9aa3a5 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:24.541115 13773 consensus_queue.cc:260] T e3247dedd2b0444e858de21f6c9aa3a5 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:24.555706 13777 ts_tablet_manager.cc:1918] T caa7f8c507de4d1b9bcc78a39e8482f9 P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:24.556239 13777 log.cc:1196] T caa7f8c507de4d1b9bcc78a39e8482f9 P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/caa7f8c507de4d1b9bcc78a39e8482f9
I20260925 09:58:24.557008 13777 ts_tablet_manager.cc:1939] T caa7f8c507de4d1b9bcc78a39e8482f9 P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:24.558171 13770 tablet_bootstrap.cc:654] T e3247dedd2b0444e858de21f6c9aa3a5 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:24.560210 13077 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:50556 (request call id 71) took 3663 ms. Trace:
I20260925 09:58:24.560510 13077 rpcz_store.cc:276] 0925 09:58:20.897096 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:20.928189 (+ 31093us) service_pool.cc:227] Handling call
0925 09:58:22.965764 (+2037575us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:58:24.558555 (+1592791us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:58:24.559414 (+ 859us) catalog_manager.cc:3616] Apply alter schema
0925 09:58:24.560147 (+ 733us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":76,"Hive Metastore.run_cpu_time_us":1780,"Hive Metastore.run_wall_time_us":590781}
I20260925 09:58:24.561736 13773 ts_tablet_manager.cc:1428] T e3247dedd2b0444e858de21f6c9aa3a5 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.029s user 0.006s sys 0.001s
I20260925 09:58:24.562613 13773 tablet_bootstrap.cc:492] T a3562966c8c34046af271a7c617cdb6b P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
W20260925 09:58:24.562722 13073 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_afc7cdc28ab84dff9a8782192aa34919 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:58:24.567055 13082 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:50556 (request call id 85) took 1596 ms. Trace:
I20260925 09:58:24.567407 13082 rpcz_store.cc:276] 0925 09:58:22.970586 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:22.971977 (+ 1391us) service_pool.cc:227] Handling call
0925 09:58:23.941343 (+969366us) catalog_manager.cc:2075] Acquired catalog manager lock
0925 09:58:23.943171 (+ 1828us) catalog_manager.cc:2121] Created new table and tablet info
0925 09:58:24.519834 (+576663us) catalog_manager.cc:2147] Created new table in HMS catalog
0925 09:58:24.566379 (+ 46545us) catalog_manager.cc:2181] Wrote table and tablets to system table
0925 09:58:24.566712 (+ 333us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0925 09:58:24.566998 (+ 286us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:58:24.531960 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:58:24.532311 (+ 351us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:58:24.532331 (+ 20us) write_op.cc:435] Acquired schema lock
0925 09:58:24.532342 (+ 11us) tablet.cc:660] Decoding operations
0925 09:58:24.532598 (+ 256us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:58:24.532631 (+ 33us) write_op.cc:641] Partition lock acquired for write op
0925 09:58:24.532650 (+ 19us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:58:24.532826 (+ 176us) tablet.cc:699] Row locks acquired
0925 09:58:24.532837 (+ 11us) write_op.cc:260] PREPARE: finished
0925 09:58:24.532933 (+ 96us) write_op.cc:270] Start()
0925 09:58:24.533006 (+ 73us) write_op.cc:276] Timestamp: P: 1790330304532883 usec, L: 0
0925 09:58:24.533022 (+ 16us) op_driver.cc:331] REPLICATION: starting
0925 09:58:24.533789 (+ 767us) log.cc:842] Serialized 433 byte log entry
0925 09:58:24.555003 (+ 21214us) op_driver.cc:444] REPLICATION: finished
0925 09:58:24.555705 (+ 702us) write_op.cc:301] APPLY: starting
0925 09:58:24.555753 (+ 48us) tablet.cc:1365] starting BulkCheckPresence
0925 09:58:24.555838 (+ 85us) tablet.cc:1368] finished BulkCheckPresence
0925 09:58:24.555944 (+ 106us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:58:24.556257 (+ 313us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:58:24.556366 (+ 109us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 09:58:24.556401 (+ 35us) write_op.cc:312] APPLY: finished
0925 09:58:24.556551 (+ 150us) log.cc:842] Serialized 29 byte log entry
0925 09:58:24.556677 (+ 126us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:24.556729 (+ 52us) write_op.cc:454] Released schema lock
0925 09:58:24.556754 (+ 25us) write_op.cc:341] FINISH: Updating metrics
0925 09:58:24.556893 (+ 139us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:24.556911 (+ 18us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":251729,"Hive Metastore.run_cpu_time_us":609,"Hive Metastore.run_wall_time_us":323823,"child_traces":[["op",{"apply.queue_time_us":552,"apply.run_cpu_time_us":1358,"apply.run_wall_time_us":1358,"mutex_wait_us":10303,"num_ops":2,"prepare.queue_time_us":10509,"prepare.run_cpu_time_us":1723,"prepare.run_wall_time_us":2003,"replication_time_us":21902}]]}
I20260925 09:58:24.575100 14014 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" instance_seqno: 1790330290432459) as {username='slave'} at 127.10.9.2:57765; Asking this server to re-register.
I20260925 09:58:24.577634 13550 heartbeater.cc:464] Registering TS with master...
I20260925 09:58:24.581164 13082 master_service.cc:946] ReplaceTablet: received request to replace tablet e3247dedd2b0444e858de21f6c9aa3a5 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:24.582836 14014 ts_manager.cc:194] Registered new tserver with Master: c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
I20260925 09:58:24.583438 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet caa7f8c507de4d1b9bcc78a39e8482f9 (table default.table_8f71132b5a004ad89115f7a2278b95d6 [id=0cdc8d8fef5445668c0b0107243e0ae9]) successfully deleted
I20260925 09:58:24.593489 13080 catalog_manager.cc:5587] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731) does not have the latest schema for tablet cedc5b0609e14c409542a83efc45f3f0 (table default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]). Expected version 1 got 0
I20260925 09:58:24.595022 13343 tablet_service.cc:1467] Processing CreateTablet for tablet e505eea45c414e82a69d7d965de6c814 (DEFAULT_TABLE table=default.table_4d3f82d91a5749679343f60e3dbbc422 [id=dc9c88f862a64597aab300baf32f6f76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:24.596001 13343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e505eea45c414e82a69d7d965de6c814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:24.596012 13075 catalog_manager.cc:5587] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327) does not have the latest schema for tablet cedc5b0609e14c409542a83efc45f3f0 (table default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]). Expected version 1 got 0
I20260925 09:58:24.603163 13079 catalog_manager.cc:5495] T 0749453435d54676803789c380179af4 P b3082cf15dc74c0480e5d33ecf592843 reported cstate change: term changed from 0 to 1, leader changed from <none> to b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b3082cf15dc74c0480e5d33ecf592843" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: UNKNOWN } } }
I20260925 09:58:24.605290 13079 catalog_manager.cc:5587] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161) does not have the latest schema for tablet 0749453435d54676803789c380179af4 (table default.table_81574dad60e9490cb79e268b12969a83 [id=8fa5a3419e394be2a52d1b13713a04d9]). Expected version 1 got 0
I20260925 09:58:24.607666 13772 tablet_bootstrap.cc:492] T e3247dedd2b0444e858de21f6c9aa3a5 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:24.609670 13770 tablet_bootstrap.cc:492] T e3247dedd2b0444e858de21f6c9aa3a5 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:24.610144 13770 ts_tablet_manager.cc:1397] T e3247dedd2b0444e858de21f6c9aa3a5 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.115s user 0.010s sys 0.000s
I20260925 09:58:24.613234 13482 tablet_service.cc:1467] Processing CreateTablet for tablet e505eea45c414e82a69d7d965de6c814 (DEFAULT_TABLE table=default.table_4d3f82d91a5749679343f60e3dbbc422 [id=dc9c88f862a64597aab300baf32f6f76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:24.614323 13482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e505eea45c414e82a69d7d965de6c814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:24.615266 13971 consensus_queue.cc:1035] T dedf3e0a273d43ba84a861b36d0707ef P b3082cf15dc74c0480e5d33ecf592843 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:24.615712 13772 ts_tablet_manager.cc:1397] T e3247dedd2b0444e858de21f6c9aa3a5 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.256s user 0.008s sys 0.001s
I20260925 09:58:24.622757 13090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "61e29afab0cf46819cbffc91248bf761" candidate_term: 5 candidate_status { last_received { term: 4 index: 129 } } ignore_live_leader: false dest_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" is_pre_election: true
I20260925 09:58:24.625056 13955 leader_election.cc:302] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61e29afab0cf46819cbffc91248bf761; no voters: 70718e3f6f9f489089c0d8354380ed6c, b7d0c1c2f1ed4e7a9ed54e73869d53b0
I20260925 09:58:24.626317 14184 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260925 09:58:24.630937 13773 tablet_bootstrap.cc:654] T a3562966c8c34046af271a7c617cdb6b P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:24.612920 13770 raft_consensus.cc:348] T e3247dedd2b0444e858de21f6c9aa3a5 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:24.634575 13077 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50556:
name: "default.Table_840c9ea5fcc04f1b8811754058184452"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:58:24.635174 13770 raft_consensus.cc:374] T e3247dedd2b0444e858de21f6c9aa3a5 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:24.638543 13770 raft_consensus.cc:729] T e3247dedd2b0444e858de21f6c9aa3a5 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:24.641383 13075 catalog_manager.cc:5495] T 368f01226d6a4974acdd79006703a54c P c4d724d4b83e4d1e817a3a505dc24eb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2). New cstate: current_term: 1 leader_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: UNKNOWN } } }
I20260925 09:58:24.644881 13075 catalog_manager.cc:5495] T cedc5b0609e14c409542a83efc45f3f0 P c4d724d4b83e4d1e817a3a505dc24eb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2). New cstate: current_term: 1 leader_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: UNKNOWN } } }
I20260925 09:58:24.646023 13075 catalog_manager.cc:5587] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327) does not have the latest schema for tablet cedc5b0609e14c409542a83efc45f3f0 (table default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]). Expected version 1 got 0
I20260925 09:58:24.649516 14212 consensus_queue.cc:1035] T dedf3e0a273d43ba84a861b36d0707ef P b3082cf15dc74c0480e5d33ecf592843 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:24.651762 13770 consensus_queue.cc:260] T e3247dedd2b0444e858de21f6c9aa3a5 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:24.629643 13781 ts_tablet_manager.cc:1918] T 3cde6676790b4e3f89f05ce60796b36e P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:58:24.657676 13781 log.cc:1196] T 3cde6676790b4e3f89f05ce60796b36e P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/3cde6676790b4e3f89f05ce60796b36e
I20260925 09:58:24.661351 13772 raft_consensus.cc:348] T e3247dedd2b0444e858de21f6c9aa3a5 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:24.661990 13772 raft_consensus.cc:374] T e3247dedd2b0444e858de21f6c9aa3a5 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:24.662314 13772 raft_consensus.cc:729] T e3247dedd2b0444e858de21f6c9aa3a5 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:24.659260 13770 ts_tablet_manager.cc:1428] T e3247dedd2b0444e858de21f6c9aa3a5 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.048s user 0.006s sys 0.000s
I20260925 09:58:24.665289 13770 tablet_bootstrap.cc:492] T d2065bf05eb2489bb09736b2333bf215 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
W20260925 09:58:24.665904 13438 consensus_peers.cc:487] T 8115a86775884efb8b9f91646f8abc48 P c4d724d4b83e4d1e817a3a505dc24eb9 -> Peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Couldn't send request to peer b3082cf15dc74c0480e5d33ecf592843. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8115a86775884efb8b9f91646f8abc48. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:58:24.668015 14014 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" instance_seqno: 1790330288439799) as {username='slave'} at 127.10.9.1:41625; Asking this server to re-register.
I20260925 09:58:24.670011 13781 ts_tablet_manager.cc:1939] T 3cde6676790b4e3f89f05ce60796b36e P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:24.671722 13409 heartbeater.cc:464] Registering TS with master...
I20260925 09:58:24.670559 13773 tablet_bootstrap.cc:492] T a3562966c8c34046af271a7c617cdb6b P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:24.673918 13773 ts_tablet_manager.cc:1397] T a3562966c8c34046af271a7c617cdb6b P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.111s user 0.009s sys 0.000s
I20260925 09:58:24.677819 13780 ts_tablet_manager.cc:1918] T caa7f8c507de4d1b9bcc78a39e8482f9 P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:24.688181 13780 log.cc:1196] T caa7f8c507de4d1b9bcc78a39e8482f9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/caa7f8c507de4d1b9bcc78a39e8482f9
I20260925 09:58:24.692160 13770 tablet_bootstrap.cc:654] T d2065bf05eb2489bb09736b2333bf215 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:24.685945 13773 raft_consensus.cc:348] T a3562966c8c34046af271a7c617cdb6b P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:24.692762 13773 raft_consensus.cc:374] T a3562966c8c34046af271a7c617cdb6b P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:24.693243 13773 raft_consensus.cc:729] T a3562966c8c34046af271a7c617cdb6b P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:24.694074 13773 consensus_queue.cc:260] T a3562966c8c34046af271a7c617cdb6b P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:24.696379 13773 ts_tablet_manager.cc:1428] T a3562966c8c34046af271a7c617cdb6b P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.022s user 0.005s sys 0.000s
I20260925 09:58:24.689028 13780 ts_tablet_manager.cc:1939] T caa7f8c507de4d1b9bcc78a39e8482f9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:24.701331 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet caa7f8c507de4d1b9bcc78a39e8482f9 (table default.table_8f71132b5a004ad89115f7a2278b95d6 [id=0cdc8d8fef5445668c0b0107243e0ae9]) successfully deleted
I20260925 09:58:24.717736 13080 catalog_manager.cc:5587] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731) does not have the latest schema for tablet cedc5b0609e14c409542a83efc45f3f0 (table default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]). Expected version 1 got 0
I20260925 09:58:24.718420 13080 catalog_manager.cc:5587] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731) does not have the latest schema for tablet 0749453435d54676803789c380179af4 (table default.table_81574dad60e9490cb79e268b12969a83 [id=8fa5a3419e394be2a52d1b13713a04d9]). Expected version 1 got 0
I20260925 09:58:24.719244 13781 tablet_replica.cc:330] T 8115a86775884efb8b9f91646f8abc48 P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:24.719942 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet 3cde6676790b4e3f89f05ce60796b36e (table default.table_d4409ab5331140e9a4bb995d57be4ec4 [id=f196d9d3750b4857905c1332d3ac763e]) successfully deleted
I20260925 09:58:24.720170 13781 raft_consensus.cc:2229] T 8115a86775884efb8b9f91646f8abc48 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:58:24.720960 13781 raft_consensus.cc:2258] T 8115a86775884efb8b9f91646f8abc48 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:24.722373 13773 tablet_bootstrap.cc:492] T 62d6d3750baa41c193c39ca39c4486a1 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:24.726732 13080 catalog_manager.cc:5587] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327) does not have the latest schema for tablet a3562966c8c34046af271a7c617cdb6b (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]). Expected version 1 got 0
I20260925 09:58:24.727113 13773 tablet_bootstrap.cc:654] T 62d6d3750baa41c193c39ca39c4486a1 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:24.728569 13781 ts_tablet_manager.cc:1905] T 8115a86775884efb8b9f91646f8abc48 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:24.732021 13649 tablet_service.cc:1467] Processing CreateTablet for tablet e505eea45c414e82a69d7d965de6c814 (DEFAULT_TABLE table=default.table_4d3f82d91a5749679343f60e3dbbc422 [id=dc9c88f862a64597aab300baf32f6f76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:24.732962 13649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e505eea45c414e82a69d7d965de6c814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:24.738699 13074 catalog_manager.cc:2685] deleted HMS catalog entry for table default.table_81574dad60e9490cb79e268b12969a83 [id=8fa5a3419e394be2a52d1b13713a04d9]
I20260925 09:58:24.739746 13770 tablet_bootstrap.cc:492] T d2065bf05eb2489bb09736b2333bf215 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:24.740144 13770 ts_tablet_manager.cc:1397] T d2065bf05eb2489bb09736b2333bf215 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.075s user 0.011s sys 0.002s
I20260925 09:58:24.742353 13770 raft_consensus.cc:348] T d2065bf05eb2489bb09736b2333bf215 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:24.742990 13770 raft_consensus.cc:374] T d2065bf05eb2489bb09736b2333bf215 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:24.749672 13773 tablet_bootstrap.cc:492] T 62d6d3750baa41c193c39ca39c4486a1 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:24.750058 13773 ts_tablet_manager.cc:1397] T 62d6d3750baa41c193c39ca39c4486a1 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.028s user 0.008s sys 0.000s
I20260925 09:58:24.750046 13651 tablet_service.cc:1467] Processing CreateTablet for tablet a8a89d2315f54a09ad5d6506f2c4797c (DEFAULT_TABLE table=default.table_47c0ac37d06f4f45a701b0910a42c579 [id=714d733484104c508e7ced7cb14148bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:24.751118 13651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8a89d2315f54a09ad5d6506f2c4797c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:24.743240 13770 raft_consensus.cc:729] T d2065bf05eb2489bb09736b2333bf215 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:24.752213 13773 raft_consensus.cc:348] T 62d6d3750baa41c193c39ca39c4486a1 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:24.755563 13770 consensus_queue.cc:260] T d2065bf05eb2489bb09736b2333bf215 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:24.756250 13773 raft_consensus.cc:374] T 62d6d3750baa41c193c39ca39c4486a1 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:24.756872 13773 raft_consensus.cc:729] T 62d6d3750baa41c193c39ca39c4486a1 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:24.761633 13770 ts_tablet_manager.cc:1428] T d2065bf05eb2489bb09736b2333bf215 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.021s user 0.005s sys 0.000s
I20260925 09:58:24.762413 13770 tablet_bootstrap.cc:492] T a3562966c8c34046af271a7c617cdb6b P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:24.767154 13773 consensus_queue.cc:260] T 62d6d3750baa41c193c39ca39c4486a1 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:24.782693 13773 ts_tablet_manager.cc:1428] T 62d6d3750baa41c193c39ca39c4486a1 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.032s user 0.006s sys 0.000s
I20260925 09:58:24.785282 13781 ts_tablet_manager.cc:1918] T 8115a86775884efb8b9f91646f8abc48 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:24.792740 13781 log.cc:1196] T 8115a86775884efb8b9f91646f8abc48 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/8115a86775884efb8b9f91646f8abc48
I20260925 09:58:24.793840 14202 raft_consensus.cc:482] T e3247dedd2b0444e858de21f6c9aa3a5 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:24.793781 13342 tablet_service.cc:1467] Processing CreateTablet for tablet a8a89d2315f54a09ad5d6506f2c4797c (DEFAULT_TABLE table=default.table_47c0ac37d06f4f45a701b0910a42c579 [id=714d733484104c508e7ced7cb14148bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:24.794283 14202 raft_consensus.cc:504] T e3247dedd2b0444e858de21f6c9aa3a5 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:24.794698 13342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8a89d2315f54a09ad5d6506f2c4797c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:24.796258 14202 leader_election.cc:288] T e3247dedd2b0444e858de21f6c9aa3a5 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161)
I20260925 09:58:24.796933 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3247dedd2b0444e858de21f6c9aa3a5" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
I20260925 09:58:24.797391 13781 ts_tablet_manager.cc:1939] T 8115a86775884efb8b9f91646f8abc48 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:24.797483 13505 raft_consensus.cc:2454] T e3247dedd2b0444e858de21f6c9aa3a5 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 0.
I20260925 09:58:24.798614 13606 leader_election.cc:302] T e3247dedd2b0444e858de21f6c9aa3a5 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:24.792430 13773 tablet_bootstrap.cc:492] T e505eea45c414e82a69d7d965de6c814 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:24.799456 14202 raft_consensus.cc:2781] T e3247dedd2b0444e858de21f6c9aa3a5 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:24.799880 14202 raft_consensus.cc:482] T e3247dedd2b0444e858de21f6c9aa3a5 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:24.800217 14202 raft_consensus.cc:3037] T e3247dedd2b0444e858de21f6c9aa3a5 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:24.800526 13361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3247dedd2b0444e858de21f6c9aa3a5" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:24.801631 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet 8115a86775884efb8b9f91646f8abc48 (table default.table_4d3f82d91a5749679343f60e3dbbc422 [id=dc9c88f862a64597aab300baf32f6f76]) successfully deleted
I20260925 09:58:24.806516 14202 raft_consensus.cc:504] T e3247dedd2b0444e858de21f6c9aa3a5 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:24.808419 13770 tablet_bootstrap.cc:654] T a3562966c8c34046af271a7c617cdb6b P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:24.813439 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3247dedd2b0444e858de21f6c9aa3a5" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9"
I20260925 09:58:24.813781 13650 tablet_service.cc:1514] Processing DeleteTablet for tablet caa7f8c507de4d1b9bcc78a39e8482f9 with delete_type TABLET_DATA_DELETED (replaced by tablet 48ffc9cb12e346e7ae20c7f916ac33d4) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:24.813957 13505 raft_consensus.cc:3037] T e3247dedd2b0444e858de21f6c9aa3a5 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:58:24.815237 13068 catalog_manager.cc:4808] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): delete failed for tablet caa7f8c507de4d1b9bcc78a39e8482f9 because the tablet was not found. No further retry: Not found: Tablet not found: caa7f8c507de4d1b9bcc78a39e8482f9
I20260925 09:58:24.819783 13781 tablet_replica.cc:330] T d2065bf05eb2489bb09736b2333bf215 P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:24.819980 13505 raft_consensus.cc:2454] T e3247dedd2b0444e858de21f6c9aa3a5 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 1.
I20260925 09:58:24.820565 13781 raft_consensus.cc:2229] T d2065bf05eb2489bb09736b2333bf215 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:24.821010 13781 raft_consensus.cc:2258] T d2065bf05eb2489bb09736b2333bf215 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:24.821182 13606 leader_election.cc:302] T e3247dedd2b0444e858de21f6c9aa3a5 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:24.822005 14211 raft_consensus.cc:2781] T e3247dedd2b0444e858de21f6c9aa3a5 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:24.822446 14211 raft_consensus.cc:686] T e3247dedd2b0444e858de21f6c9aa3a5 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 LEADER]: Becoming Leader. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Running, Role: LEADER
I20260925 09:58:24.822642 13773 tablet_bootstrap.cc:654] T e505eea45c414e82a69d7d965de6c814 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:24.823179 14211 consensus_queue.cc:237] T e3247dedd2b0444e858de21f6c9aa3a5 P a06c3ef7a6a94f7cbcc03e7cd232f237 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:24.828305 13781 ts_tablet_manager.cc:1905] T d2065bf05eb2489bb09736b2333bf215 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:24.830894 14202 leader_election.cc:288] T e3247dedd2b0444e858de21f6c9aa3a5 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 election: Requested vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161)
I20260925 09:58:24.830859 13362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3247dedd2b0444e858de21f6c9aa3a5" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843"
W20260925 09:58:24.730131 13080 catalog_manager.cc:5687] Failed to send AlterTable RPC for tablet a3562966c8c34046af271a7c617cdb6b (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet a3562966c8c34046af271a7c617cdb6b consensus state has no leader
I20260925 09:58:24.848416 13770 tablet_bootstrap.cc:492] T a3562966c8c34046af271a7c617cdb6b P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:24.848848 13770 ts_tablet_manager.cc:1397] T a3562966c8c34046af271a7c617cdb6b P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.087s user 0.010s sys 0.000s
I20260925 09:58:24.851265 13482 tablet_service.cc:1467] Processing CreateTablet for tablet a8a89d2315f54a09ad5d6506f2c4797c (DEFAULT_TABLE table=default.table_47c0ac37d06f4f45a701b0910a42c579 [id=714d733484104c508e7ced7cb14148bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:24.852486 13482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8a89d2315f54a09ad5d6506f2c4797c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:24.856710 13082 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet e3247dedd2b0444e858de21f6c9aa3a5
I20260925 09:58:24.860121 13341 tablet_service.cc:1514] Processing DeleteTablet for tablet e3247dedd2b0444e858de21f6c9aa3a5 with delete_type TABLET_DATA_DELETED (replaced by tablet 90d39744f6334e42bd32c26d13e6912c) from {username='slave'} at 127.0.0.1:49260
W20260925 09:58:24.861651 13066 catalog_manager.cc:4821] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): delete failed for tablet e3247dedd2b0444e858de21f6c9aa3a5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e3247dedd2b0444e858de21f6c9aa3a5 already in progress: creating tablet
I20260925 09:58:24.861639 13481 tablet_service.cc:1514] Processing DeleteTablet for tablet e3247dedd2b0444e858de21f6c9aa3a5 with delete_type TABLET_DATA_DELETED (replaced by tablet 90d39744f6334e42bd32c26d13e6912c) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:24.861784 13082 catalog_manager.cc:6565] ReplaceTablet: tablet e3247dedd2b0444e858de21f6c9aa3a5 deleted and replaced by tablet 90d39744f6334e42bd32c26d13e6912c
I20260925 09:58:24.863667 13650 tablet_service.cc:1514] Processing DeleteTablet for tablet e3247dedd2b0444e858de21f6c9aa3a5 with delete_type TABLET_DATA_DELETED (replaced by tablet 90d39744f6334e42bd32c26d13e6912c) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:24.863992 13773 tablet_bootstrap.cc:492] T e505eea45c414e82a69d7d965de6c814 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:24.864358 13773 ts_tablet_manager.cc:1397] T e505eea45c414e82a69d7d965de6c814 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.072s user 0.009s sys 0.000s
I20260925 09:58:24.864377 13780 tablet_replica.cc:330] T e3247dedd2b0444e858de21f6c9aa3a5 P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:24.864583 13770 raft_consensus.cc:348] T a3562966c8c34046af271a7c617cdb6b P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:24.865101 13780 raft_consensus.cc:2229] T e3247dedd2b0444e858de21f6c9aa3a5 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:58:24.865197 13770 raft_consensus.cc:374] T a3562966c8c34046af271a7c617cdb6b P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:24.865428 13770 raft_consensus.cc:729] T a3562966c8c34046af271a7c617cdb6b P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:24.865741 13780 pending_rounds.cc:62] T e3247dedd2b0444e858de21f6c9aa3a5 P a06c3ef7a6a94f7cbcc03e7cd232f237: Trying to abort 1 pending ops.
I20260925 09:58:24.865949 13780 pending_rounds.cc:68] T e3247dedd2b0444e858de21f6c9aa3a5 P a06c3ef7a6a94f7cbcc03e7cd232f237: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192928563744768 op_type: NO_OP noop_request { }
I20260925 09:58:24.866322 13780 raft_consensus.cc:2866] T e3247dedd2b0444e858de21f6c9aa3a5 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:58:24.866015 13770 consensus_queue.cc:260] T a3562966c8c34046af271a7c617cdb6b P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:24.866628 13780 raft_consensus.cc:2258] T e3247dedd2b0444e858de21f6c9aa3a5 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:24.866429 13773 raft_consensus.cc:348] T e505eea45c414e82a69d7d965de6c814 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:24.867034 13773 raft_consensus.cc:374] T e505eea45c414e82a69d7d965de6c814 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:24.867282 13773 raft_consensus.cc:729] T e505eea45c414e82a69d7d965de6c814 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:24.867484 13082 master_service.cc:946] ReplaceTablet: received request to replace tablet a3562966c8c34046af271a7c617cdb6b from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:24.867875 13773 consensus_queue.cc:260] T e505eea45c414e82a69d7d965de6c814 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:24.872124 13773 ts_tablet_manager.cc:1428] T e505eea45c414e82a69d7d965de6c814 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20260925 09:58:24.874416 13781 ts_tablet_manager.cc:1918] T d2065bf05eb2489bb09736b2333bf215 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:24.874842 13781 log.cc:1196] T d2065bf05eb2489bb09736b2333bf215 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/d2065bf05eb2489bb09736b2333bf215
I20260925 09:58:24.875579 13781 ts_tablet_manager.cc:1939] T d2065bf05eb2489bb09736b2333bf215 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:24.881130 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet d2065bf05eb2489bb09736b2333bf215 (table default.table_41304460d5a345c88866a7429d816aca [id=77d343c035fa4fd2b3faadcfc4fec52f]) successfully deleted
I20260925 09:58:24.883204 13078 catalog_manager.cc:3411] renamed table default.table_4d3f82d91a5749679343f60e3dbbc422 [id=dc9c88f862a64597aab300baf32f6f76] in HMS: new name default.table_1310f782dc864b068b28842f01c96b9c
I20260925 09:58:24.885176 13781 tablet_replica.cc:330] T caa7f8c507de4d1b9bcc78a39e8482f9 P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:24.887115 13781 raft_consensus.cc:2229] T caa7f8c507de4d1b9bcc78a39e8482f9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:24.888038 13781 raft_consensus.cc:2258] T caa7f8c507de4d1b9bcc78a39e8482f9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:24.889842 13773 tablet_bootstrap.cc:492] T a8a89d2315f54a09ad5d6506f2c4797c P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:24.891459 13770 ts_tablet_manager.cc:1428] T a3562966c8c34046af271a7c617cdb6b P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.042s user 0.005s sys 0.001s
I20260925 09:58:24.894761 13773 tablet_bootstrap.cc:654] T a8a89d2315f54a09ad5d6506f2c4797c P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:24.897199 13781 ts_tablet_manager.cc:1905] T caa7f8c507de4d1b9bcc78a39e8482f9 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:24.897300 13780 ts_tablet_manager.cc:1905] T e3247dedd2b0444e858de21f6c9aa3a5 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:24.900048 13770 tablet_bootstrap.cc:492] T 62d6d3750baa41c193c39ca39c4486a1 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:24.907877 13770 tablet_bootstrap.cc:654] T 62d6d3750baa41c193c39ca39c4486a1 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:24.871385 13772 consensus_queue.cc:260] T e3247dedd2b0444e858de21f6c9aa3a5 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:24.917042 13361 raft_consensus.cc:2454] T e3247dedd2b0444e858de21f6c9aa3a5 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 0.
I20260925 09:58:24.921604 13772 ts_tablet_manager.cc:1428] T e3247dedd2b0444e858de21f6c9aa3a5 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.306s user 0.006s sys 0.000s
I20260925 09:58:24.922451 13772 tablet_bootstrap.cc:492] T a3562966c8c34046af271a7c617cdb6b P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:24.935276 13773 tablet_bootstrap.cc:492] T a8a89d2315f54a09ad5d6506f2c4797c P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:24.935838 13773 ts_tablet_manager.cc:1397] T a8a89d2315f54a09ad5d6506f2c4797c P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.046s user 0.007s sys 0.002s
I20260925 09:58:24.938179 13773 raft_consensus.cc:348] T a8a89d2315f54a09ad5d6506f2c4797c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } }
I20260925 09:58:24.938971 13773 raft_consensus.cc:374] T a8a89d2315f54a09ad5d6506f2c4797c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:24.939270 13773 raft_consensus.cc:729] T a8a89d2315f54a09ad5d6506f2c4797c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:24.940026 13773 consensus_queue.cc:260] T a8a89d2315f54a09ad5d6506f2c4797c P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } }
I20260925 09:58:24.941931 13781 ts_tablet_manager.cc:1918] T caa7f8c507de4d1b9bcc78a39e8482f9 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:24.949024 13781 log.cc:1196] T caa7f8c507de4d1b9bcc78a39e8482f9 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/caa7f8c507de4d1b9bcc78a39e8482f9
I20260925 09:58:24.949932 13781 ts_tablet_manager.cc:1939] T caa7f8c507de4d1b9bcc78a39e8482f9 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:24.951243 14199 raft_consensus.cc:482] T e505eea45c414e82a69d7d965de6c814 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:24.951718 14199 raft_consensus.cc:504] T e505eea45c414e82a69d7d965de6c814 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:24.953611 14199 leader_election.cc:288] T e505eea45c414e82a69d7d965de6c814 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:24.954921 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e505eea45c414e82a69d7d965de6c814" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:24.955564 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet caa7f8c507de4d1b9bcc78a39e8482f9 (table default.table_8f71132b5a004ad89115f7a2278b95d6 [id=0cdc8d8fef5445668c0b0107243e0ae9]) successfully deleted
I20260925 09:58:24.955590 13361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e505eea45c414e82a69d7d965de6c814" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
W20260925 09:58:24.956594 13440 leader_election.cc:341] T e505eea45c414e82a69d7d965de6c814 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Illegal state: must be running to vote when last-logged opid is not known
W20260925 09:58:24.957132 13438 leader_election.cc:341] T e505eea45c414e82a69d7d965de6c814 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:24.957337 13781 tablet_replica.cc:330] T e3247dedd2b0444e858de21f6c9aa3a5 P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:24.957516 13438 leader_election.cc:302] T e505eea45c414e82a69d7d965de6c814 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4d724d4b83e4d1e817a3a505dc24eb9; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, b3082cf15dc74c0480e5d33ecf592843
I20260925 09:58:24.958607 14199 raft_consensus.cc:2726] T e505eea45c414e82a69d7d965de6c814 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:58:24.961056 13770 tablet_bootstrap.cc:492] T 62d6d3750baa41c193c39ca39c4486a1 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:24.961444 13770 ts_tablet_manager.cc:1397] T 62d6d3750baa41c193c39ca39c4486a1 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.062s user 0.014s sys 0.011s
I20260925 09:58:24.963874 13770 raft_consensus.cc:348] T 62d6d3750baa41c193c39ca39c4486a1 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:24.964713 13770 raft_consensus.cc:374] T 62d6d3750baa41c193c39ca39c4486a1 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:24.964998 13770 raft_consensus.cc:729] T 62d6d3750baa41c193c39ca39c4486a1 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:24.965703 13770 consensus_queue.cc:260] T 62d6d3750baa41c193c39ca39c4486a1 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:24.970549 13770 ts_tablet_manager.cc:1428] T 62d6d3750baa41c193c39ca39c4486a1 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20260925 09:58:24.970778 13780 ts_tablet_manager.cc:1918] T e3247dedd2b0444e858de21f6c9aa3a5 P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:24.976572 13773 ts_tablet_manager.cc:1428] T a8a89d2315f54a09ad5d6506f2c4797c P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.040s user 0.006s sys 0.000s
I20260925 09:58:24.981436 13073 catalog_manager.cc:2148] created HMS catalog entry for table [id=a74c28d6faf4436385410132d67371eb]
I20260925 09:58:24.977330 13781 raft_consensus.cc:2229] T e3247dedd2b0444e858de21f6c9aa3a5 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:24.982196 13781 raft_consensus.cc:2258] T e3247dedd2b0444e858de21f6c9aa3a5 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:24.984197 13781 ts_tablet_manager.cc:1905] T e3247dedd2b0444e858de21f6c9aa3a5 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:24.971342 13770 tablet_bootstrap.cc:492] T e505eea45c414e82a69d7d965de6c814 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:24.989037 13780 log.cc:1196] T e3247dedd2b0444e858de21f6c9aa3a5 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/e3247dedd2b0444e858de21f6c9aa3a5
I20260925 09:58:24.989792 13780 ts_tablet_manager.cc:1939] T e3247dedd2b0444e858de21f6c9aa3a5 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:24.991680 14199 raft_consensus.cc:482] T 62d6d3750baa41c193c39ca39c4486a1 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:24.992075 14199 raft_consensus.cc:504] T 62d6d3750baa41c193c39ca39c4486a1 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:24.993850 14199 leader_election.cc:288] T 62d6d3750baa41c193c39ca39c4486a1 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:24.994444 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet e3247dedd2b0444e858de21f6c9aa3a5 (table default.table_ca5cce29b409479f9191e1b06eb5bac0 [id=db33e4ea5b1d4f0abedf9b9dde8614e2]) successfully deleted
I20260925 09:58:24.995831 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62d6d3750baa41c193c39ca39c4486a1" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:24.996337 13671 raft_consensus.cc:2454] T 62d6d3750baa41c193c39ca39c4486a1 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 0.
I20260925 09:58:24.997314 13440 leader_election.cc:302] T 62d6d3750baa41c193c39ca39c4486a1 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:24.997429 13361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62d6d3750baa41c193c39ca39c4486a1" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
W20260925 09:58:24.998739 13438 leader_election.cc:341] T 62d6d3750baa41c193c39ca39c4486a1 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:24.999542 14199 raft_consensus.cc:2781] T 62d6d3750baa41c193c39ca39c4486a1 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:24.999856 14199 raft_consensus.cc:482] T 62d6d3750baa41c193c39ca39c4486a1 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:25.000151 14199 raft_consensus.cc:3037] T 62d6d3750baa41c193c39ca39c4486a1 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:25.004612 13770 tablet_bootstrap.cc:654] T e505eea45c414e82a69d7d965de6c814 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:25.008181 13781 ts_tablet_manager.cc:1918] T e3247dedd2b0444e858de21f6c9aa3a5 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:25.008678 13781 log.cc:1196] T e3247dedd2b0444e858de21f6c9aa3a5 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/e3247dedd2b0444e858de21f6c9aa3a5
I20260925 09:58:25.009488 13269 catalog_manager.cc:2709] Deleting table default.table_81574dad60e9490cb79e268b12969a83 [id=8fa5a3419e394be2a52d1b13713a04d9] in response to Hive Metastore notification log event 27
I20260925 09:58:25.009516 13781 ts_tablet_manager.cc:1939] T e3247dedd2b0444e858de21f6c9aa3a5 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:25.012113 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet e3247dedd2b0444e858de21f6c9aa3a5 (table default.table_ca5cce29b409479f9191e1b06eb5bac0 [id=db33e4ea5b1d4f0abedf9b9dde8614e2]) successfully deleted
I20260925 09:58:25.013242 13343 tablet_service.cc:1514] Processing DeleteTablet for tablet e3247dedd2b0444e858de21f6c9aa3a5 with delete_type TABLET_DATA_DELETED (replaced by tablet 90d39744f6334e42bd32c26d13e6912c) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:25.014691 13777 tablet_replica.cc:330] T e3247dedd2b0444e858de21f6c9aa3a5 P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:25.015430 13777 raft_consensus.cc:2229] T e3247dedd2b0444e858de21f6c9aa3a5 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:25.015852 13777 raft_consensus.cc:2258] T e3247dedd2b0444e858de21f6c9aa3a5 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:25.018513 13777 ts_tablet_manager.cc:1905] T e3247dedd2b0444e858de21f6c9aa3a5 P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:25.019282 14221 catalog_manager.cc:1259] Loaded cluster ID: 85565ad29e6546888a40444865b055de
I20260925 09:58:25.025218 14221 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761: loading cluster ID for follower catalog manager: success
I20260925 09:58:25.010461 14199 raft_consensus.cc:504] T 62d6d3750baa41c193c39ca39c4486a1 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:25.034351 13073 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:50556 (request call id 94) took 1176 ms. Trace:
I20260925 09:58:25.034722 13073 rpcz_store.cc:276] 0925 09:58:23.858048 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:23.858608 (+ 560us) service_pool.cc:227] Handling call
0925 09:58:24.563102 (+704494us) catalog_manager.cc:2075] Acquired catalog manager lock
0925 09:58:24.564713 (+ 1611us) catalog_manager.cc:2121] Created new table and tablet info
0925 09:58:24.981395 (+416682us) catalog_manager.cc:2147] Created new table in HMS catalog
0925 09:58:25.002821 (+ 21426us) catalog_manager.cc:2181] Wrote table and tablets to system table
0925 09:58:25.034018 (+ 31197us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0925 09:58:25.034290 (+ 272us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:58:24.983017 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:58:24.983374 (+ 357us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:58:24.983426 (+ 52us) write_op.cc:435] Acquired schema lock
0925 09:58:24.983438 (+ 12us) tablet.cc:660] Decoding operations
0925 09:58:24.983687 (+ 249us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:58:24.983723 (+ 36us) write_op.cc:641] Partition lock acquired for write op
0925 09:58:24.983744 (+ 21us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:58:24.983924 (+ 180us) tablet.cc:699] Row locks acquired
0925 09:58:24.983937 (+ 13us) write_op.cc:260] PREPARE: finished
0925 09:58:24.984036 (+ 99us) write_op.cc:270] Start()
0925 09:58:24.984105 (+ 69us) write_op.cc:276] Timestamp: P: 1790330304983993 usec, L: 0
0925 09:58:24.984119 (+ 14us) op_driver.cc:331] REPLICATION: starting
0925 09:58:24.984625 (+ 506us) log.cc:842] Serialized 433 byte log entry
0925 09:58:24.993220 (+ 8595us) op_driver.cc:444] REPLICATION: finished
0925 09:58:25.000944 (+ 7724us) write_op.cc:301] APPLY: starting
0925 09:58:25.000998 (+ 54us) tablet.cc:1365] starting BulkCheckPresence
0925 09:58:25.001106 (+ 108us) tablet.cc:1368] finished BulkCheckPresence
0925 09:58:25.001206 (+ 100us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:58:25.001557 (+ 351us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:58:25.001685 (+ 128us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 09:58:25.001728 (+ 43us) write_op.cc:312] APPLY: finished
0925 09:58:25.001916 (+ 188us) log.cc:842] Serialized 29 byte log entry
0925 09:58:25.002527 (+ 611us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:25.002595 (+ 68us) write_op.cc:454] Released schema lock
0925 09:58:25.002631 (+ 36us) write_op.cc:341] FINISH: Updating metrics
0925 09:58:25.003240 (+ 609us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:25.003293 (+ 53us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":317124,"Hive Metastore.run_cpu_time_us":726,"Hive Metastore.run_wall_time_us":97981,"child_traces":[["op",{"apply.queue_time_us":7562,"apply.run_cpu_time_us":1675,"apply.run_wall_time_us":2608,"num_ops":2,"prepare.queue_time_us":224,"prepare.run_cpu_time_us":1769,"prepare.run_wall_time_us":1842,"raft.queue_time_us":1041,"raft.run_cpu_time_us":651,"raft.run_wall_time_us":17850,"replication_time_us":9027}]]}
I20260925 09:58:25.040102 13361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62d6d3750baa41c193c39ca39c4486a1" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843"
I20260925 09:58:25.041055 13671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62d6d3750baa41c193c39ca39c4486a1" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
W20260925 09:58:25.041473 13438 leader_election.cc:341] T 62d6d3750baa41c193c39ca39c4486a1 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:25.041594 13671 raft_consensus.cc:3037] T 62d6d3750baa41c193c39ca39c4486a1 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:25.042461 13073 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:50556:
table { table_name: "default.Table_e59da12e7b95405787a2f9f73efdc76b" } new_table_name: "default.Table_eda42a4c592f447b8084b26441b81b7c" modify_external_catalogs: true
I20260925 09:58:25.047513 13671 raft_consensus.cc:2454] T 62d6d3750baa41c193c39ca39c4486a1 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:25.048666 13440 leader_election.cc:302] T 62d6d3750baa41c193c39ca39c4486a1 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9; no voters: b3082cf15dc74c0480e5d33ecf592843
I20260925 09:58:25.049472 14196 raft_consensus.cc:2781] T 62d6d3750baa41c193c39ca39c4486a1 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:25.049887 14196 raft_consensus.cc:686] T 62d6d3750baa41c193c39ca39c4486a1 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: Becoming Leader. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Running, Role: LEADER
I20260925 09:58:25.050624 14196 consensus_queue.cc:237] T 62d6d3750baa41c193c39ca39c4486a1 P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:25.065477 13082 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet a3562966c8c34046af271a7c617cdb6b
I20260925 09:58:25.067158 13341 tablet_service.cc:1514] Processing DeleteTablet for tablet a3562966c8c34046af271a7c617cdb6b with delete_type TABLET_DATA_DELETED (replaced by tablet 3970efec7a0d4e7c9275c1f6b88ff4a9) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:25.072804 13482 tablet_service.cc:1514] Processing DeleteTablet for tablet a3562966c8c34046af271a7c617cdb6b with delete_type TABLET_DATA_DELETED (replaced by tablet 3970efec7a0d4e7c9275c1f6b88ff4a9) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:25.073635 13080 catalog_manager.cc:5587] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327) does not have the latest schema for tablet 62d6d3750baa41c193c39ca39c4486a1 (table default.table_d4409ab5331140e9a4bb995d57be4ec4 [id=f196d9d3750b4857905c1332d3ac763e]). Expected version 1 got 0
I20260925 09:58:25.074165 13781 tablet_replica.cc:330] T a3562966c8c34046af271a7c617cdb6b P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:25.074805 13781 raft_consensus.cc:2229] T a3562966c8c34046af271a7c617cdb6b P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:25.075218 13781 raft_consensus.cc:2258] T a3562966c8c34046af271a7c617cdb6b P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:25.075531 14199 leader_election.cc:288] T 62d6d3750baa41c193c39ca39c4486a1 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Requested vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:25.077785 13781 ts_tablet_manager.cc:1905] T a3562966c8c34046af271a7c617cdb6b P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:25.077782 13269 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet 0749453435d54676803789c380179af4
I20260925 09:58:25.079380 13341 tablet_service.cc:1514] Processing DeleteTablet for tablet 0749453435d54676803789c380179af4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:58:25 UTC) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:25.081614 13482 tablet_service.cc:1514] Processing DeleteTablet for tablet 0749453435d54676803789c380179af4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:58:25 UTC) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:25.084581 13648 tablet_service.cc:1514] Processing DeleteTablet for tablet a3562966c8c34046af271a7c617cdb6b with delete_type TABLET_DATA_DELETED (replaced by tablet 3970efec7a0d4e7c9275c1f6b88ff4a9) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:25.085474 13780 tablet_replica.cc:330] T a3562966c8c34046af271a7c617cdb6b P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:25.085721 13082 catalog_manager.cc:6565] ReplaceTablet: tablet a3562966c8c34046af271a7c617cdb6b deleted and replaced by tablet 3970efec7a0d4e7c9275c1f6b88ff4a9
I20260925 09:58:25.086205 13780 raft_consensus.cc:2229] T a3562966c8c34046af271a7c617cdb6b P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:25.086618 13648 tablet_service.cc:1514] Processing DeleteTablet for tablet a3562966c8c34046af271a7c617cdb6b with delete_type TABLET_DATA_DELETED (replaced by tablet 3970efec7a0d4e7c9275c1f6b88ff4a9) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:25.086683 13082 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50556:
name: "default.Table_688d7c9730444646b6d7a901d5537807"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:58:25.086710 13780 raft_consensus.cc:2258] T a3562966c8c34046af271a7c617cdb6b P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:25.087750 13772 tablet_bootstrap.cc:654] T a3562966c8c34046af271a7c617cdb6b P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:25.089298 13780 ts_tablet_manager.cc:1905] T a3562966c8c34046af271a7c617cdb6b P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 09:58:25.090744 13080 catalog_manager.cc:5687] Failed to send AlterTable RPC for tablet 62d6d3750baa41c193c39ca39c4486a1 (table default.table_d4409ab5331140e9a4bb995d57be4ec4 [id=f196d9d3750b4857905c1332d3ac763e], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 62d6d3750baa41c193c39ca39c4486a1 consensus state has no leader
I20260925 09:58:25.091670 13080 master_service.cc:946] ReplaceTablet: received request to replace tablet 0749453435d54676803789c380179af4 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:25.091312 13650 tablet_service.cc:1514] Processing DeleteTablet for tablet d2065bf05eb2489bb09736b2333bf215 with delete_type TABLET_DATA_DELETED (replaced by tablet ace8260d4ac94cafbb9147ebeb01ed85) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:25.104447 13649 tablet_service.cc:1514] Processing DeleteTablet for tablet e3247dedd2b0444e858de21f6c9aa3a5 with delete_type TABLET_DATA_DELETED (replaced by tablet 90d39744f6334e42bd32c26d13e6912c) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:25.105145 13651 tablet_service.cc:1514] Processing DeleteTablet for tablet 0749453435d54676803789c380179af4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:58:25 UTC) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:25.105865 13079 catalog_manager.cc:5587] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731) does not have the latest schema for tablet 62d6d3750baa41c193c39ca39c4486a1 (table default.table_d4409ab5331140e9a4bb995d57be4ec4 [id=f196d9d3750b4857905c1332d3ac763e]). Expected version 1 got 0
I20260925 09:58:25.110169 13777 ts_tablet_manager.cc:1918] T e3247dedd2b0444e858de21f6c9aa3a5 P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:25.113092 13772 tablet_bootstrap.cc:492] T a3562966c8c34046af271a7c617cdb6b P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:25.113534 13772 ts_tablet_manager.cc:1397] T a3562966c8c34046af271a7c617cdb6b P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.191s user 0.009s sys 0.000s
I20260925 09:58:25.110423 13080 catalog_manager.cc:5495] T 62d6d3750baa41c193c39ca39c4486a1 P c4d724d4b83e4d1e817a3a505dc24eb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2). New cstate: current_term: 1 leader_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: UNKNOWN } } }
I20260925 09:58:25.114989 13780 ts_tablet_manager.cc:1918] T a3562966c8c34046af271a7c617cdb6b P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:25.115509 13080 catalog_manager.cc:5587] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327) does not have the latest schema for tablet 62d6d3750baa41c193c39ca39c4486a1 (table default.table_d4409ab5331140e9a4bb995d57be4ec4 [id=f196d9d3750b4857905c1332d3ac763e]). Expected version 1 got 0
I20260925 09:58:25.115741 13772 raft_consensus.cc:348] T a3562966c8c34046af271a7c617cdb6b P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:25.118166 13772 raft_consensus.cc:374] T a3562966c8c34046af271a7c617cdb6b P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:25.118451 13772 raft_consensus.cc:729] T a3562966c8c34046af271a7c617cdb6b P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:25.119120 13772 consensus_queue.cc:260] T a3562966c8c34046af271a7c617cdb6b P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:25.120990 13781 ts_tablet_manager.cc:1918] T a3562966c8c34046af271a7c617cdb6b P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:25.121371 13781 log.cc:1196] T a3562966c8c34046af271a7c617cdb6b P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/a3562966c8c34046af271a7c617cdb6b
I20260925 09:58:25.122006 13781 ts_tablet_manager.cc:1939] T a3562966c8c34046af271a7c617cdb6b P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:25.116901 13770 tablet_bootstrap.cc:492] T e505eea45c414e82a69d7d965de6c814 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:25.122730 13770 ts_tablet_manager.cc:1397] T e505eea45c414e82a69d7d965de6c814 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.152s user 0.007s sys 0.008s
I20260925 09:58:25.124894 13770 raft_consensus.cc:348] T e505eea45c414e82a69d7d965de6c814 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:25.125578 13770 raft_consensus.cc:374] T e505eea45c414e82a69d7d965de6c814 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:25.125854 13770 raft_consensus.cc:729] T e505eea45c414e82a69d7d965de6c814 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:25.126466 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet a3562966c8c34046af271a7c617cdb6b (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]) successfully deleted
I20260925 09:58:25.126560 13770 consensus_queue.cc:260] T e505eea45c414e82a69d7d965de6c814 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:25.128422 13770 ts_tablet_manager.cc:1428] T e505eea45c414e82a69d7d965de6c814 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260925 09:58:25.128749 13780 log.cc:1196] T a3562966c8c34046af271a7c617cdb6b P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/a3562966c8c34046af271a7c617cdb6b
I20260925 09:58:25.129568 13780 ts_tablet_manager.cc:1939] T a3562966c8c34046af271a7c617cdb6b P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:25.130523 13770 tablet_bootstrap.cc:492] T a8a89d2315f54a09ad5d6506f2c4797c P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:25.134202 13781 tablet_replica.cc:330] T 0749453435d54676803789c380179af4 P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:25.135026 13781 raft_consensus.cc:2229] T 0749453435d54676803789c380179af4 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:25.135555 13781 raft_consensus.cc:2258] T 0749453435d54676803789c380179af4 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:25.137162 13770 tablet_bootstrap.cc:654] T a8a89d2315f54a09ad5d6506f2c4797c P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:25.137454 13781 ts_tablet_manager.cc:1905] T 0749453435d54676803789c380179af4 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:25.145764 13780 tablet_replica.cc:330] T d2065bf05eb2489bb09736b2333bf215 P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:25.150445 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet a3562966c8c34046af271a7c617cdb6b (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]) successfully deleted
W20260925 09:58:25.151086 13068 catalog_manager.cc:4808] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): delete failed for tablet a3562966c8c34046af271a7c617cdb6b because the tablet was not found. No further retry: Not found: Tablet not found: a3562966c8c34046af271a7c617cdb6b
I20260925 09:58:25.153147 13772 ts_tablet_manager.cc:1428] T a3562966c8c34046af271a7c617cdb6b P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.039s user 0.004s sys 0.000s
I20260925 09:58:25.154520 14196 consensus_queue.cc:1035] T 368f01226d6a4974acdd79006703a54c P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 09:58:25.155143 13438 consensus_peers.cc:487] T 62d6d3750baa41c193c39ca39c4486a1 P c4d724d4b83e4d1e817a3a505dc24eb9 -> Peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Couldn't send request to peer b3082cf15dc74c0480e5d33ecf592843. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:58:25.156261 13671 raft_consensus.cc:1261] T 62d6d3750baa41c193c39ca39c4486a1 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Refusing update from remote peer c4d724d4b83e4d1e817a3a505dc24eb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:58:25.156422 13772 tablet_bootstrap.cc:492] T 62d6d3750baa41c193c39ca39c4486a1 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:25.157825 14196 consensus_queue.cc:1035] T 62d6d3750baa41c193c39ca39c4486a1 P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:25.161337 13772 tablet_bootstrap.cc:654] T 62d6d3750baa41c193c39ca39c4486a1 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:25.166767 14015 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" instance_seqno: 1790330292494054) as {username='slave'} at 127.10.9.3:48505; Asking this server to re-register.
I20260925 09:58:25.168607 13770 tablet_bootstrap.cc:492] T a8a89d2315f54a09ad5d6506f2c4797c P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:25.169176 13770 ts_tablet_manager.cc:1397] T a8a89d2315f54a09ad5d6506f2c4797c P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.039s user 0.010s sys 0.001s
I20260925 09:58:25.171849 13770 raft_consensus.cc:348] T a8a89d2315f54a09ad5d6506f2c4797c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } }
I20260925 09:58:25.173040 13770 raft_consensus.cc:374] T a8a89d2315f54a09ad5d6506f2c4797c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:25.173336 13770 raft_consensus.cc:729] T a8a89d2315f54a09ad5d6506f2c4797c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:25.175207 13772 tablet_bootstrap.cc:492] T 62d6d3750baa41c193c39ca39c4486a1 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:25.173988 13770 consensus_queue.cc:260] T a8a89d2315f54a09ad5d6506f2c4797c P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } }
I20260925 09:58:25.175643 13772 ts_tablet_manager.cc:1397] T 62d6d3750baa41c193c39ca39c4486a1 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.019s user 0.007s sys 0.007s
I20260925 09:58:25.177733 13772 raft_consensus.cc:348] T 62d6d3750baa41c193c39ca39c4486a1 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:25.178467 13772 raft_consensus.cc:374] T 62d6d3750baa41c193c39ca39c4486a1 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:25.178730 13772 raft_consensus.cc:729] T 62d6d3750baa41c193c39ca39c4486a1 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:25.179369 13772 consensus_queue.cc:260] T 62d6d3750baa41c193c39ca39c4486a1 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:25.186630 13772 ts_tablet_manager.cc:1428] T 62d6d3750baa41c193c39ca39c4486a1 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.011s user 0.000s sys 0.004s
I20260925 09:58:25.188310 13770 ts_tablet_manager.cc:1428] T a8a89d2315f54a09ad5d6506f2c4797c P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.019s user 0.002s sys 0.003s
W20260925 09:58:25.188827 13269 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e505eea45c414e82a69d7d965de6c814 consensus state has no leader
I20260925 09:58:25.192401 13668 raft_consensus.cc:1261] T cedc5b0609e14c409542a83efc45f3f0 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Refusing update from remote peer c4d724d4b83e4d1e817a3a505dc24eb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:58:25.192492 13361 raft_consensus.cc:1261] T cedc5b0609e14c409542a83efc45f3f0 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Refusing update from remote peer c4d724d4b83e4d1e817a3a505dc24eb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:58:25.192775 13772 tablet_bootstrap.cc:492] T e505eea45c414e82a69d7d965de6c814 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:25.193921 14199 consensus_queue.cc:1035] T cedc5b0609e14c409542a83efc45f3f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:25.194693 14196 consensus_queue.cc:1035] T cedc5b0609e14c409542a83efc45f3f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:25.197770 13772 tablet_bootstrap.cc:654] T e505eea45c414e82a69d7d965de6c814 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:25.178268 14012 ts_manager.cc:194] Registered new tserver with Master: b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161)
I20260925 09:58:25.207789 14212 raft_consensus.cc:482] T 62d6d3750baa41c193c39ca39c4486a1 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:25.208271 14212 raft_consensus.cc:504] T 62d6d3750baa41c193c39ca39c4486a1 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:25.210192 14212 leader_election.cc:288] T 62d6d3750baa41c193c39ca39c4486a1 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:25.215049 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62d6d3750baa41c193c39ca39c4486a1" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
I20260925 09:58:25.216221 13670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62d6d3750baa41c193c39ca39c4486a1" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:25.217715 14211 raft_consensus.cc:482] T e505eea45c414e82a69d7d965de6c814 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:25.218134 14211 raft_consensus.cc:504] T e505eea45c414e82a69d7d965de6c814 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:25.219965 14211 leader_election.cc:288] T e505eea45c414e82a69d7d965de6c814 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
I20260925 09:58:25.110574 13777 log.cc:1196] T e3247dedd2b0444e858de21f6c9aa3a5 P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/e3247dedd2b0444e858de21f6c9aa3a5
I20260925 09:58:25.221766 13777 ts_tablet_manager.cc:1939] T e3247dedd2b0444e858de21f6c9aa3a5 P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
W20260925 09:58:25.224056 13067 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet e505eea45c414e82a69d7d965de6c814 (table default.table_1310f782dc864b068b28842f01c96b9c [id=dc9c88f862a64597aab300baf32f6f76], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet e505eea45c414e82a69d7d965de6c814 consensus state has no leader
I20260925 09:58:25.224068 13362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e505eea45c414e82a69d7d965de6c814" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:25.225239 13780 raft_consensus.cc:2229] T d2065bf05eb2489bb09736b2333bf215 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:25.225224 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e505eea45c414e82a69d7d965de6c814" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
I20260925 09:58:25.225798 13505 raft_consensus.cc:2454] T e505eea45c414e82a69d7d965de6c814 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 0.
I20260925 09:58:25.228317 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet e3247dedd2b0444e858de21f6c9aa3a5 (table default.table_ca5cce29b409479f9191e1b06eb5bac0 [id=db33e4ea5b1d4f0abedf9b9dde8614e2]) successfully deleted
I20260925 09:58:25.228523 13297 leader_election.cc:302] T 62d6d3750baa41c193c39ca39c4486a1 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:58:25.230825 14212 raft_consensus.cc:2726] T 62d6d3750baa41c193c39ca39c4486a1 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:58:25.234556 13298 consensus_peers.cc:487] T 0749453435d54676803789c380179af4 P b3082cf15dc74c0480e5d33ecf592843 -> Peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Couldn't send request to peer c4d724d4b83e4d1e817a3a505dc24eb9. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:58:25.243006 13075 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50556:
name: "default.Table_aa3ad1f6adf841ae9a2efad3eed2f203"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:58:25.246054 14221 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761: acquiring CA information for follower catalog manager: success
I20260925 09:58:25.251276 14199 consensus_queue.cc:1035] T 368f01226d6a4974acdd79006703a54c P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260925 09:58:25.256790 13772 tablet_bootstrap.cc:492] T e505eea45c414e82a69d7d965de6c814 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:25.257604 13772 ts_tablet_manager.cc:1397] T e505eea45c414e82a69d7d965de6c814 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.065s user 0.003s sys 0.013s
I20260925 09:58:25.261739 13772 raft_consensus.cc:348] T e505eea45c414e82a69d7d965de6c814 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:25.263230 13772 raft_consensus.cc:374] T e505eea45c414e82a69d7d965de6c814 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:25.263815 13772 raft_consensus.cc:729] T e505eea45c414e82a69d7d965de6c814 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:25.265056 13772 consensus_queue.cc:260] T e505eea45c414e82a69d7d965de6c814 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:25.269747 13772 ts_tablet_manager.cc:1428] T e505eea45c414e82a69d7d965de6c814 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20260925 09:58:25.271301 13772 tablet_bootstrap.cc:492] T a8a89d2315f54a09ad5d6506f2c4797c P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:25.280758 13772 tablet_bootstrap.cc:654] T a8a89d2315f54a09ad5d6506f2c4797c P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
W20260925 09:58:25.285976 13076 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_76fed9ed8d204d969883e54e5c7330c7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:58:25.226918 13606 leader_election.cc:302] T e505eea45c414e82a69d7d965de6c814 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:25.295697 14211 raft_consensus.cc:2781] T e505eea45c414e82a69d7d965de6c814 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260925 09:58:25.295599 13606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.2:44327, user_credentials={real_user=slave}} blocked reactor thread for 68759.2us
I20260925 09:58:25.296068 14211 raft_consensus.cc:482] T e505eea45c414e82a69d7d965de6c814 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260925 09:58:25.296238 13603 leader_election.cc:341] T e505eea45c414e82a69d7d965de6c814 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:25.296380 14211 raft_consensus.cc:3037] T e505eea45c414e82a69d7d965de6c814 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:58:25.297463 13603 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.1:42161, user_credentials={real_user=slave}} blocked reactor thread for 69609.8us
I20260925 09:58:25.298789 14202 raft_consensus.cc:482] T 0749453435d54676803789c380179af4 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:25.233525 13716 heartbeater.cc:464] Registering TS with master...
I20260925 09:58:25.299155 14202 raft_consensus.cc:504] T 0749453435d54676803789c380179af4 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:25.301867 14202 leader_election.cc:288] T 0749453435d54676803789c380179af4 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
I20260925 09:58:25.303068 14211 raft_consensus.cc:504] T e505eea45c414e82a69d7d965de6c814 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:25.303413 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0749453435d54676803789c380179af4" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
I20260925 09:58:25.304061 13780 raft_consensus.cc:2258] T d2065bf05eb2489bb09736b2333bf215 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:25.304433 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0749453435d54676803789c380179af4" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:25.305060 14211 leader_election.cc:288] T e505eea45c414e82a69d7d965de6c814 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 election: Requested vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
I20260925 09:58:25.305982 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e505eea45c414e82a69d7d965de6c814" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843"
W20260925 09:58:25.306196 13606 leader_election.cc:341] T 0749453435d54676803789c380179af4 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Illegal state: Tablet not RUNNING: STOPPED
I20260925 09:58:25.306448 13363 raft_consensus.cc:3037] T e505eea45c414e82a69d7d965de6c814 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:25.306964 13603 leader_election.cc:302] T 0749453435d54676803789c380179af4 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237; no voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:58:25.308598 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e505eea45c414e82a69d7d965de6c814" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9"
I20260925 09:58:25.309024 13780 ts_tablet_manager.cc:1905] T d2065bf05eb2489bb09736b2333bf215 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:25.309123 13505 raft_consensus.cc:3037] T e505eea45c414e82a69d7d965de6c814 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:25.312868 13772 tablet_bootstrap.cc:492] T a8a89d2315f54a09ad5d6506f2c4797c P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:25.313251 13772 ts_tablet_manager.cc:1397] T a8a89d2315f54a09ad5d6506f2c4797c P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.042s user 0.015s sys 0.002s
I20260925 09:58:25.315279 14202 raft_consensus.cc:2726] T 0749453435d54676803789c380179af4 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:58:25.315511 14018 raft_consensus.cc:482] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:25.315949 14018 raft_consensus.cc:504] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:25.318483 14018 leader_election.cc:288] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
I20260925 09:58:25.320986 13772 raft_consensus.cc:348] T a8a89d2315f54a09ad5d6506f2c4797c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } }
I20260925 09:58:25.321645 13772 raft_consensus.cc:374] T a8a89d2315f54a09ad5d6506f2c4797c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:25.321879 13772 raft_consensus.cc:729] T a8a89d2315f54a09ad5d6506f2c4797c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
W20260925 09:58:25.323818 13606 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
I20260925 09:58:25.334345 13342 tablet_service.cc:1467] Processing CreateTablet for tablet 2e7cc97b8b594e92bf3bf74131f20456 (DEFAULT_TABLE table=default.table_5f3286f12ae947ef8c1dfa71d8dd21d2 [id=07c6108c574e407f827bc4ab4ffdf345]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:25.335479 13342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e7cc97b8b594e92bf3bf74131f20456. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:25.335631 13772 consensus_queue.cc:260] T a8a89d2315f54a09ad5d6506f2c4797c P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } }
I20260925 09:58:25.339200 13482 tablet_service.cc:1467] Processing CreateTablet for tablet 2e7cc97b8b594e92bf3bf74131f20456 (DEFAULT_TABLE table=default.table_5f3286f12ae947ef8c1dfa71d8dd21d2 [id=07c6108c574e407f827bc4ab4ffdf345]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:25.340325 13482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e7cc97b8b594e92bf3bf74131f20456. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:25.346936 14221 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260925 09:58:25.350242 13603 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
I20260925 09:58:25.350607 13651 tablet_service.cc:1467] Processing CreateTablet for tablet 2e7cc97b8b594e92bf3bf74131f20456 (DEFAULT_TABLE table=default.table_5f3286f12ae947ef8c1dfa71d8dd21d2 [id=07c6108c574e407f827bc4ab4ffdf345]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:25.350857 13603 leader_election.cc:302] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237; no voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:58:25.351658 13651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e7cc97b8b594e92bf3bf74131f20456. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:25.363480 13363 raft_consensus.cc:2454] T e505eea45c414e82a69d7d965de6c814 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 1.
I20260925 09:58:25.365221 13603 leader_election.cc:302] T e505eea45c414e82a69d7d965de6c814 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, b3082cf15dc74c0480e5d33ecf592843; no voters:
I20260925 09:58:25.366068 14018 raft_consensus.cc:2726] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:58:25.366597 14018 raft_consensus.cc:2781] T e505eea45c414e82a69d7d965de6c814 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:25.367148 13770 tablet_bootstrap.cc:492] T 2e7cc97b8b594e92bf3bf74131f20456 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:25.367097 14018 raft_consensus.cc:686] T e505eea45c414e82a69d7d965de6c814 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 LEADER]: Becoming Leader. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Running, Role: LEADER
I20260925 09:58:25.368000 14018 consensus_queue.cc:237] T e505eea45c414e82a69d7d965de6c814 P a06c3ef7a6a94f7cbcc03e7cd232f237 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:25.373124 13770 tablet_bootstrap.cc:654] T 2e7cc97b8b594e92bf3bf74131f20456 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:25.374022 13505 raft_consensus.cc:2454] T e505eea45c414e82a69d7d965de6c814 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 1.
I20260925 09:58:25.383579 13081 catalog_manager.cc:2685] deleted HMS catalog entry for table default.table_2c3c283350d644f39f1903713ab06589 [id=fec42fd086714e079c23b924ca453b65]
I20260925 09:58:25.385224 14254 tablet.cc:1722] T cedc5b0609e14c409542a83efc45f3f0 P c4d724d4b83e4d1e817a3a505dc24eb9: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:58:25.388737 14255 tablet.cc:1722] T cedc5b0609e14c409542a83efc45f3f0 P a06c3ef7a6a94f7cbcc03e7cd232f237: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:58:25.397715 13777 tablet_replica.cc:330] T a3562966c8c34046af271a7c617cdb6b P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:25.399997 13777 raft_consensus.cc:2229] T a3562966c8c34046af271a7c617cdb6b P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:25.401575 13269 catalog_manager.cc:2709] Deleting table default.table_2c3c283350d644f39f1903713ab06589 [id=fec42fd086714e079c23b924ca453b65] in response to Hive Metastore notification log event 30
I20260925 09:58:25.403541 13343 tablet_service.cc:1467] Processing CreateTablet for tablet ace8260d4ac94cafbb9147ebeb01ed85 (DEFAULT_TABLE table=default.table_41304460d5a345c88866a7429d816aca [id=77d343c035fa4fd2b3faadcfc4fec52f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:25.404474 13343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ace8260d4ac94cafbb9147ebeb01ed85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:25.405468 13079 catalog_manager.cc:5495] T e505eea45c414e82a69d7d965de6c814 P a06c3ef7a6a94f7cbcc03e7cd232f237 reported cstate change: term changed from 0 to 1, leader changed from <none> to a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3). New cstate: current_term: 1 leader_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: HEALTHY } } }
I20260925 09:58:25.408205 13079 catalog_manager.cc:5587] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731) does not have the latest schema for tablet e505eea45c414e82a69d7d965de6c814 (table default.table_1310f782dc864b068b28842f01c96b9c [id=dc9c88f862a64597aab300baf32f6f76]). Expected version 1 got 0
I20260925 09:58:25.419248 14256 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b7d0c1c2f1ed4e7a9ed54e73869d53b0)
I20260925 09:58:25.419801 14256 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:25.425303 14012 ts_manager.cc:194] Registered new tserver with Master: a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:25.426271 13079 catalog_manager.cc:5587] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731) does not have the latest schema for tablet 62d6d3750baa41c193c39ca39c4486a1 (table default.table_d4409ab5331140e9a4bb995d57be4ec4 [id=f196d9d3750b4857905c1332d3ac763e]). Expected version 1 got 0
I20260925 09:58:25.452868 13777 raft_consensus.cc:2258] T a3562966c8c34046af271a7c617cdb6b P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:25.454715 13777 ts_tablet_manager.cc:1905] T a3562966c8c34046af271a7c617cdb6b P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:25.467562 13080 catalog_manager.cc:5587] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161) does not have the latest schema for tablet 62d6d3750baa41c193c39ca39c4486a1 (table default.table_d4409ab5331140e9a4bb995d57be4ec4 [id=f196d9d3750b4857905c1332d3ac763e]). Expected version 1 got 0
I20260925 09:58:25.468204 13080 catalog_manager.cc:5587] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161) does not have the latest schema for tablet cedc5b0609e14c409542a83efc45f3f0 (table default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]). Expected version 1 got 0
I20260925 09:58:25.468758 13080 catalog_manager.cc:5587] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161) does not have the latest schema for tablet e505eea45c414e82a69d7d965de6c814 (table default.table_1310f782dc864b068b28842f01c96b9c [id=dc9c88f862a64597aab300baf32f6f76]). Expected version 1 got 0
I20260925 09:58:25.471239 13269 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet 368f01226d6a4974acdd79006703a54c
I20260925 09:58:25.472103 13772 ts_tablet_manager.cc:1428] T a8a89d2315f54a09ad5d6506f2c4797c P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.159s user 0.014s sys 0.000s
I20260925 09:58:25.473506 13074 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:50556 (request call id 88) took 2299 ms. Trace:
I20260925 09:58:25.473907 13341 tablet_service.cc:1514] Processing DeleteTablet for tablet 368f01226d6a4974acdd79006703a54c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:58:25 UTC) from {username='slave'} at 127.0.0.1:49260
W20260925 09:58:25.474226 13077 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_840c9ea5fcc04f1b8811754058184452 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:58:25.473800 13074 rpcz_store.cc:276] 0925 09:58:23.173876 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:23.174011 (+ 135us) service_pool.cc:227] Handling call
0925 09:58:24.558305 (+1384294us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:58:25.473453 (+915148us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":85,"Hive Metastore.run_cpu_time_us":350,"Hive Metastore.run_wall_time_us":179198}
I20260925 09:58:25.475059 13074 master_service.cc:946] ReplaceTablet: received request to replace tablet cedc5b0609e14c409542a83efc45f3f0 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:25.478219 13078 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:50556 (request call id 89) took 2289 ms. Trace:
W20260925 09:58:25.478256 13082 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_688d7c9730444646b6d7a901d5537807 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:58:25.478569 13078 rpcz_store.cc:276] 0925 09:58:23.188518 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:23.215870 (+ 27352us) service_pool.cc:227] Handling call
0925 09:58:24.558421 (+1342551us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:58:25.476480 (+918059us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:58:25.477371 (+ 891us) catalog_manager.cc:3616] Apply alter schema
0925 09:58:25.478167 (+ 796us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":179130,"Hive Metastore.run_cpu_time_us":1696,"Hive Metastore.run_wall_time_us":144660}
I20260925 09:58:25.479908 13078 master_service.cc:946] ReplaceTablet: received request to replace tablet 8313c1ae006341bc84438085dd38f8ad from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:25.482301 13341 tablet_service.cc:1467] Processing CreateTablet for tablet 48ffc9cb12e346e7ae20c7f916ac33d4 (DEFAULT_TABLE table=default.table_8f71132b5a004ad89115f7a2278b95d6 [id=0cdc8d8fef5445668c0b0107243e0ae9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:25.483278 13341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48ffc9cb12e346e7ae20c7f916ac33d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:25.483898 13340 tablet_service.cc:1467] Processing CreateTablet for tablet 90d39744f6334e42bd32c26d13e6912c (DEFAULT_TABLE table=default.table_ca5cce29b409479f9191e1b06eb5bac0 [id=db33e4ea5b1d4f0abedf9b9dde8614e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:25.484927 13340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90d39744f6334e42bd32c26d13e6912c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:25.507325 13074 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet cedc5b0609e14c409542a83efc45f3f0
I20260925 09:58:25.509054 13339 tablet_service.cc:1514] Processing DeleteTablet for tablet cedc5b0609e14c409542a83efc45f3f0 with delete_type TABLET_DATA_DELETED (replaced by tablet d3efa619592045b8be65e6b789b8291a) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:25.509660 13074 catalog_manager.cc:6565] ReplaceTablet: tablet cedc5b0609e14c409542a83efc45f3f0 deleted and replaced by tablet d3efa619592045b8be65e6b789b8291a
I20260925 09:58:25.515928 13772 tablet_bootstrap.cc:492] T 90d39744f6334e42bd32c26d13e6912c P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:25.524286 13078 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet 8313c1ae006341bc84438085dd38f8ad
I20260925 09:58:25.524617 13362 raft_consensus.cc:1261] T e505eea45c414e82a69d7d965de6c814 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Refusing update from remote peer a06c3ef7a6a94f7cbcc03e7cd232f237: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:58:25.525573 13772 tablet_bootstrap.cc:654] T 90d39744f6334e42bd32c26d13e6912c P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:25.525815 13339 tablet_service.cc:1514] Processing DeleteTablet for tablet 8313c1ae006341bc84438085dd38f8ad with delete_type TABLET_DATA_DELETED (replaced by tablet 3d18f12528cf40a49a7cd0c5c97001bf) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:25.528527 13078 catalog_manager.cc:6565] ReplaceTablet: tablet 8313c1ae006341bc84438085dd38f8ad deleted and replaced by tablet 3d18f12528cf40a49a7cd0c5c97001bf
I20260925 09:58:25.529376 13017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "61e29afab0cf46819cbffc91248bf761" candidate_term: 5 candidate_status { last_received { term: 4 index: 129 } } ignore_live_leader: false dest_uuid: "70718e3f6f9f489089c0d8354380ed6c" is_pre_election: true
I20260925 09:58:25.533787 14270 tablet.cc:1722] T cedc5b0609e14c409542a83efc45f3f0 P b3082cf15dc74c0480e5d33ecf592843: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:58:25.552331 14256 leader_election.cc:288] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 70718e3f6f9f489089c0d8354380ed6c (127.10.9.62:43437), b7d0c1c2f1ed4e7a9ed54e73869d53b0 (127.10.9.61:40425)
I20260925 09:58:25.390048 14262 tablet.cc:1722] T 62d6d3750baa41c193c39ca39c4486a1 P c4d724d4b83e4d1e817a3a505dc24eb9: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:58:25.483196 13481 tablet_service.cc:1514] Processing DeleteTablet for tablet 368f01226d6a4974acdd79006703a54c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:58:25 UTC) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:25.484262 13484 tablet_service.cc:1467] Processing CreateTablet for tablet ace8260d4ac94cafbb9147ebeb01ed85 (DEFAULT_TABLE table=default.table_41304460d5a345c88866a7429d816aca [id=77d343c035fa4fd2b3faadcfc4fec52f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:25.553960 13090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "61e29afab0cf46819cbffc91248bf761" candidate_term: 5 candidate_status { last_received { term: 4 index: 129 } } ignore_live_leader: false dest_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" is_pre_election: true
I20260925 09:58:25.554425 13484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ace8260d4ac94cafbb9147ebeb01ed85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:25.555567 13955 leader_election.cc:302] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61e29afab0cf46819cbffc91248bf761; no voters: 70718e3f6f9f489089c0d8354380ed6c, b7d0c1c2f1ed4e7a9ed54e73869d53b0
I20260925 09:58:25.485208 13485 tablet_service.cc:1467] Processing CreateTablet for tablet 48ffc9cb12e346e7ae20c7f916ac33d4 (DEFAULT_TABLE table=default.table_8f71132b5a004ad89115f7a2278b95d6 [id=0cdc8d8fef5445668c0b0107243e0ae9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:25.560163 13485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48ffc9cb12e346e7ae20c7f916ac33d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:25.486968 13483 tablet_service.cc:1467] Processing CreateTablet for tablet 90d39744f6334e42bd32c26d13e6912c (DEFAULT_TABLE table=default.table_ca5cce29b409479f9191e1b06eb5bac0 [id=db33e4ea5b1d4f0abedf9b9dde8614e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:25.563979 13483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90d39744f6334e42bd32c26d13e6912c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:25.516187 13480 tablet_service.cc:1514] Processing DeleteTablet for tablet cedc5b0609e14c409542a83efc45f3f0 with delete_type TABLET_DATA_DELETED (replaced by tablet d3efa619592045b8be65e6b789b8291a) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:25.522424 13505 raft_consensus.cc:1261] T e505eea45c414e82a69d7d965de6c814 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Refusing update from remote peer a06c3ef7a6a94f7cbcc03e7cd232f237: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:58:25.532177 13479 tablet_service.cc:1514] Processing DeleteTablet for tablet 8313c1ae006341bc84438085dd38f8ad with delete_type TABLET_DATA_DELETED (replaced by tablet 3d18f12528cf40a49a7cd0c5c97001bf) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:25.578192 14256 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260925 09:58:25.586611 13772 tablet_bootstrap.cc:492] T 90d39744f6334e42bd32c26d13e6912c P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:25.587024 13772 ts_tablet_manager.cc:1397] T 90d39744f6334e42bd32c26d13e6912c P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.071s user 0.009s sys 0.002s
I20260925 09:58:25.589389 13772 raft_consensus.cc:348] T 90d39744f6334e42bd32c26d13e6912c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:25.590022 13772 raft_consensus.cc:374] T 90d39744f6334e42bd32c26d13e6912c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:25.590261 13772 raft_consensus.cc:729] T 90d39744f6334e42bd32c26d13e6912c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:25.590796 13772 consensus_queue.cc:260] T 90d39744f6334e42bd32c26d13e6912c P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:25.603729 13772 ts_tablet_manager.cc:1428] T 90d39744f6334e42bd32c26d13e6912c P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.016s user 0.004s sys 0.003s
W20260925 09:58:25.605965 13440 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.3:40731, user_credentials={real_user=slave}} blocked reactor thread for 216348us
I20260925 09:58:25.545480 13781 ts_tablet_manager.cc:1918] T 0749453435d54676803789c380179af4 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:25.607689 13781 log.cc:1196] T 0749453435d54676803789c380179af4 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/0749453435d54676803789c380179af4
I20260925 09:58:25.610224 13781 ts_tablet_manager.cc:1939] T 0749453435d54676803789c380179af4 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:25.455843 13671 raft_consensus.cc:3037] T 0749453435d54676803789c380179af4 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:25.477200 13651 tablet_service.cc:1514] Processing DeleteTablet for tablet 368f01226d6a4974acdd79006703a54c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:58:25 UTC) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:25.480561 13649 tablet_service.cc:1467] Processing CreateTablet for tablet ace8260d4ac94cafbb9147ebeb01ed85 (DEFAULT_TABLE table=default.table_41304460d5a345c88866a7429d816aca [id=77d343c035fa4fd2b3faadcfc4fec52f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:25.612443 13649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ace8260d4ac94cafbb9147ebeb01ed85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:25.612690 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet 0749453435d54676803789c380179af4 (table default.table_81574dad60e9490cb79e268b12969a83 [id=8fa5a3419e394be2a52d1b13713a04d9]) successfully deleted
I20260925 09:58:25.488929 13650 tablet_service.cc:1467] Processing CreateTablet for tablet 48ffc9cb12e346e7ae20c7f916ac33d4 (DEFAULT_TABLE table=default.table_8f71132b5a004ad89115f7a2278b95d6 [id=0cdc8d8fef5445668c0b0107243e0ae9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:25.616029 13650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48ffc9cb12e346e7ae20c7f916ac33d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:25.510613 13648 tablet_service.cc:1514] Processing DeleteTablet for tablet cedc5b0609e14c409542a83efc45f3f0 with delete_type TABLET_DATA_DELETED (replaced by tablet d3efa619592045b8be65e6b789b8291a) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:25.526348 14018 consensus_queue.cc:1035] T e505eea45c414e82a69d7d965de6c814 P a06c3ef7a6a94f7cbcc03e7cd232f237 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:25.528874 13647 tablet_service.cc:1514] Processing DeleteTablet for tablet 8313c1ae006341bc84438085dd38f8ad with delete_type TABLET_DATA_DELETED (replaced by tablet 3d18f12528cf40a49a7cd0c5c97001bf) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:25.620628 13770 tablet_bootstrap.cc:492] T 2e7cc97b8b594e92bf3bf74131f20456 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:25.620993 13770 ts_tablet_manager.cc:1397] T 2e7cc97b8b594e92bf3bf74131f20456 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.254s user 0.016s sys 0.022s
I20260925 09:58:25.623507 14211 consensus_queue.cc:1035] T e505eea45c414e82a69d7d965de6c814 P a06c3ef7a6a94f7cbcc03e7cd232f237 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:25.628087 13772 tablet_bootstrap.cc:492] T 48ffc9cb12e346e7ae20c7f916ac33d4 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:25.634495 13671 raft_consensus.cc:1261] T 0749453435d54676803789c380179af4 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Refusing update from remote peer b3082cf15dc74c0480e5d33ecf592843: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:58:25.643074 14199 raft_consensus.cc:482] T a8a89d2315f54a09ad5d6506f2c4797c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:25.643556 14199 raft_consensus.cc:504] T a8a89d2315f54a09ad5d6506f2c4797c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } }
I20260925 09:58:25.645012 13777 ts_tablet_manager.cc:1918] T a3562966c8c34046af271a7c617cdb6b P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:25.645362 13777 log.cc:1196] T a3562966c8c34046af271a7c617cdb6b P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/a3562966c8c34046af271a7c617cdb6b
I20260925 09:58:25.645406 14199 leader_election.cc:288] T a8a89d2315f54a09ad5d6506f2c4797c P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731), b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161)
I20260925 09:58:25.646013 13777 ts_tablet_manager.cc:1939] T a3562966c8c34046af271a7c617cdb6b P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:25.647161 13781 tablet_replica.cc:330] T 368f01226d6a4974acdd79006703a54c P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:25.642835 14272 tablet.cc:1722] T 62d6d3750baa41c193c39ca39c4486a1 P a06c3ef7a6a94f7cbcc03e7cd232f237: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:58:25.648010 13781 raft_consensus.cc:2229] T 368f01226d6a4974acdd79006703a54c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:58:25.649497 13781 raft_consensus.cc:2258] T 368f01226d6a4974acdd79006703a54c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:25.679032 13772 tablet_bootstrap.cc:654] T 48ffc9cb12e346e7ae20c7f916ac33d4 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:25.679630 13781 ts_tablet_manager.cc:1905] T 368f01226d6a4974acdd79006703a54c P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:25.683109 13080 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:50556:
table { table_name: "default.Table_5f3286f12ae947ef8c1dfa71d8dd21d2" } new_table_name: "default.Table_49a233eb03e6426081975d230536b222" modify_external_catalogs: true
I20260925 09:58:25.685746 14267 consensus_queue.cc:1035] T 0749453435d54676803789c380179af4 P b3082cf15dc74c0480e5d33ecf592843 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:25.689708 13780 ts_tablet_manager.cc:1918] T d2065bf05eb2489bb09736b2333bf215 P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:25.690441 13078 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50556:
table { table_name: "default.Table_41304460d5a345c88866a7429d816aca" } modify_external_catalogs: true
I20260925 09:58:25.691071 13780 log.cc:1196] T d2065bf05eb2489bb09736b2333bf215 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/d2065bf05eb2489bb09736b2333bf215
I20260925 09:58:25.691632 13078 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50556:
table { table_name: "default.Table_41304460d5a345c88866a7429d816aca" } modify_external_catalogs: true
I20260925 09:58:25.693941 13770 raft_consensus.cc:348] T 2e7cc97b8b594e92bf3bf74131f20456 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:25.694621 13770 raft_consensus.cc:374] T 2e7cc97b8b594e92bf3bf74131f20456 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:25.694865 13770 raft_consensus.cc:729] T 2e7cc97b8b594e92bf3bf74131f20456 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:25.695267 13780 ts_tablet_manager.cc:1939] T d2065bf05eb2489bb09736b2333bf215 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:25.694907 13076 catalog_manager.cc:2148] created HMS catalog entry for table [id=929b4a72ba9b4357929f4354783971d6]
I20260925 09:58:25.708530 13074 catalog_manager.cc:5587] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731) does not have the latest schema for tablet e505eea45c414e82a69d7d965de6c814 (table default.table_1310f782dc864b068b28842f01c96b9c [id=dc9c88f862a64597aab300baf32f6f76]). Expected version 1 got 0
I20260925 09:58:25.727303 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8a89d2315f54a09ad5d6506f2c4797c" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
W20260925 09:58:25.549636 14258 log.cc:924] Time spent T e505eea45c414e82a69d7d965de6c814 P a06c3ef7a6a94f7cbcc03e7cd232f237: Append to log took a long time: real 0.163s user 0.001s sys 0.000s
I20260925 09:58:25.583748 13646 tablet_service.cc:1467] Processing CreateTablet for tablet 90d39744f6334e42bd32c26d13e6912c (DEFAULT_TABLE table=default.table_ca5cce29b409479f9191e1b06eb5bac0 [id=db33e4ea5b1d4f0abedf9b9dde8614e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:25.729553 13363 raft_consensus.cc:2454] T a8a89d2315f54a09ad5d6506f2c4797c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 0.
I20260925 09:58:25.730698 13438 leader_election.cc:302] T a8a89d2315f54a09ad5d6506f2c4797c P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:25.731766 14199 raft_consensus.cc:2781] T a8a89d2315f54a09ad5d6506f2c4797c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:25.732095 14199 raft_consensus.cc:482] T a8a89d2315f54a09ad5d6506f2c4797c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:25.732388 14199 raft_consensus.cc:3037] T a8a89d2315f54a09ad5d6506f2c4797c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:25.670712 13668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8a89d2315f54a09ad5d6506f2c4797c" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:25.733214 13668 raft_consensus.cc:2454] T a8a89d2315f54a09ad5d6506f2c4797c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 0.
I20260925 09:58:25.700462 13770 consensus_queue.cc:260] T 2e7cc97b8b594e92bf3bf74131f20456 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:25.740077 14199 raft_consensus.cc:504] T a8a89d2315f54a09ad5d6506f2c4797c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } }
I20260925 09:58:25.745249 13669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8a89d2315f54a09ad5d6506f2c4797c" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
I20260925 09:58:25.745791 13669 raft_consensus.cc:3037] T a8a89d2315f54a09ad5d6506f2c4797c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:25.748226 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8a89d2315f54a09ad5d6506f2c4797c" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843"
I20260925 09:58:25.748827 13363 raft_consensus.cc:3037] T a8a89d2315f54a09ad5d6506f2c4797c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:25.754607 13646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90d39744f6334e42bd32c26d13e6912c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:25.754675 14199 leader_election.cc:288] T a8a89d2315f54a09ad5d6506f2c4797c P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Requested vote from peers a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731), b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161)
I20260925 09:58:25.758597 13770 ts_tablet_manager.cc:1428] T 2e7cc97b8b594e92bf3bf74131f20456 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.137s user 0.006s sys 0.001s
I20260925 09:58:25.760121 13770 tablet_bootstrap.cc:492] T ace8260d4ac94cafbb9147ebeb01ed85 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:25.763355 13777 tablet_replica.cc:330] T 0749453435d54676803789c380179af4 P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:25.763486 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet a3562966c8c34046af271a7c617cdb6b (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]) successfully deleted
I20260925 09:58:25.764122 13777 raft_consensus.cc:2229] T 0749453435d54676803789c380179af4 P b3082cf15dc74c0480e5d33ecf592843 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:58:25.764765 13777 pending_rounds.cc:62] T 0749453435d54676803789c380179af4 P b3082cf15dc74c0480e5d33ecf592843: Trying to abort 2 pending ops.
I20260925 09:58:25.765671 13079 master_service.cc:946] ReplaceTablet: received request to replace tablet a8a89d2315f54a09ad5d6506f2c4797c from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:25.770035 14274 tablet_bootstrap.cc:492] T 90d39744f6334e42bd32c26d13e6912c P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:25.786489 13076 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:50556 (request call id 96) took 1744 ms. Trace:
I20260925 09:58:25.786827 13076 rpcz_store.cc:276] 0925 09:58:24.041648 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:24.041787 (+ 139us) service_pool.cc:227] Handling call
0925 09:58:25.286461 (+1244674us) catalog_manager.cc:2075] Acquired catalog manager lock
0925 09:58:25.288236 (+ 1775us) catalog_manager.cc:2121] Created new table and tablet info
0925 09:58:25.694871 (+406635us) catalog_manager.cc:2147] Created new table in HMS catalog
0925 09:58:25.785787 (+ 90916us) catalog_manager.cc:2181] Wrote table and tablets to system table
0925 09:58:25.786137 (+ 350us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0925 09:58:25.786430 (+ 293us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:58:25.698275 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:58:25.698611 (+ 336us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:58:25.698634 (+ 23us) write_op.cc:435] Acquired schema lock
0925 09:58:25.698647 (+ 13us) tablet.cc:660] Decoding operations
0925 09:58:25.698903 (+ 256us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:58:25.698940 (+ 37us) write_op.cc:641] Partition lock acquired for write op
0925 09:58:25.698961 (+ 21us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:58:25.699134 (+ 173us) tablet.cc:699] Row locks acquired
0925 09:58:25.699148 (+ 14us) write_op.cc:260] PREPARE: finished
0925 09:58:25.699249 (+ 101us) write_op.cc:270] Start()
0925 09:58:25.699321 (+ 72us) write_op.cc:276] Timestamp: P: 1790330305699070 usec, L: 0
0925 09:58:25.699336 (+ 15us) op_driver.cc:331] REPLICATION: starting
0925 09:58:25.700270 (+ 934us) log.cc:842] Serialized 433 byte log entry
0925 09:58:25.784261 (+ 83991us) op_driver.cc:444] REPLICATION: finished
0925 09:58:25.784541 (+ 280us) write_op.cc:301] APPLY: starting
0925 09:58:25.784589 (+ 48us) tablet.cc:1365] starting BulkCheckPresence
0925 09:58:25.784675 (+ 86us) tablet.cc:1368] finished BulkCheckPresence
0925 09:58:25.784753 (+ 78us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:58:25.785074 (+ 321us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:58:25.785186 (+ 112us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 09:58:25.785222 (+ 36us) write_op.cc:312] APPLY: finished
0925 09:58:25.785367 (+ 145us) log.cc:842] Serialized 29 byte log entry
0925 09:58:25.785496 (+ 129us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:25.785556 (+ 60us) write_op.cc:454] Released schema lock
0925 09:58:25.785582 (+ 26us) write_op.cc:341] FINISH: Updating metrics
0925 09:58:25.785743 (+ 161us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:25.785764 (+ 21us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":109144,"Hive Metastore.run_cpu_time_us":554,"Hive Metastore.run_wall_time_us":296307,"child_traces":[["op",{"apply.queue_time_us":156,"apply.run_cpu_time_us":1361,"apply.run_wall_time_us":1361,"num_ops":2,"prepare.queue_time_us":1497,"prepare.run_cpu_time_us":1699,"prepare.run_wall_time_us":2158,"replication_time_us":84845}]]}
I20260925 09:58:25.794952 13772 tablet_bootstrap.cc:492] T 48ffc9cb12e346e7ae20c7f916ac33d4 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:25.795373 13772 ts_tablet_manager.cc:1397] T 48ffc9cb12e346e7ae20c7f916ac33d4 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.168s user 0.010s sys 0.004s
I20260925 09:58:25.797151 13076 catalog_manager.cc:5587] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327) does not have the latest schema for tablet e505eea45c414e82a69d7d965de6c814 (table default.table_1310f782dc864b068b28842f01c96b9c [id=dc9c88f862a64597aab300baf32f6f76]). Expected version 1 got 0
I20260925 09:58:25.797494 13772 raft_consensus.cc:348] T 48ffc9cb12e346e7ae20c7f916ac33d4 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:25.798136 13772 raft_consensus.cc:374] T 48ffc9cb12e346e7ae20c7f916ac33d4 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:25.798375 13772 raft_consensus.cc:729] T 48ffc9cb12e346e7ae20c7f916ac33d4 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:25.798951 13772 consensus_queue.cc:260] T 48ffc9cb12e346e7ae20c7f916ac33d4 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:25.805747 13772 ts_tablet_manager.cc:1428] T 48ffc9cb12e346e7ae20c7f916ac33d4 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.010s user 0.004s sys 0.000s
I20260925 09:58:25.764983 13777 pending_rounds.cc:68] T 0749453435d54676803789c380179af4 P b3082cf15dc74c0480e5d33ecf592843: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7333192930640949248 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "0749453435d54676803789c380179af4" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 1 new_table_name: "default.table_81574dad60e9490cb79e268b12969a83" dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" new_extra_config { } }
W20260925 09:58:25.807999 13777 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:49260: Aborted: Op aborted
I20260925 09:58:25.808856 13777 pending_rounds.cc:68] T 0749453435d54676803789c380179af4 P b3082cf15dc74c0480e5d33ecf592843: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192926758592512 op_type: NO_OP noop_request { }
I20260925 09:58:25.809218 13777 raft_consensus.cc:2866] T 0749453435d54676803789c380179af4 P b3082cf15dc74c0480e5d33ecf592843 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:58:25.809516 13777 raft_consensus.cc:2258] T 0749453435d54676803789c380179af4 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:25.807096 13772 tablet_bootstrap.cc:492] T 2e7cc97b8b594e92bf3bf74131f20456 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:25.828617 13669 raft_consensus.cc:2454] T a8a89d2315f54a09ad5d6506f2c4797c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:25.830806 13770 tablet_bootstrap.cc:654] T ace8260d4ac94cafbb9147ebeb01ed85 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:25.832047 13781 ts_tablet_manager.cc:1918] T 368f01226d6a4974acdd79006703a54c P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:25.831542 13074 master_service.cc:946] ReplaceTablet: received request to replace tablet dedf3e0a273d43ba84a861b36d0707ef from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:25.832434 13781 log.cc:1196] T 368f01226d6a4974acdd79006703a54c P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/368f01226d6a4974acdd79006703a54c
I20260925 09:58:25.834149 13781 ts_tablet_manager.cc:1939] T 368f01226d6a4974acdd79006703a54c P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:25.834877 13079 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet a8a89d2315f54a09ad5d6506f2c4797c
I20260925 09:58:25.841547 14272 tablet.cc:1722] T e505eea45c414e82a69d7d965de6c814 P a06c3ef7a6a94f7cbcc03e7cd232f237: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:58:25.846282 13650 tablet_service.cc:1514] Processing DeleteTablet for tablet a8a89d2315f54a09ad5d6506f2c4797c with delete_type TABLET_DATA_DELETED (replaced by tablet fafc2b657ace4a4381452ce43fad39b6) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:25.846338 13777 ts_tablet_manager.cc:1905] T 0749453435d54676803789c380179af4 P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:25.851708 14262 tablet.cc:1722] T e505eea45c414e82a69d7d965de6c814 P c4d724d4b83e4d1e817a3a505dc24eb9: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:58:25.856768 13483 tablet_service.cc:1514] Processing DeleteTablet for tablet a8a89d2315f54a09ad5d6506f2c4797c with delete_type TABLET_DATA_DELETED (replaced by tablet fafc2b657ace4a4381452ce43fad39b6) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:25.857513 13079 catalog_manager.cc:6565] ReplaceTablet: tablet a8a89d2315f54a09ad5d6506f2c4797c deleted and replaced by tablet fafc2b657ace4a4381452ce43fad39b6
I20260925 09:58:25.824561 13363 raft_consensus.cc:2454] T a8a89d2315f54a09ad5d6506f2c4797c P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:25.862126 14274 tablet_bootstrap.cc:654] T 90d39744f6334e42bd32c26d13e6912c P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:25.862946 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet 368f01226d6a4974acdd79006703a54c (table default.table_2c3c283350d644f39f1903713ab06589 [id=fec42fd086714e079c23b924ca453b65]) successfully deleted
I20260925 09:58:25.864224 13781 tablet_replica.cc:330] T cedc5b0609e14c409542a83efc45f3f0 P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:25.867833 13772 tablet_bootstrap.cc:654] T 2e7cc97b8b594e92bf3bf74131f20456 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:25.868666 14211 raft_consensus.cc:482] T 2e7cc97b8b594e92bf3bf74131f20456 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:25.869100 14211 raft_consensus.cc:504] T 2e7cc97b8b594e92bf3bf74131f20456 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:25.870164 13781 raft_consensus.cc:2229] T cedc5b0609e14c409542a83efc45f3f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:58:25.870908 14211 leader_election.cc:288] T 2e7cc97b8b594e92bf3bf74131f20456 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
I20260925 09:58:25.872368 13781 raft_consensus.cc:2258] T cedc5b0609e14c409542a83efc45f3f0 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:25.872783 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e7cc97b8b594e92bf3bf74131f20456" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
W20260925 09:58:25.874418 13606 leader_election.cc:341] T 2e7cc97b8b594e92bf3bf74131f20456 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:25.876766 13076 catalog_manager.cc:6040] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: default.table_d4409ab5331140e9a4bb995d57be4ec4 [id=f196d9d3750b4857905c1332d3ac763e] alter complete (version 1)
I20260925 09:58:25.879423 13483 tablet_service.cc:1514] Processing DeleteTablet for tablet 368f01226d6a4974acdd79006703a54c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 09:58:25 UTC) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:25.893584 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet d2065bf05eb2489bb09736b2333bf215 (table default.table_41304460d5a345c88866a7429d816aca [id=77d343c035fa4fd2b3faadcfc4fec52f]) successfully deleted
I20260925 09:58:25.895289 13440 leader_election.cc:302] T a8a89d2315f54a09ad5d6506f2c4797c P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:25.896381 14199 raft_consensus.cc:2781] T a8a89d2315f54a09ad5d6506f2c4797c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:25.896878 14199 raft_consensus.cc:686] T a8a89d2315f54a09ad5d6506f2c4797c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: Becoming Leader. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Running, Role: LEADER
I20260925 09:58:25.897672 14199 consensus_queue.cc:237] T a8a89d2315f54a09ad5d6506f2c4797c P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } }
I20260925 09:58:25.904814 13074 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet dedf3e0a273d43ba84a861b36d0707ef
I20260925 09:58:25.904992 13781 ts_tablet_manager.cc:1905] T cedc5b0609e14c409542a83efc45f3f0 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:25.921118 13483 tablet_service.cc:1514] Processing DeleteTablet for tablet dedf3e0a273d43ba84a861b36d0707ef with delete_type TABLET_DATA_DELETED (replaced by tablet 4b61aaac33514c999482b824e0839a72) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:25.922619 13074 catalog_manager.cc:6565] ReplaceTablet: tablet dedf3e0a273d43ba84a861b36d0707ef deleted and replaced by tablet 4b61aaac33514c999482b824e0839a72
I20260925 09:58:25.922812 13646 tablet_service.cc:1514] Processing DeleteTablet for tablet dedf3e0a273d43ba84a861b36d0707ef with delete_type TABLET_DATA_DELETED (replaced by tablet 4b61aaac33514c999482b824e0839a72) from {username='slave'} at 127.0.0.1:54548
W20260925 09:58:25.924690 13068 catalog_manager.cc:4808] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): delete failed for tablet e3247dedd2b0444e858de21f6c9aa3a5 because the tablet was not found. No further retry: Not found: Tablet not found: e3247dedd2b0444e858de21f6c9aa3a5
I20260925 09:58:25.925846 13780 tablet_replica.cc:330] T 0749453435d54676803789c380179af4 P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:25.926301 13770 tablet_bootstrap.cc:492] T ace8260d4ac94cafbb9147ebeb01ed85 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:25.928016 13079 catalog_manager.cc:6040] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: default.table_1310f782dc864b068b28842f01c96b9c [id=dc9c88f862a64597aab300baf32f6f76] alter complete (version 1)
I20260925 09:58:25.929337 13770 ts_tablet_manager.cc:1397] T ace8260d4ac94cafbb9147ebeb01ed85 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.170s user 0.011s sys 0.001s
I20260925 09:58:25.931932 13770 raft_consensus.cc:348] T ace8260d4ac94cafbb9147ebeb01ed85 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:25.932578 13770 raft_consensus.cc:374] T ace8260d4ac94cafbb9147ebeb01ed85 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:25.932833 13770 raft_consensus.cc:729] T ace8260d4ac94cafbb9147ebeb01ed85 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:25.933580 13770 consensus_queue.cc:260] T ace8260d4ac94cafbb9147ebeb01ed85 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:25.967165 13770 ts_tablet_manager.cc:1428] T ace8260d4ac94cafbb9147ebeb01ed85 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.038s user 0.005s sys 0.000s
I20260925 09:58:25.977439 13770 tablet_bootstrap.cc:492] T 48ffc9cb12e346e7ae20c7f916ac33d4 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:26.001739 13074 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50556:
name: "default.Table_1b826b3631274aca84923893f7383ee3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:58:26.004281 13361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e7cc97b8b594e92bf3bf74131f20456" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:26.008396 13772 tablet_bootstrap.cc:492] T 2e7cc97b8b594e92bf3bf74131f20456 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:26.008549 13777 ts_tablet_manager.cc:1918] T 0749453435d54676803789c380179af4 P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:58:26.009179 13772 ts_tablet_manager.cc:1397] T 2e7cc97b8b594e92bf3bf74131f20456 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.202s user 0.009s sys 0.002s
I20260925 09:58:26.009246 13777 log.cc:1196] T 0749453435d54676803789c380179af4 P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/0749453435d54676803789c380179af4
I20260925 09:58:26.011279 13780 raft_consensus.cc:2229] T 0749453435d54676803789c380179af4 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:26.013351 13646 tablet_service.cc:1467] Processing CreateTablet for tablet 0e3f1d9381b4450cb6c15525699efbc9 (DEFAULT_TABLE table=default.table_afc7cdc28ab84dff9a8782192aa34919 [id=a74c28d6faf4436385410132d67371eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:26.015657 13650 tablet_service.cc:1467] Processing CreateTablet for tablet 3d18f12528cf40a49a7cd0c5c97001bf (DEFAULT_TABLE table=default.table_1238f9df428247dc9b9ca7c59823bfe4 [id=5eb4ef6bf2c344bdb26e429f484c59a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:26.018528 13770 tablet_bootstrap.cc:654] T 48ffc9cb12e346e7ae20c7f916ac33d4 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:26.019158 13650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d18f12528cf40a49a7cd0c5c97001bf. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:58:26.025837 13603 leader_election.cc:341] T 2e7cc97b8b594e92bf3bf74131f20456 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:26.026271 13603 leader_election.cc:302] T 2e7cc97b8b594e92bf3bf74131f20456 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237; no voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:58:26.027488 14211 raft_consensus.cc:2726] T 2e7cc97b8b594e92bf3bf74131f20456 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:58:26.028728 13646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e3f1d9381b4450cb6c15525699efbc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:26.045711 14274 tablet_bootstrap.cc:492] T 90d39744f6334e42bd32c26d13e6912c P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:26.046286 14274 ts_tablet_manager.cc:1397] T 90d39744f6334e42bd32c26d13e6912c P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.277s user 0.006s sys 0.007s
I20260925 09:58:26.047251 13649 tablet_service.cc:1467] Processing CreateTablet for tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 (DEFAULT_TABLE table=default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:26.048267 13649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3970efec7a0d4e7c9275c1f6b88ff4a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:26.049490 13647 tablet_service.cc:1467] Processing CreateTablet for tablet d3efa619592045b8be65e6b789b8291a (DEFAULT_TABLE table=default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:26.050581 13647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3efa619592045b8be65e6b789b8291a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:58:26.041884 13438 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.9.1:42161, user_credentials={real_user=slave}} blocked reactor thread for 49262.7us
I20260925 09:58:26.057041 13483 tablet_service.cc:1467] Processing CreateTablet for tablet 0e3f1d9381b4450cb6c15525699efbc9 (DEFAULT_TABLE table=default.table_afc7cdc28ab84dff9a8782192aa34919 [id=a74c28d6faf4436385410132d67371eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:26.057315 13485 tablet_service.cc:1467] Processing CreateTablet for tablet 3d18f12528cf40a49a7cd0c5c97001bf (DEFAULT_TABLE table=default.table_1238f9df428247dc9b9ca7c59823bfe4 [id=5eb4ef6bf2c344bdb26e429f484c59a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:26.058018 13483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e3f1d9381b4450cb6c15525699efbc9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:58:26.058135 14284 log.cc:924] Time spent T a8a89d2315f54a09ad5d6506f2c4797c P c4d724d4b83e4d1e817a3a505dc24eb9: Append to log took a long time: real 0.106s user 0.001s sys 0.000s
I20260925 09:58:26.058632 13484 tablet_service.cc:1467] Processing CreateTablet for tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 (DEFAULT_TABLE table=default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:26.059538 13484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3970efec7a0d4e7c9275c1f6b88ff4a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:26.059932 14274 raft_consensus.cc:348] T 90d39744f6334e42bd32c26d13e6912c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.060602 14274 raft_consensus.cc:374] T 90d39744f6334e42bd32c26d13e6912c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:26.060847 14274 raft_consensus.cc:729] T 90d39744f6334e42bd32c26d13e6912c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:26.061508 14274 consensus_queue.cc:260] T 90d39744f6334e42bd32c26d13e6912c P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.039265 13781 ts_tablet_manager.cc:1918] T cedc5b0609e14c409542a83efc45f3f0 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:58:26.063144 13781 log.cc:1196] T cedc5b0609e14c409542a83efc45f3f0 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/cedc5b0609e14c409542a83efc45f3f0
I20260925 09:58:26.063550 14274 ts_tablet_manager.cc:1428] T 90d39744f6334e42bd32c26d13e6912c P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.017s user 0.001s sys 0.005s
I20260925 09:58:26.063484 13482 tablet_service.cc:1467] Processing CreateTablet for tablet d3efa619592045b8be65e6b789b8291a (DEFAULT_TABLE table=default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:26.076841 14274 tablet_bootstrap.cc:492] T 2e7cc97b8b594e92bf3bf74131f20456 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:26.089289 13780 pending_rounds.cc:62] T 0749453435d54676803789c380179af4 P a06c3ef7a6a94f7cbcc03e7cd232f237: Trying to abort 2 pending ops.
I20260925 09:58:26.089841 13780 pending_rounds.cc:68] T 0749453435d54676803789c380179af4 P a06c3ef7a6a94f7cbcc03e7cd232f237: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7333192930640949248 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "0749453435d54676803789c380179af4" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 1 new_table_name: "default.table_81574dad60e9490cb79e268b12969a83" dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" new_extra_config { } }
I20260925 09:58:26.063937 13781 ts_tablet_manager.cc:1939] T cedc5b0609e14c409542a83efc45f3f0 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:26.066926 13485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d18f12528cf40a49a7cd0c5c97001bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:26.098482 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet cedc5b0609e14c409542a83efc45f3f0 (table default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]) successfully deleted
I20260925 09:58:26.099232 13482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3efa619592045b8be65e6b789b8291a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:26.009739 14270 tablet.cc:1722] T e505eea45c414e82a69d7d965de6c814 P b3082cf15dc74c0480e5d33ecf592843: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:58:26.121520 13781 tablet_replica.cc:330] T 8313c1ae006341bc84438085dd38f8ad P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:26.122588 13781 raft_consensus.cc:2229] T 8313c1ae006341bc84438085dd38f8ad P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:58:26.122826 13777 ts_tablet_manager.cc:1939] T 0749453435d54676803789c380179af4 P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:26.123883 13781 raft_consensus.cc:2258] T 8313c1ae006341bc84438085dd38f8ad P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:26.125680 13781 ts_tablet_manager.cc:1905] T 8313c1ae006341bc84438085dd38f8ad P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:26.127095 13343 tablet_service.cc:1514] Processing DeleteTablet for tablet a8a89d2315f54a09ad5d6506f2c4797c with delete_type TABLET_DATA_DELETED (replaced by tablet fafc2b657ace4a4381452ce43fad39b6) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:26.128114 13342 tablet_service.cc:1514] Processing DeleteTablet for tablet dedf3e0a273d43ba84a861b36d0707ef with delete_type TABLET_DATA_DELETED (replaced by tablet 4b61aaac33514c999482b824e0839a72) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:26.129838 13339 tablet_service.cc:1467] Processing CreateTablet for tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 (DEFAULT_TABLE table=default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:26.136518 13338 tablet_service.cc:1467] Processing CreateTablet for tablet d3efa619592045b8be65e6b789b8291a (DEFAULT_TABLE table=default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:26.138636 13341 tablet_service.cc:1467] Processing CreateTablet for tablet 3d18f12528cf40a49a7cd0c5c97001bf (DEFAULT_TABLE table=default.table_1238f9df428247dc9b9ca7c59823bfe4 [id=5eb4ef6bf2c344bdb26e429f484c59a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:26.139729 13340 tablet_service.cc:1467] Processing CreateTablet for tablet 0e3f1d9381b4450cb6c15525699efbc9 (DEFAULT_TABLE table=default.table_afc7cdc28ab84dff9a8782192aa34919 [id=a74c28d6faf4436385410132d67371eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:26.091655 13780 pending_rounds.cc:68] T 0749453435d54676803789c380179af4 P a06c3ef7a6a94f7cbcc03e7cd232f237: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192926758592512 op_type: NO_OP noop_request { }
I20260925 09:58:26.146329 13780 raft_consensus.cc:2866] T 0749453435d54676803789c380179af4 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:58:26.146763 13780 raft_consensus.cc:2258] T 0749453435d54676803789c380179af4 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:26.159734 13777 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49260 (request call id 132) took 1080 ms. Trace:
I20260925 09:58:26.163703 13777 rpcz_store.cc:276] 0925 09:58:25.079131 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:25.079258 (+ 127us) service_pool.cc:227] Handling call
0925 09:58:25.763211 (+683953us) ts_tablet_manager.cc:1145] Deleting tablet 0749453435d54676803789c380179af4
0925 09:58:25.763248 (+ 37us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:58:25.896320 (+133072us) tablet_metadata.cc:641] Metadata flushed
0925 09:58:26.008113 (+111793us) tablet_metadata.cc:641] Metadata flushed
0925 09:58:26.159658 (+151545us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":683442}
I20260925 09:58:26.166371 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet 0749453435d54676803789c380179af4 (table default.table_81574dad60e9490cb79e268b12969a83 [id=8fa5a3419e394be2a52d1b13713a04d9]) successfully deleted
I20260925 09:58:26.173374 13772 raft_consensus.cc:348] T 2e7cc97b8b594e92bf3bf74131f20456 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.174404 13772 raft_consensus.cc:374] T 2e7cc97b8b594e92bf3bf74131f20456 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:26.174679 13341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d18f12528cf40a49a7cd0c5c97001bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:26.176316 13780 ts_tablet_manager.cc:1905] T 0749453435d54676803789c380179af4 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:26.181779 13363 raft_consensus.cc:3037] T 62d6d3750baa41c193c39ca39c4486a1 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:26.174705 13772 raft_consensus.cc:729] T 2e7cc97b8b594e92bf3bf74131f20456 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:26.192024 13772 consensus_queue.cc:260] T 2e7cc97b8b594e92bf3bf74131f20456 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.192667 13077 catalog_manager.cc:2148] created HMS catalog entry for table [id=84e4b25904ae485fbf3816bd06643b42]
I20260925 09:58:26.194490 13772 ts_tablet_manager.cc:1428] T 2e7cc97b8b594e92bf3bf74131f20456 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.185s user 0.003s sys 0.005s
I20260925 09:58:26.158854 14274 tablet_bootstrap.cc:654] T 2e7cc97b8b594e92bf3bf74131f20456 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:26.206614 13079 master_service.cc:946] ReplaceTablet: received request to replace tablet cedc5b0609e14c409542a83efc45f3f0 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:26.208134 13340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e3f1d9381b4450cb6c15525699efbc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:26.211890 13338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3efa619592045b8be65e6b789b8291a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:26.216343 13339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3970efec7a0d4e7c9275c1f6b88ff4a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:26.219672 13777 tablet_replica.cc:330] T 368f01226d6a4974acdd79006703a54c P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:26.220433 13777 raft_consensus.cc:2229] T 368f01226d6a4974acdd79006703a54c P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:26.220925 13777 raft_consensus.cc:2258] T 368f01226d6a4974acdd79006703a54c P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:26.225970 13077 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:50556 (request call id 101) took 1736 ms. Trace:
I20260925 09:58:26.226439 13076 master_service.cc:946] ReplaceTablet: received request to replace tablet cedc5b0609e14c409542a83efc45f3f0 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:26.228295 13777 ts_tablet_manager.cc:1905] T 368f01226d6a4974acdd79006703a54c P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 09:58:26.230269 13076 catalog_manager.cc:5328] Ignoring report from unknown tablet dedf3e0a273d43ba84a861b36d0707ef
I20260925 09:58:26.226255 13077 rpcz_store.cc:276] 0925 09:58:24.489590 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:24.561526 (+ 71936us) service_pool.cc:227] Handling call
0925 09:58:24.634411 (+ 72885us) spinlock_profiling.cc:243] Waited 70.5 ms on lock 0x7b5400021554. stack: 00007fc37b38f033 00007fc37b38ef30 00007fc37af10534 00000000004db2b4 00000000004db249 00007fc38229baf0 00007fc38231ef2a 00007fc3841ead6c 00007fc3842e6156 00007fc381e23034 00007fc381e22fc2 00007fc381e22ef1 00007fc381e22e6c 00007fc381e22122 00007fc3800c594c 00007fc3800c4d86
0925 09:58:25.474589 (+840178us) catalog_manager.cc:2075] Acquired catalog manager lock
0925 09:58:25.476251 (+ 1662us) catalog_manager.cc:2121] Created new table and tablet info
0925 09:58:26.192629 (+716378us) catalog_manager.cc:2147] Created new table in HMS catalog
0925 09:58:26.225363 (+ 32734us) catalog_manager.cc:2181] Wrote table and tablets to system table
0925 09:58:26.225654 (+ 291us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0925 09:58:26.225906 (+ 252us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:58:26.196210 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:58:26.196550 (+ 340us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:58:26.196573 (+ 23us) write_op.cc:435] Acquired schema lock
0925 09:58:26.196585 (+ 12us) tablet.cc:660] Decoding operations
0925 09:58:26.196823 (+ 238us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:58:26.196858 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0925 09:58:26.196880 (+ 22us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:58:26.197059 (+ 179us) tablet.cc:699] Row locks acquired
0925 09:58:26.197073 (+ 14us) write_op.cc:260] PREPARE: finished
0925 09:58:26.197177 (+ 104us) write_op.cc:270] Start()
0925 09:58:26.197250 (+ 73us) write_op.cc:276] Timestamp: P: 1790330306197116 usec, L: 0
0925 09:58:26.197264 (+ 14us) op_driver.cc:331] REPLICATION: starting
0925 09:58:26.197723 (+ 459us) log.cc:842] Serialized 433 byte log entry
0925 09:58:26.215085 (+ 17362us) op_driver.cc:444] REPLICATION: finished
0925 09:58:26.223541 (+ 8456us) write_op.cc:301] APPLY: starting
0925 09:58:26.223589 (+ 48us) tablet.cc:1365] starting BulkCheckPresence
0925 09:58:26.223671 (+ 82us) tablet.cc:1368] finished BulkCheckPresence
0925 09:58:26.223750 (+ 79us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:58:26.224054 (+ 304us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:58:26.224161 (+ 107us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 09:58:26.224195 (+ 34us) write_op.cc:312] APPLY: finished
0925 09:58:26.224339 (+ 144us) log.cc:842] Serialized 29 byte log entry
0925 09:58:26.224450 (+ 111us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:26.224506 (+ 56us) write_op.cc:454] Released schema lock
0925 09:58:26.224531 (+ 25us) write_op.cc:341] FINISH: Updating metrics
0925 09:58:26.224663 (+ 132us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:26.224683 (+ 20us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":217602,"Hive Metastore.run_cpu_time_us":573,"Hive Metastore.run_wall_time_us":497924,"spinlock_wait_cycles":162111744,"child_traces":[["op",{"apply.queue_time_us":8306,"apply.run_cpu_time_us":1275,"apply.run_wall_time_us":1275,"num_ops":2,"prepare.queue_time_us":2094,"prepare.run_cpu_time_us":1694,"prepare.run_wall_time_us":1694,"replication_time_us":17750}]]}
I20260925 09:58:26.232849 13770 tablet_bootstrap.cc:492] T 48ffc9cb12e346e7ae20c7f916ac33d4 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:26.233326 13770 ts_tablet_manager.cc:1397] T 48ffc9cb12e346e7ae20c7f916ac33d4 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.256s user 0.015s sys 0.037s
I20260925 09:58:26.244292 14274 tablet_bootstrap.cc:492] T 2e7cc97b8b594e92bf3bf74131f20456 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:26.244755 14274 ts_tablet_manager.cc:1397] T 2e7cc97b8b594e92bf3bf74131f20456 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.168s user 0.004s sys 0.006s
I20260925 09:58:26.247502 14274 raft_consensus.cc:348] T 2e7cc97b8b594e92bf3bf74131f20456 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.248206 14274 raft_consensus.cc:374] T 2e7cc97b8b594e92bf3bf74131f20456 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:26.248539 14274 raft_consensus.cc:729] T 2e7cc97b8b594e92bf3bf74131f20456 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:26.250036 14274 consensus_queue.cc:260] T 2e7cc97b8b594e92bf3bf74131f20456 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.264954 13770 raft_consensus.cc:348] T 48ffc9cb12e346e7ae20c7f916ac33d4 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.266978 14274 ts_tablet_manager.cc:1428] T 2e7cc97b8b594e92bf3bf74131f20456 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.022s user 0.005s sys 0.000s
I20260925 09:58:26.267813 13770 raft_consensus.cc:374] T 48ffc9cb12e346e7ae20c7f916ac33d4 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:26.268472 13770 raft_consensus.cc:729] T 48ffc9cb12e346e7ae20c7f916ac33d4 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:26.275883 13772 tablet_bootstrap.cc:492] T ace8260d4ac94cafbb9147ebeb01ed85 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:26.281869 14274 tablet_bootstrap.cc:492] T ace8260d4ac94cafbb9147ebeb01ed85 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:26.270871 13770 consensus_queue.cc:260] T 48ffc9cb12e346e7ae20c7f916ac33d4 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.294026 13770 ts_tablet_manager.cc:1428] T 48ffc9cb12e346e7ae20c7f916ac33d4 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.060s user 0.005s sys 0.001s
I20260925 09:58:26.296525 13770 tablet_bootstrap.cc:492] T 90d39744f6334e42bd32c26d13e6912c P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:26.301286 13781 ts_tablet_manager.cc:1918] T 8313c1ae006341bc84438085dd38f8ad P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:26.301649 13781 log.cc:1196] T 8313c1ae006341bc84438085dd38f8ad P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/8313c1ae006341bc84438085dd38f8ad
I20260925 09:58:26.302330 13781 ts_tablet_manager.cc:1939] T 8313c1ae006341bc84438085dd38f8ad P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:26.310119 13770 tablet_bootstrap.cc:654] T 90d39744f6334e42bd32c26d13e6912c P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:26.291250 14274 tablet_bootstrap.cc:654] T ace8260d4ac94cafbb9147ebeb01ed85 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:26.291314 13780 ts_tablet_manager.cc:1918] T 0749453435d54676803789c380179af4 P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:58:26.315227 13780 log.cc:1196] T 0749453435d54676803789c380179af4 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/0749453435d54676803789c380179af4
I20260925 09:58:26.316123 13780 ts_tablet_manager.cc:1939] T 0749453435d54676803789c380179af4 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:26.322887 14270 tablet.cc:1722] T 62d6d3750baa41c193c39ca39c4486a1 P b3082cf15dc74c0480e5d33ecf592843: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:58:26.328939 13772 tablet_bootstrap.cc:654] T ace8260d4ac94cafbb9147ebeb01ed85 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:26.333833 13780 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54548 (request call id 124) took 1253 ms. Trace:
I20260925 09:58:26.335572 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet 8313c1ae006341bc84438085dd38f8ad (table default.table_1238f9df428247dc9b9ca7c59823bfe4 [id=5eb4ef6bf2c344bdb26e429f484c59a4]) successfully deleted
I20260925 09:58:26.337225 13777 ts_tablet_manager.cc:1918] T 368f01226d6a4974acdd79006703a54c P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:26.337576 13777 log.cc:1196] T 368f01226d6a4974acdd79006703a54c P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/368f01226d6a4974acdd79006703a54c
I20260925 09:58:26.338275 13777 ts_tablet_manager.cc:1939] T 368f01226d6a4974acdd79006703a54c P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:26.338896 13781 tablet_replica.cc:330] T a8a89d2315f54a09ad5d6506f2c4797c P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:26.340665 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet 368f01226d6a4974acdd79006703a54c (table default.table_2c3c283350d644f39f1903713ab06589 [id=fec42fd086714e079c23b924ca453b65]) successfully deleted
I20260925 09:58:26.340945 13777 tablet_replica.cc:330] T cedc5b0609e14c409542a83efc45f3f0 P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:26.341655 13777 raft_consensus.cc:2229] T cedc5b0609e14c409542a83efc45f3f0 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:26.342213 13777 raft_consensus.cc:2258] T cedc5b0609e14c409542a83efc45f3f0 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:26.347255 13076 catalog_manager.cc:5587] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161) does not have the latest schema for tablet 62d6d3750baa41c193c39ca39c4486a1 (table default.table_d4409ab5331140e9a4bb995d57be4ec4 [id=f196d9d3750b4857905c1332d3ac763e]). Expected version 1 got 0
I20260925 09:58:26.337881 13780 rpcz_store.cc:276] 0925 09:58:25.080595 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:25.105025 (+ 24430us) service_pool.cc:227] Handling call
0925 09:58:25.925681 (+820656us) ts_tablet_manager.cc:1145] Deleting tablet 0749453435d54676803789c380179af4
0925 09:58:25.925734 (+ 53us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:58:26.088940 (+163206us) spinlock_profiling.cc:243] Waited 81.7 ms on lock 0x7b5400097ed4. stack: 00007f95e2f61033 00007f95e2f60f30 00007f95e2ae2534 00000000004db2b4 00000000004db249 00007f95e9e6d471 00007f95e9ef0352 00007f95ea30626c 00007f95eaf13cfa 00007f95eaf43e10 00007f95eaf2ae0d 00007f95eaf2adb9 00007f95eaf2ad49 00007f95eaf2ad11 00007f95eaf29f3d 0000000000580784
0925 09:58:26.249096 (+160156us) tablet_metadata.cc:641] Metadata flushed
0925 09:58:26.291077 (+ 41981us) tablet_metadata.cc:641] Metadata flushed
0925 09:58:26.333764 (+ 42687us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":187833216,"tablet-delete.queue_time_us":820131}
I20260925 09:58:26.341673 13781 raft_consensus.cc:2229] T a8a89d2315f54a09ad5d6506f2c4797c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:58:26.357803 14274 tablet_bootstrap.cc:492] T ace8260d4ac94cafbb9147ebeb01ed85 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:26.359009 14274 ts_tablet_manager.cc:1397] T ace8260d4ac94cafbb9147ebeb01ed85 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.077s user 0.008s sys 0.002s
I20260925 09:58:26.359916 13780 tablet_replica.cc:330] T 368f01226d6a4974acdd79006703a54c P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:26.361509 14274 raft_consensus.cc:348] T ace8260d4ac94cafbb9147ebeb01ed85 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.362886 13780 raft_consensus.cc:2229] T 368f01226d6a4974acdd79006703a54c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:26.356012 13781 pending_rounds.cc:62] T a8a89d2315f54a09ad5d6506f2c4797c P c4d724d4b83e4d1e817a3a505dc24eb9: Trying to abort 1 pending ops.
I20260925 09:58:26.363286 13781 pending_rounds.cc:68] T a8a89d2315f54a09ad5d6506f2c4797c P c4d724d4b83e4d1e817a3a505dc24eb9: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192932969230336 op_type: NO_OP noop_request { }
I20260925 09:58:26.362525 14274 raft_consensus.cc:374] T ace8260d4ac94cafbb9147ebeb01ed85 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:26.363801 13781 raft_consensus.cc:2866] T a8a89d2315f54a09ad5d6506f2c4797c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:58:26.364178 13781 raft_consensus.cc:2258] T a8a89d2315f54a09ad5d6506f2c4797c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:26.363988 14274 raft_consensus.cc:729] T ace8260d4ac94cafbb9147ebeb01ed85 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:26.364974 13780 pending_rounds.cc:62] T 368f01226d6a4974acdd79006703a54c P a06c3ef7a6a94f7cbcc03e7cd232f237: Trying to abort 1 pending ops.
I20260925 09:58:26.365460 14274 consensus_queue.cc:260] T ace8260d4ac94cafbb9147ebeb01ed85 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.366014 13780 pending_rounds.cc:68] T 368f01226d6a4974acdd79006703a54c P a06c3ef7a6a94f7cbcc03e7cd232f237: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192926495469568 op_type: NO_OP noop_request { }
I20260925 09:58:26.376003 13780 raft_consensus.cc:2866] T 368f01226d6a4974acdd79006703a54c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:58:26.384233 13780 raft_consensus.cc:2258] T 368f01226d6a4974acdd79006703a54c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:26.385210 13781 ts_tablet_manager.cc:1905] T a8a89d2315f54a09ad5d6506f2c4797c P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:26.386653 13777 ts_tablet_manager.cc:1905] T cedc5b0609e14c409542a83efc45f3f0 P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:26.390552 13780 ts_tablet_manager.cc:1905] T 368f01226d6a4974acdd79006703a54c P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:26.409274 13772 tablet_bootstrap.cc:492] T ace8260d4ac94cafbb9147ebeb01ed85 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:26.409691 13772 ts_tablet_manager.cc:1397] T ace8260d4ac94cafbb9147ebeb01ed85 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.134s user 0.004s sys 0.005s
I20260925 09:58:26.411826 13772 raft_consensus.cc:348] T ace8260d4ac94cafbb9147ebeb01ed85 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.412557 13772 raft_consensus.cc:374] T ace8260d4ac94cafbb9147ebeb01ed85 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:26.412794 13772 raft_consensus.cc:729] T ace8260d4ac94cafbb9147ebeb01ed85 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:26.413350 13772 consensus_queue.cc:260] T ace8260d4ac94cafbb9147ebeb01ed85 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.414852 14274 ts_tablet_manager.cc:1428] T ace8260d4ac94cafbb9147ebeb01ed85 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.056s user 0.006s sys 0.000s
I20260925 09:58:26.415148 13772 ts_tablet_manager.cc:1428] T ace8260d4ac94cafbb9147ebeb01ed85 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260925 09:58:26.417085 13770 tablet_bootstrap.cc:492] T 90d39744f6334e42bd32c26d13e6912c P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:26.417511 13770 ts_tablet_manager.cc:1397] T 90d39744f6334e42bd32c26d13e6912c P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.121s user 0.004s sys 0.008s
I20260925 09:58:26.420534 14199 raft_consensus.cc:482] T 2e7cc97b8b594e92bf3bf74131f20456 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:26.420969 14199 raft_consensus.cc:504] T 2e7cc97b8b594e92bf3bf74131f20456 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.422844 14199 leader_election.cc:288] T 2e7cc97b8b594e92bf3bf74131f20456 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:26.424957 13670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e7cc97b8b594e92bf3bf74131f20456" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:26.426683 13670 raft_consensus.cc:2454] T 2e7cc97b8b594e92bf3bf74131f20456 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 0.
I20260925 09:58:26.428193 13772 tablet_bootstrap.cc:492] T 3d18f12528cf40a49a7cd0c5c97001bf P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:26.428088 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e7cc97b8b594e92bf3bf74131f20456" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:26.428655 13363 raft_consensus.cc:2454] T 2e7cc97b8b594e92bf3bf74131f20456 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 0.
I20260925 09:58:26.434953 13440 leader_election.cc:302] T 2e7cc97b8b594e92bf3bf74131f20456 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:26.437124 13079 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50556:
name: "default.Table_489bd8fcb4594ceabc28fdbc0a951dd0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:58:26.439215 13772 tablet_bootstrap.cc:654] T 3d18f12528cf40a49a7cd0c5c97001bf P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:26.447024 13077 master_service.cc:946] ReplaceTablet: received request to replace tablet 2e7cc97b8b594e92bf3bf74131f20456 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:26.449617 13777 ts_tablet_manager.cc:1918] T cedc5b0609e14c409542a83efc45f3f0 P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:58:26.449987 13777 log.cc:1196] T cedc5b0609e14c409542a83efc45f3f0 P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/cedc5b0609e14c409542a83efc45f3f0
I20260925 09:58:26.450130 14199 raft_consensus.cc:2781] T 2e7cc97b8b594e92bf3bf74131f20456 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:26.450484 14199 raft_consensus.cc:482] T 2e7cc97b8b594e92bf3bf74131f20456 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:26.450780 14199 raft_consensus.cc:3037] T 2e7cc97b8b594e92bf3bf74131f20456 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:26.461858 13772 tablet_bootstrap.cc:492] T 3d18f12528cf40a49a7cd0c5c97001bf P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:26.462289 13772 ts_tablet_manager.cc:1397] T 3d18f12528cf40a49a7cd0c5c97001bf P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.034s user 0.009s sys 0.004s
I20260925 09:58:26.463310 13770 raft_consensus.cc:348] T 90d39744f6334e42bd32c26d13e6912c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.464685 13772 raft_consensus.cc:348] T 3d18f12528cf40a49a7cd0c5c97001bf P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.465278 13772 raft_consensus.cc:374] T 3d18f12528cf40a49a7cd0c5c97001bf P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:26.465528 13772 raft_consensus.cc:729] T 3d18f12528cf40a49a7cd0c5c97001bf P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:26.466051 13772 consensus_queue.cc:260] T 3d18f12528cf40a49a7cd0c5c97001bf P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.480808 13772 ts_tablet_manager.cc:1428] T 3d18f12528cf40a49a7cd0c5c97001bf P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.018s user 0.005s sys 0.000s
I20260925 09:58:26.482410 13772 tablet_bootstrap.cc:492] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:26.495046 14269 raft_consensus.cc:482] T 90d39744f6334e42bd32c26d13e6912c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:26.495728 14269 raft_consensus.cc:504] T 90d39744f6334e42bd32c26d13e6912c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.497928 14269 leader_election.cc:288] T 90d39744f6334e42bd32c26d13e6912c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:26.500808 13772 tablet_bootstrap.cc:654] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:26.500926 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90d39744f6334e42bd32c26d13e6912c" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
I20260925 09:58:26.501511 13505 raft_consensus.cc:2454] T 90d39744f6334e42bd32c26d13e6912c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 0.
I20260925 09:58:26.503182 13298 leader_election.cc:302] T 90d39744f6334e42bd32c26d13e6912c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:26.503535 13781 ts_tablet_manager.cc:1918] T a8a89d2315f54a09ad5d6506f2c4797c P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:26.503868 13781 log.cc:1196] T a8a89d2315f54a09ad5d6506f2c4797c P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/a8a89d2315f54a09ad5d6506f2c4797c
I20260925 09:58:26.504035 14269 raft_consensus.cc:2781] T 90d39744f6334e42bd32c26d13e6912c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:26.504333 14269 raft_consensus.cc:482] T 90d39744f6334e42bd32c26d13e6912c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:26.504566 13781 ts_tablet_manager.cc:1939] T a8a89d2315f54a09ad5d6506f2c4797c P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:26.504629 14269 raft_consensus.cc:3037] T 90d39744f6334e42bd32c26d13e6912c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:26.506913 13069 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet a8a89d2315f54a09ad5d6506f2c4797c (table default.table_47c0ac37d06f4f45a701b0910a42c579 [id=714d733484104c508e7ced7cb14148bf]) successfully deleted
I20260925 09:58:26.512154 13770 raft_consensus.cc:374] T 90d39744f6334e42bd32c26d13e6912c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:26.513088 14269 raft_consensus.cc:504] T 90d39744f6334e42bd32c26d13e6912c P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.513830 14274 tablet_bootstrap.cc:492] T 48ffc9cb12e346e7ae20c7f916ac33d4 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:26.526037 13482 tablet_service.cc:1467] Processing CreateTablet for tablet b9cf55829fed4c93a8eab92cee8920c9 (DEFAULT_TABLE table=default.table_840c9ea5fcc04f1b8811754058184452 [id=84e4b25904ae485fbf3816bd06643b42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:26.527122 13482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9cf55829fed4c93a8eab92cee8920c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:26.499589 13670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90d39744f6334e42bd32c26d13e6912c" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:26.529428 13770 raft_consensus.cc:729] T 90d39744f6334e42bd32c26d13e6912c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:26.530324 13646 tablet_service.cc:1467] Processing CreateTablet for tablet b9cf55829fed4c93a8eab92cee8920c9 (DEFAULT_TABLE table=default.table_840c9ea5fcc04f1b8811754058184452 [id=84e4b25904ae485fbf3816bd06643b42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:26.531517 13646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9cf55829fed4c93a8eab92cee8920c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:26.538820 13770 consensus_queue.cc:260] T 90d39744f6334e42bd32c26d13e6912c P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.540354 13770 ts_tablet_manager.cc:1428] T 90d39744f6334e42bd32c26d13e6912c P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.123s user 0.005s sys 0.000s
I20260925 09:58:26.518929 13772 tablet_bootstrap.cc:492] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:26.541857 13772 ts_tablet_manager.cc:1397] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.060s user 0.010s sys 0.000s
I20260925 09:58:26.542450 13670 raft_consensus.cc:2454] T 90d39744f6334e42bd32c26d13e6912c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 0.
I20260925 09:58:26.543054 14269 leader_election.cc:288] T 90d39744f6334e42bd32c26d13e6912c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Requested vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:26.520246 14199 raft_consensus.cc:504] T 2e7cc97b8b594e92bf3bf74131f20456 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.544525 13772 raft_consensus.cc:348] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.545123 13772 raft_consensus.cc:374] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:26.545338 13772 raft_consensus.cc:729] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:26.545687 14199 leader_election.cc:288] T 2e7cc97b8b594e92bf3bf74131f20456 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Requested vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:26.537151 13338 tablet_service.cc:1467] Processing CreateTablet for tablet b9cf55829fed4c93a8eab92cee8920c9 (DEFAULT_TABLE table=default.table_840c9ea5fcc04f1b8811754058184452 [id=84e4b25904ae485fbf3816bd06643b42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:26.538496 13340 tablet_service.cc:1467] Processing CreateTablet for tablet 4b61aaac33514c999482b824e0839a72 (DEFAULT_TABLE table=default.table_f83185097e7343848b564d3d5881c9ab [id=7f3f3dd4c3d74f21878caad38c9c25b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:26.547816 13338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9cf55829fed4c93a8eab92cee8920c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:26.548375 13340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b61aaac33514c999482b824e0839a72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:26.549079 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90d39744f6334e42bd32c26d13e6912c" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9"
I20260925 09:58:26.549588 13505 raft_consensus.cc:3037] T 90d39744f6334e42bd32c26d13e6912c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:26.551012 13777 ts_tablet_manager.cc:1939] T cedc5b0609e14c409542a83efc45f3f0 P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:26.552850 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e7cc97b8b594e92bf3bf74131f20456" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843"
I20260925 09:58:26.553367 13777 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49260 (request call id 139) took 1044 ms. Trace:
I20260925 09:58:26.553378 13363 raft_consensus.cc:3037] T 2e7cc97b8b594e92bf3bf74131f20456 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:26.545859 13772 consensus_queue.cc:260] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.556455 13505 raft_consensus.cc:2454] T 90d39744f6334e42bd32c26d13e6912c P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 1.
I20260925 09:58:26.556813 13772 ts_tablet_manager.cc:1428] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.015s user 0.002s sys 0.003s
I20260925 09:58:26.557646 13772 tablet_bootstrap.cc:492] T d3efa619592045b8be65e6b789b8291a P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:26.557498 13298 leader_election.cc:302] T 90d39744f6334e42bd32c26d13e6912c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:26.564504 13483 tablet_service.cc:1467] Processing CreateTablet for tablet 4b61aaac33514c999482b824e0839a72 (DEFAULT_TABLE table=default.table_f83185097e7343848b564d3d5881c9ab [id=7f3f3dd4c3d74f21878caad38c9c25b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:26.565026 14295 raft_consensus.cc:2781] T 90d39744f6334e42bd32c26d13e6912c P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:26.565407 14295 raft_consensus.cc:686] T 90d39744f6334e42bd32c26d13e6912c P b3082cf15dc74c0480e5d33ecf592843 [term 1 LEADER]: Becoming Leader. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Running, Role: LEADER
I20260925 09:58:26.565485 13483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b61aaac33514c999482b824e0839a72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:26.566139 14295 consensus_queue.cc:237] T 90d39744f6334e42bd32c26d13e6912c P b3082cf15dc74c0480e5d33ecf592843 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.572273 13772 tablet_bootstrap.cc:654] T d3efa619592045b8be65e6b789b8291a P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:26.573621 13077 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Sending DeleteTablet for 3 replicas of tablet 2e7cc97b8b594e92bf3bf74131f20456
I20260925 09:58:26.575078 13077 catalog_manager.cc:6565] ReplaceTablet: tablet 2e7cc97b8b594e92bf3bf74131f20456 deleted and replaced by tablet e370ffe0dec24f40a3b446ed4cdc831b
I20260925 09:58:26.575544 13484 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e7cc97b8b594e92bf3bf74131f20456 with delete_type TABLET_DATA_DELETED (replaced by tablet e370ffe0dec24f40a3b446ed4cdc831b) from {username='slave'} at 127.0.0.1:44532
I20260925 09:58:26.576292 13339 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e7cc97b8b594e92bf3bf74131f20456 with delete_type TABLET_DATA_DELETED (replaced by tablet e370ffe0dec24f40a3b446ed4cdc831b) from {username='slave'} at 127.0.0.1:49260
I20260925 09:58:26.577672 13670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90d39744f6334e42bd32c26d13e6912c" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
I20260925 09:58:26.578179 13670 raft_consensus.cc:3037] T 90d39744f6334e42bd32c26d13e6912c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:26.578806 13650 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e7cc97b8b594e92bf3bf74131f20456 with delete_type TABLET_DATA_DELETED (replaced by tablet e370ffe0dec24f40a3b446ed4cdc831b) from {username='slave'} at 127.0.0.1:54548
I20260925 09:58:26.578945 13770 tablet_bootstrap.cc:492] T d3efa619592045b8be65e6b789b8291a P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:26.587914 13077 catalog_manager.cc:5587] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161) does not have the latest schema for tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]). Expected version 1 got 0
I20260925 09:58:26.590328 13670 raft_consensus.cc:2454] T 90d39744f6334e42bd32c26d13e6912c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 1.
W20260925 09:58:26.590847 13077 catalog_manager.cc:5687] Failed to send AlterTable RPC for tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 consensus state has no leader
I20260925 09:58:26.593037 13077 master_service.cc:946] ReplaceTablet: received request to replace tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 from {username='slave'} at 127.0.0.1:50556
I20260925 09:58:26.592767 14274 tablet_bootstrap.cc:654] T 48ffc9cb12e346e7ae20c7f916ac33d4 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:26.601765 13650 tablet_service.cc:1467] Processing CreateTablet for tablet 4b61aaac33514c999482b824e0839a72 (DEFAULT_TABLE table=default.table_f83185097e7343848b564d3d5881c9ab [id=7f3f3dd4c3d74f21878caad38c9c25b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:26.602752 13650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b61aaac33514c999482b824e0839a72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:26.614477 13770 tablet_bootstrap.cc:654] T d3efa619592045b8be65e6b789b8291a P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:26.617421 13363 raft_consensus.cc:2454] T 2e7cc97b8b594e92bf3bf74131f20456 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:26.622756 14274 tablet_bootstrap.cc:492] T 48ffc9cb12e346e7ae20c7f916ac33d4 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:26.619197 13338 tablet_service.cc:1467] Processing CreateTablet for tablet fafc2b657ace4a4381452ce43fad39b6 (DEFAULT_TABLE table=default.table_47c0ac37d06f4f45a701b0910a42c579 [id=714d733484104c508e7ced7cb14148bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:26.623723 13338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fafc2b657ace4a4381452ce43fad39b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:26.623948 13438 leader_election.cc:302] T 2e7cc97b8b594e92bf3bf74131f20456 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:26.625116 14199 raft_consensus.cc:2781] T 2e7cc97b8b594e92bf3bf74131f20456 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:26.625572 14199 raft_consensus.cc:686] T 2e7cc97b8b594e92bf3bf74131f20456 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: Becoming Leader. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Running, Role: LEADER
I20260925 09:58:26.626387 14199 consensus_queue.cc:237] T 2e7cc97b8b594e92bf3bf74131f20456 P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.635515 14274 ts_tablet_manager.cc:1397] T 48ffc9cb12e346e7ae20c7f916ac33d4 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.135s user 0.008s sys 0.003s
I20260925 09:58:26.639171 14274 raft_consensus.cc:348] T 48ffc9cb12e346e7ae20c7f916ac33d4 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.643103 14292 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b7d0c1c2f1ed4e7a9ed54e73869d53b0)
I20260925 09:58:26.644364 13647 tablet_service.cc:1467] Processing CreateTablet for tablet fafc2b657ace4a4381452ce43fad39b6 (DEFAULT_TABLE table=default.table_47c0ac37d06f4f45a701b0910a42c579 [id=714d733484104c508e7ced7cb14148bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:26.643937 14292 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:26.645449 13647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fafc2b657ace4a4381452ce43fad39b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:26.652786 14274 raft_consensus.cc:374] T 48ffc9cb12e346e7ae20c7f916ac33d4 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:26.653129 14274 raft_consensus.cc:729] T 48ffc9cb12e346e7ae20c7f916ac33d4 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
W20260925 09:58:26.654402 13067 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 consensus state has no leader
I20260925 09:58:26.656463 14274 consensus_queue.cc:260] T 48ffc9cb12e346e7ae20c7f916ac33d4 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.666113 14295 raft_consensus.cc:482] T 48ffc9cb12e346e7ae20c7f916ac33d4 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:26.666579 14295 raft_consensus.cc:504] T 48ffc9cb12e346e7ae20c7f916ac33d4 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.553917 13777 rpcz_store.cc:276] 0925 09:58:25.508724 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:25.508911 (+ 187us) service_pool.cc:227] Handling call
0925 09:58:26.340794 (+831883us) ts_tablet_manager.cc:1145] Deleting tablet cedc5b0609e14c409542a83efc45f3f0
0925 09:58:26.340840 (+ 46us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:58:26.441692 (+100852us) tablet_metadata.cc:641] Metadata flushed
0925 09:58:26.449415 (+ 7723us) tablet_metadata.cc:641] Metadata flushed
0925 09:58:26.553313 (+103898us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":831268}
I20260925 09:58:26.668649 14295 leader_election.cc:288] T 48ffc9cb12e346e7ae20c7f916ac33d4 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:26.668684 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet cedc5b0609e14c409542a83efc45f3f0 (table default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]) successfully deleted
I20260925 09:58:26.670141 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48ffc9cb12e346e7ae20c7f916ac33d4" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
I20260925 09:58:26.670683 13505 raft_consensus.cc:2454] T 48ffc9cb12e346e7ae20c7f916ac33d4 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 0.
I20260925 09:58:26.671130 13777 tablet_replica.cc:330] T 8313c1ae006341bc84438085dd38f8ad P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:26.671056 13670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48ffc9cb12e346e7ae20c7f916ac33d4" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:26.671761 13670 raft_consensus.cc:2454] T 48ffc9cb12e346e7ae20c7f916ac33d4 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 0.
I20260925 09:58:26.671937 13298 leader_election.cc:302] T 48ffc9cb12e346e7ae20c7f916ac33d4 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:26.675037 13780 ts_tablet_manager.cc:1918] T 368f01226d6a4974acdd79006703a54c P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:26.675706 13780 log.cc:1196] T 368f01226d6a4974acdd79006703a54c P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/368f01226d6a4974acdd79006703a54c
I20260925 09:58:26.677541 13780 ts_tablet_manager.cc:1939] T 368f01226d6a4974acdd79006703a54c P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:26.678884 13670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e7cc97b8b594e92bf3bf74131f20456" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
I20260925 09:58:26.679558 13670 raft_consensus.cc:3037] T 2e7cc97b8b594e92bf3bf74131f20456 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:26.680159 13777 raft_consensus.cc:2229] T 8313c1ae006341bc84438085dd38f8ad P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:26.680354 14295 raft_consensus.cc:2781] T 48ffc9cb12e346e7ae20c7f916ac33d4 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:26.680734 14295 raft_consensus.cc:482] T 48ffc9cb12e346e7ae20c7f916ac33d4 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:26.680796 13777 raft_consensus.cc:2258] T 8313c1ae006341bc84438085dd38f8ad P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:26.681066 14295 raft_consensus.cc:3037] T 48ffc9cb12e346e7ae20c7f916ac33d4 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:26.681389 10276 master-stress-test.cc:578] Killing test thread 1/10
I20260925 09:58:26.683032 13777 ts_tablet_manager.cc:1905] T 8313c1ae006341bc84438085dd38f8ad P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:26.686790 13338 tablet_service.cc:1467] Processing CreateTablet for tablet 0090f685d0184822b41fd3e731874567 (DEFAULT_TABLE table=default.table_76fed9ed8d204d969883e54e5c7330c7 [id=929b4a72ba9b4357929f4354783971d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:26.687708 13649 tablet_service.cc:1467] Processing CreateTablet for tablet 0090f685d0184822b41fd3e731874567 (DEFAULT_TABLE table=default.table_76fed9ed8d204d969883e54e5c7330c7 [id=929b4a72ba9b4357929f4354783971d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:26.688649 13649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0090f685d0184822b41fd3e731874567. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:26.688684 14295 raft_consensus.cc:504] T 48ffc9cb12e346e7ae20c7f916ac33d4 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.691578 13668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48ffc9cb12e346e7ae20c7f916ac33d4" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
I20260925 09:58:26.692134 13668 raft_consensus.cc:3037] T 48ffc9cb12e346e7ae20c7f916ac33d4 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:26.698336 13668 raft_consensus.cc:2454] T 48ffc9cb12e346e7ae20c7f916ac33d4 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 1.
I20260925 09:58:26.699486 13297 leader_election.cc:302] T 48ffc9cb12e346e7ae20c7f916ac33d4 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, b3082cf15dc74c0480e5d33ecf592843; no voters:
I20260925 09:58:26.701213 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48ffc9cb12e346e7ae20c7f916ac33d4" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9"
I20260925 09:58:26.701717 13505 raft_consensus.cc:3037] T 48ffc9cb12e346e7ae20c7f916ac33d4 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:26.704319 13017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "61e29afab0cf46819cbffc91248bf761" candidate_term: 5 candidate_status { last_received { term: 4 index: 129 } } ignore_live_leader: false dest_uuid: "70718e3f6f9f489089c0d8354380ed6c" is_pre_election: true
I20260925 09:58:26.704356 13090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "61e29afab0cf46819cbffc91248bf761" candidate_term: 5 candidate_status { last_received { term: 4 index: 129 } } ignore_live_leader: false dest_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" is_pre_election: true
I20260925 09:58:26.705286 14292 leader_election.cc:288] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 70718e3f6f9f489089c0d8354380ed6c (127.10.9.62:43437), b7d0c1c2f1ed4e7a9ed54e73869d53b0 (127.10.9.61:40425)
I20260925 09:58:26.706413 13955 leader_election.cc:302] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61e29afab0cf46819cbffc91248bf761; no voters: 70718e3f6f9f489089c0d8354380ed6c, b7d0c1c2f1ed4e7a9ed54e73869d53b0
I20260925 09:58:26.707743 14269 raft_consensus.cc:2781] T 48ffc9cb12e346e7ae20c7f916ac33d4 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:26.709321 14269 raft_consensus.cc:686] T 48ffc9cb12e346e7ae20c7f916ac33d4 P b3082cf15dc74c0480e5d33ecf592843 [term 1 LEADER]: Becoming Leader. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Running, Role: LEADER
I20260925 09:58:26.710146 14269 consensus_queue.cc:237] T 48ffc9cb12e346e7ae20c7f916ac33d4 P b3082cf15dc74c0480e5d33ecf592843 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.714040 13505 raft_consensus.cc:2454] T 48ffc9cb12e346e7ae20c7f916ac33d4 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 1.
I20260925 09:58:26.714841 14292 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260925 09:58:26.731783 14295 leader_election.cc:288] T 48ffc9cb12e346e7ae20c7f916ac33d4 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Requested vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:26.737023 14274 ts_tablet_manager.cc:1428] T 48ffc9cb12e346e7ae20c7f916ac33d4 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.100s user 0.008s sys 0.002s
I20260925 09:58:26.740772 13781 tablet_replica.cc:330] T dedf3e0a273d43ba84a861b36d0707ef P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:26.742066 13781 raft_consensus.cc:2229] T dedf3e0a273d43ba84a861b36d0707ef P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:26.743044 13781 raft_consensus.cc:2258] T dedf3e0a273d43ba84a861b36d0707ef P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:26.744652 14274 tablet_bootstrap.cc:492] T 3d18f12528cf40a49a7cd0c5c97001bf P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:26.750993 13338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0090f685d0184822b41fd3e731874567. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:26.762120 14274 tablet_bootstrap.cc:654] T 3d18f12528cf40a49a7cd0c5c97001bf P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:26.772974 13772 tablet_bootstrap.cc:492] T d3efa619592045b8be65e6b789b8291a P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:26.773393 13772 ts_tablet_manager.cc:1397] T d3efa619592045b8be65e6b789b8291a P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.216s user 0.026s sys 0.039s
I20260925 09:58:26.774026 13770 tablet_bootstrap.cc:492] T d3efa619592045b8be65e6b789b8291a P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:26.775673 13770 ts_tablet_manager.cc:1397] T d3efa619592045b8be65e6b789b8291a P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.197s user 0.009s sys 0.022s
I20260925 09:58:26.775597 13772 raft_consensus.cc:348] T d3efa619592045b8be65e6b789b8291a P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.776268 13772 raft_consensus.cc:374] T d3efa619592045b8be65e6b789b8291a P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:26.776531 13772 raft_consensus.cc:729] T d3efa619592045b8be65e6b789b8291a P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:26.759291 13780 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54548 (request call id 130) took 1286 ms. Trace:
I20260925 09:58:26.781111 13772 consensus_queue.cc:260] T d3efa619592045b8be65e6b789b8291a P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.781771 13780 rpcz_store.cc:276] 0925 09:58:25.472996 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:25.477065 (+ 4069us) service_pool.cc:227] Handling call
0925 09:58:26.359753 (+882688us) ts_tablet_manager.cc:1145] Deleting tablet 368f01226d6a4974acdd79006703a54c
0925 09:58:26.359802 (+ 49us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:58:26.443502 (+ 83700us) tablet_metadata.cc:641] Metadata flushed
0925 09:58:26.674773 (+231271us) tablet_metadata.cc:641] Metadata flushed
0925 09:58:26.759215 (+ 84442us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":748109}
I20260925 09:58:26.782948 13780 tablet_replica.cc:330] T cedc5b0609e14c409542a83efc45f3f0 P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:26.783216 13772 ts_tablet_manager.cc:1428] T d3efa619592045b8be65e6b789b8291a P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20260925 09:58:26.783149 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet 368f01226d6a4974acdd79006703a54c (table default.table_2c3c283350d644f39f1903713ab06589 [id=fec42fd086714e079c23b924ca453b65]) successfully deleted
I20260925 09:58:26.783672 13780 raft_consensus.cc:2229] T cedc5b0609e14c409542a83efc45f3f0 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:26.783998 13770 raft_consensus.cc:348] T d3efa619592045b8be65e6b789b8291a P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.784448 13780 raft_consensus.cc:2258] T cedc5b0609e14c409542a83efc45f3f0 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:26.786581 14274 tablet_bootstrap.cc:492] T 3d18f12528cf40a49a7cd0c5c97001bf P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:26.787235 14274 ts_tablet_manager.cc:1397] T 3d18f12528cf40a49a7cd0c5c97001bf P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.043s user 0.012s sys 0.002s
I20260925 09:58:26.784623 13770 raft_consensus.cc:374] T d3efa619592045b8be65e6b789b8291a P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:26.788156 13770 raft_consensus.cc:729] T d3efa619592045b8be65e6b789b8291a P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:26.789016 13770 consensus_queue.cc:260] T d3efa619592045b8be65e6b789b8291a P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.795580 13772 tablet_bootstrap.cc:492] T 0e3f1d9381b4450cb6c15525699efbc9 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:26.799610 13781 ts_tablet_manager.cc:1905] T dedf3e0a273d43ba84a861b36d0707ef P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:26.800444 13772 tablet_bootstrap.cc:654] T 0e3f1d9381b4450cb6c15525699efbc9 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:26.821678 13770 ts_tablet_manager.cc:1428] T d3efa619592045b8be65e6b789b8291a P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.045s user 0.003s sys 0.001s
I20260925 09:58:26.822943 14274 raft_consensus.cc:348] T 3d18f12528cf40a49a7cd0c5c97001bf P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.823917 13772 tablet_bootstrap.cc:492] T 0e3f1d9381b4450cb6c15525699efbc9 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:26.824298 13772 ts_tablet_manager.cc:1397] T 0e3f1d9381b4450cb6c15525699efbc9 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.040s user 0.010s sys 0.002s
I20260925 09:58:26.824643 13770 tablet_bootstrap.cc:492] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:26.824958 14274 raft_consensus.cc:374] T 3d18f12528cf40a49a7cd0c5c97001bf P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:26.826445 13772 raft_consensus.cc:348] T 0e3f1d9381b4450cb6c15525699efbc9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:26.827065 13772 raft_consensus.cc:374] T 0e3f1d9381b4450cb6c15525699efbc9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:26.827127 14274 raft_consensus.cc:729] T 3d18f12528cf40a49a7cd0c5c97001bf P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:26.827306 13772 raft_consensus.cc:729] T 0e3f1d9381b4450cb6c15525699efbc9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:26.811645 13670 raft_consensus.cc:2454] T 2e7cc97b8b594e92bf3bf74131f20456 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:26.828085 13772 consensus_queue.cc:260] T 0e3f1d9381b4450cb6c15525699efbc9 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:26.829949 13772 ts_tablet_manager.cc:1428] T 0e3f1d9381b4450cb6c15525699efbc9 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260925 09:58:26.831452 13772 tablet_bootstrap.cc:492] T b9cf55829fed4c93a8eab92cee8920c9 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:26.829767 13780 ts_tablet_manager.cc:1905] T cedc5b0609e14c409542a83efc45f3f0 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:26.836696 13772 tablet_bootstrap.cc:654] T b9cf55829fed4c93a8eab92cee8920c9 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:26.851280 14274 consensus_queue.cc:260] T 3d18f12528cf40a49a7cd0c5c97001bf P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.852802 13777 ts_tablet_manager.cc:1918] T 8313c1ae006341bc84438085dd38f8ad P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:26.856634 13772 tablet_bootstrap.cc:492] T b9cf55829fed4c93a8eab92cee8920c9 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:26.857007 13772 ts_tablet_manager.cc:1397] T b9cf55829fed4c93a8eab92cee8920c9 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.026s user 0.011s sys 0.000s
I20260925 09:58:26.865835 14274 ts_tablet_manager.cc:1428] T 3d18f12528cf40a49a7cd0c5c97001bf P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.078s user 0.005s sys 0.000s
I20260925 09:58:26.867470 13777 log.cc:1196] T 8313c1ae006341bc84438085dd38f8ad P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/8313c1ae006341bc84438085dd38f8ad
I20260925 09:58:26.868281 13777 ts_tablet_manager.cc:1939] T 8313c1ae006341bc84438085dd38f8ad P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:26.859058 13772 raft_consensus.cc:348] T b9cf55829fed4c93a8eab92cee8920c9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.870796 13772 raft_consensus.cc:374] T b9cf55829fed4c93a8eab92cee8920c9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:26.871034 13772 raft_consensus.cc:729] T b9cf55829fed4c93a8eab92cee8920c9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:26.880327 14274 tablet_bootstrap.cc:492] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:26.880764 13777 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49260 (request call id 140) took 1355 ms. Trace:
I20260925 09:58:26.881469 13770 tablet_bootstrap.cc:654] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:26.889195 14274 tablet_bootstrap.cc:654] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:26.889398 13777 rpcz_store.cc:276] 0925 09:58:25.525566 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:25.525689 (+ 123us) service_pool.cc:227] Handling call
0925 09:58:26.670899 (+1145210us) ts_tablet_manager.cc:1145] Deleting tablet 8313c1ae006341bc84438085dd38f8ad
0925 09:58:26.670993 (+ 94us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:58:26.789830 (+118837us) tablet_metadata.cc:641] Metadata flushed
0925 09:58:26.852527 (+ 62697us) tablet_metadata.cc:641] Metadata flushed
0925 09:58:26.880712 (+ 28185us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1144671}
I20260925 09:58:26.880661 13772 consensus_queue.cc:260] T b9cf55829fed4c93a8eab92cee8920c9 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.890745 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet 8313c1ae006341bc84438085dd38f8ad (table default.table_1238f9df428247dc9b9ca7c59823bfe4 [id=5eb4ef6bf2c344bdb26e429f484c59a4]) successfully deleted
I20260925 09:58:26.891759 13777 tablet_replica.cc:330] T a8a89d2315f54a09ad5d6506f2c4797c P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:26.894691 13770 tablet_bootstrap.cc:492] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:26.893339 13772 ts_tablet_manager.cc:1428] T b9cf55829fed4c93a8eab92cee8920c9 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.036s user 0.004s sys 0.001s
I20260925 09:58:26.895675 13777 raft_consensus.cc:2229] T a8a89d2315f54a09ad5d6506f2c4797c P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:26.896245 13777 raft_consensus.cc:2258] T a8a89d2315f54a09ad5d6506f2c4797c P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:26.898051 13777 ts_tablet_manager.cc:1905] T a8a89d2315f54a09ad5d6506f2c4797c P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:26.902567 13772 tablet_bootstrap.cc:492] T fafc2b657ace4a4381452ce43fad39b6 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:26.903347 14018 raft_consensus.cc:482] T ace8260d4ac94cafbb9147ebeb01ed85 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:26.903833 14018 raft_consensus.cc:504] T ace8260d4ac94cafbb9147ebeb01ed85 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.905607 14018 leader_election.cc:288] T ace8260d4ac94cafbb9147ebeb01ed85 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
I20260925 09:58:26.906930 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ace8260d4ac94cafbb9147ebeb01ed85" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:26.906980 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ace8260d4ac94cafbb9147ebeb01ed85" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
I20260925 09:58:26.907521 13363 raft_consensus.cc:2454] T ace8260d4ac94cafbb9147ebeb01ed85 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 0.
I20260925 09:58:26.907662 13505 raft_consensus.cc:2454] T ace8260d4ac94cafbb9147ebeb01ed85 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 0.
I20260925 09:58:26.908805 13606 leader_election.cc:302] T ace8260d4ac94cafbb9147ebeb01ed85 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:26.909899 14018 raft_consensus.cc:2781] T ace8260d4ac94cafbb9147ebeb01ed85 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:26.910229 14018 raft_consensus.cc:482] T ace8260d4ac94cafbb9147ebeb01ed85 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:26.910498 14018 raft_consensus.cc:3037] T ace8260d4ac94cafbb9147ebeb01ed85 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:26.916124 13772 tablet_bootstrap.cc:654] T fafc2b657ace4a4381452ce43fad39b6 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:26.895088 13770 ts_tablet_manager.cc:1397] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.071s user 0.008s sys 0.000s
I20260925 09:58:26.924924 13770 raft_consensus.cc:348] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.925559 13770 raft_consensus.cc:374] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:26.925850 13770 raft_consensus.cc:729] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:26.925616 14018 raft_consensus.cc:504] T ace8260d4ac94cafbb9147ebeb01ed85 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.926478 13770 consensus_queue.cc:260] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.932873 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ace8260d4ac94cafbb9147ebeb01ed85" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843"
I20260925 09:58:26.933439 13363 raft_consensus.cc:3037] T ace8260d4ac94cafbb9147ebeb01ed85 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:26.940781 13363 raft_consensus.cc:2454] T ace8260d4ac94cafbb9147ebeb01ed85 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 1.
I20260925 09:58:26.941869 13603 leader_election.cc:302] T ace8260d4ac94cafbb9147ebeb01ed85 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, b3082cf15dc74c0480e5d33ecf592843; no voters:
I20260925 09:58:26.954361 14309 raft_consensus.cc:2781] T ace8260d4ac94cafbb9147ebeb01ed85 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:26.956535 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ace8260d4ac94cafbb9147ebeb01ed85" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9"
I20260925 09:58:26.957077 13505 raft_consensus.cc:3037] T ace8260d4ac94cafbb9147ebeb01ed85 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:26.958658 13781 ts_tablet_manager.cc:1918] T dedf3e0a273d43ba84a861b36d0707ef P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:26.961515 13781 log.cc:1196] T dedf3e0a273d43ba84a861b36d0707ef P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/dedf3e0a273d43ba84a861b36d0707ef
I20260925 09:58:26.963452 13505 raft_consensus.cc:2454] T ace8260d4ac94cafbb9147ebeb01ed85 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 1.
I20260925 09:58:26.967293 13781 ts_tablet_manager.cc:1939] T dedf3e0a273d43ba84a861b36d0707ef P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:26.969201 14274 tablet_bootstrap.cc:492] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:26.969702 14274 ts_tablet_manager.cc:1397] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.090s user 0.005s sys 0.005s
I20260925 09:58:26.972302 14274 raft_consensus.cc:348] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.973156 14274 raft_consensus.cc:374] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:26.973483 14274 raft_consensus.cc:729] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:26.974292 14274 consensus_queue.cc:260] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.956738 13770 ts_tablet_manager.cc:1428] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.034s user 0.005s sys 0.001s
I20260925 09:58:26.967505 13780 ts_tablet_manager.cc:1918] T cedc5b0609e14c409542a83efc45f3f0 P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 09:58:26.978688 13780 log.cc:1196] T cedc5b0609e14c409542a83efc45f3f0 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/cedc5b0609e14c409542a83efc45f3f0
I20260925 09:58:26.979172 13770 tablet_bootstrap.cc:492] T 3d18f12528cf40a49a7cd0c5c97001bf P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:26.979872 13780 ts_tablet_manager.cc:1939] T cedc5b0609e14c409542a83efc45f3f0 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:26.982708 13780 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54548 (request call id 133) took 1472 ms. Trace:
I20260925 09:58:26.989141 13780 rpcz_store.cc:276] 0925 09:58:25.510313 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:25.510461 (+ 148us) service_pool.cc:227] Handling call
0925 09:58:26.782777 (+1272316us) ts_tablet_manager.cc:1145] Deleting tablet cedc5b0609e14c409542a83efc45f3f0
0925 09:58:26.782829 (+ 52us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:58:26.894466 (+111637us) tablet_metadata.cc:641] Metadata flushed
0925 09:58:26.965840 (+ 71374us) tablet_metadata.cc:641] Metadata flushed
0925 09:58:26.982658 (+ 16818us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":3,"tablet-delete.queue_time_us":1163209}
I20260925 09:58:26.996307 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet cedc5b0609e14c409542a83efc45f3f0 (table default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]) successfully deleted
I20260925 09:58:26.978071 14018 leader_election.cc:288] T ace8260d4ac94cafbb9147ebeb01ed85 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 election: Requested vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
I20260925 09:58:26.978546 14309 raft_consensus.cc:686] T ace8260d4ac94cafbb9147ebeb01ed85 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 LEADER]: Becoming Leader. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Running, Role: LEADER
I20260925 09:58:26.998113 14309 consensus_queue.cc:237] T ace8260d4ac94cafbb9147ebeb01ed85 P a06c3ef7a6a94f7cbcc03e7cd232f237 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:26.999375 13780 tablet_replica.cc:330] T 8313c1ae006341bc84438085dd38f8ad P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:27.002166 13772 tablet_bootstrap.cc:492] T fafc2b657ace4a4381452ce43fad39b6 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:27.002560 13772 ts_tablet_manager.cc:1397] T fafc2b657ace4a4381452ce43fad39b6 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.100s user 0.007s sys 0.004s
I20260925 09:58:27.004910 13772 raft_consensus.cc:348] T fafc2b657ace4a4381452ce43fad39b6 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.005570 13772 raft_consensus.cc:374] T fafc2b657ace4a4381452ce43fad39b6 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:27.005801 13772 raft_consensus.cc:729] T fafc2b657ace4a4381452ce43fad39b6 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:27.004441 13780 raft_consensus.cc:2229] T 8313c1ae006341bc84438085dd38f8ad P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:26.983703 13781 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44532 (request call id 135) took 1062 ms. Trace:
I20260925 09:58:27.007607 13781 rpcz_store.cc:276] 0925 09:58:25.920837 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:25.920989 (+ 152us) service_pool.cc:227] Handling call
0925 09:58:26.740578 (+819589us) ts_tablet_manager.cc:1145] Deleting tablet dedf3e0a273d43ba84a861b36d0707ef
0925 09:58:26.740636 (+ 58us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:58:26.895761 (+155125us) tablet_metadata.cc:641] Metadata flushed
0925 09:58:26.956560 (+ 60799us) tablet_metadata.cc:641] Metadata flushed
0925 09:58:26.983646 (+ 27086us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":8064,"tablet-delete.queue_time_us":819085}
I20260925 09:58:27.009583 13781 tablet_replica.cc:330] T 2e7cc97b8b594e92bf3bf74131f20456 P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:27.010327 13781 raft_consensus.cc:2229] T 2e7cc97b8b594e92bf3bf74131f20456 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:58:27.011030 13781 pending_rounds.cc:62] T 2e7cc97b8b594e92bf3bf74131f20456 P c4d724d4b83e4d1e817a3a505dc24eb9: Trying to abort 1 pending ops.
I20260925 09:58:27.011336 13781 pending_rounds.cc:68] T 2e7cc97b8b594e92bf3bf74131f20456 P c4d724d4b83e4d1e817a3a505dc24eb9: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333192935949271040 op_type: NO_OP noop_request { }
I20260925 09:58:27.011837 13781 raft_consensus.cc:2866] T 2e7cc97b8b594e92bf3bf74131f20456 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260925 09:58:27.012264 13781 raft_consensus.cc:2258] T 2e7cc97b8b594e92bf3bf74131f20456 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:27.013882 13780 raft_consensus.cc:2258] T 8313c1ae006341bc84438085dd38f8ad P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:27.019462 13772 consensus_queue.cc:260] T fafc2b657ace4a4381452ce43fad39b6 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.021417 13772 ts_tablet_manager.cc:1428] T fafc2b657ace4a4381452ce43fad39b6 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.019s user 0.004s sys 0.001s
I20260925 09:58:27.013477 14274 ts_tablet_manager.cc:1428] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.043s user 0.000s sys 0.005s
I20260925 09:58:27.022648 13777 ts_tablet_manager.cc:1918] T a8a89d2315f54a09ad5d6506f2c4797c P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:27.024283 13777 log.cc:1196] T a8a89d2315f54a09ad5d6506f2c4797c P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/a8a89d2315f54a09ad5d6506f2c4797c
I20260925 09:58:27.025295 13777 ts_tablet_manager.cc:1939] T a8a89d2315f54a09ad5d6506f2c4797c P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:27.031625 13770 tablet_bootstrap.cc:654] T 3d18f12528cf40a49a7cd0c5c97001bf P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:27.034842 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet a8a89d2315f54a09ad5d6506f2c4797c (table default.table_47c0ac37d06f4f45a701b0910a42c579 [id=714d733484104c508e7ced7cb14148bf]) successfully deleted
I20260925 09:58:27.023527 13772 tablet_bootstrap.cc:492] T 0090f685d0184822b41fd3e731874567 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:27.038614 14274 tablet_bootstrap.cc:492] T 0e3f1d9381b4450cb6c15525699efbc9 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:27.040318 13781 ts_tablet_manager.cc:1905] T 2e7cc97b8b594e92bf3bf74131f20456 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:27.049685 13777 tablet_replica.cc:330] T dedf3e0a273d43ba84a861b36d0707ef P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:27.050479 13777 raft_consensus.cc:2229] T dedf3e0a273d43ba84a861b36d0707ef P b3082cf15dc74c0480e5d33ecf592843 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:58:27.040654 13780 ts_tablet_manager.cc:1905] T 8313c1ae006341bc84438085dd38f8ad P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:27.059249 13772 tablet_bootstrap.cc:654] T 0090f685d0184822b41fd3e731874567 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:27.067831 13777 raft_consensus.cc:2258] T dedf3e0a273d43ba84a861b36d0707ef P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:27.068428 13770 tablet_bootstrap.cc:492] T 3d18f12528cf40a49a7cd0c5c97001bf P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:27.070987 13770 ts_tablet_manager.cc:1397] T 3d18f12528cf40a49a7cd0c5c97001bf P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.092s user 0.007s sys 0.002s
I20260925 09:58:27.074278 14269 raft_consensus.cc:482] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:27.074709 14269 raft_consensus.cc:504] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.076699 14269 leader_election.cc:288] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:27.078397 13670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3970efec7a0d4e7c9275c1f6b88ff4a9" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:27.078850 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3970efec7a0d4e7c9275c1f6b88ff4a9" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
I20260925 09:58:27.079010 13670 raft_consensus.cc:2454] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 0.
I20260925 09:58:27.079422 13505 raft_consensus.cc:2454] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 0.
I20260925 09:58:27.080205 13297 leader_election.cc:302] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, b3082cf15dc74c0480e5d33ecf592843; no voters:
I20260925 09:58:27.081667 13777 ts_tablet_manager.cc:1905] T dedf3e0a273d43ba84a861b36d0707ef P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:27.082427 14269 raft_consensus.cc:2781] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:27.082758 14269 raft_consensus.cc:482] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:27.083046 14269 raft_consensus.cc:3037] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:27.090175 14269 raft_consensus.cc:504] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.093829 13073 catalog_manager.cc:3411] renamed table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7] in HMS: new name default.table_eda42a4c592f447b8084b26441b81b7c
I20260925 09:58:27.094938 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3970efec7a0d4e7c9275c1f6b88ff4a9" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9"
I20260925 09:58:27.095252 13670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3970efec7a0d4e7c9275c1f6b88ff4a9" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
I20260925 09:58:27.095567 13505 raft_consensus.cc:3037] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:27.095808 13670 raft_consensus.cc:3037] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:27.108271 13770 raft_consensus.cc:348] T 3d18f12528cf40a49a7cd0c5c97001bf P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.108510 13505 raft_consensus.cc:2454] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 1.
I20260925 09:58:27.109730 13298 leader_election.cc:302] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:27.110611 14295 raft_consensus.cc:2781] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:27.111014 14295 raft_consensus.cc:686] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P b3082cf15dc74c0480e5d33ecf592843 [term 1 LEADER]: Becoming Leader. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Running, Role: LEADER
I20260925 09:58:27.111886 14295 consensus_queue.cc:237] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P b3082cf15dc74c0480e5d33ecf592843 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.123581 13770 raft_consensus.cc:374] T 3d18f12528cf40a49a7cd0c5c97001bf P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:27.123914 13770 raft_consensus.cc:729] T 3d18f12528cf40a49a7cd0c5c97001bf P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:27.124557 13770 consensus_queue.cc:260] T 3d18f12528cf40a49a7cd0c5c97001bf P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.103780 13670 raft_consensus.cc:2454] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 1.
I20260925 09:58:27.127657 13770 ts_tablet_manager.cc:1428] T 3d18f12528cf40a49a7cd0c5c97001bf P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.056s user 0.003s sys 0.003s
I20260925 09:58:27.135600 13770 tablet_bootstrap.cc:492] T 0e3f1d9381b4450cb6c15525699efbc9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:27.137475 14316 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b7d0c1c2f1ed4e7a9ed54e73869d53b0)
I20260925 09:58:27.140501 13770 tablet_bootstrap.cc:654] T 0e3f1d9381b4450cb6c15525699efbc9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:27.141120 14316 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:27.144182 14269 leader_election.cc:288] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Requested vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:27.146239 14274 tablet_bootstrap.cc:654] T 0e3f1d9381b4450cb6c15525699efbc9 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:27.152802 13772 tablet_bootstrap.cc:492] T 0090f685d0184822b41fd3e731874567 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:27.153249 13772 ts_tablet_manager.cc:1397] T 0090f685d0184822b41fd3e731874567 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.130s user 0.012s sys 0.000s
I20260925 09:58:27.157271 13772 raft_consensus.cc:348] T 0090f685d0184822b41fd3e731874567 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.157994 13772 raft_consensus.cc:374] T 0090f685d0184822b41fd3e731874567 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:27.159315 13772 raft_consensus.cc:729] T 0090f685d0184822b41fd3e731874567 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:27.160019 13772 consensus_queue.cc:260] T 0090f685d0184822b41fd3e731874567 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.162559 13772 ts_tablet_manager.cc:1428] T 0090f685d0184822b41fd3e731874567 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.009s user 0.004s sys 0.000s
I20260925 09:58:27.163794 13772 tablet_bootstrap.cc:492] T 4b61aaac33514c999482b824e0839a72 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:27.177624 14295 raft_consensus.cc:482] T d3efa619592045b8be65e6b789b8291a P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:27.178097 14295 raft_consensus.cc:504] T d3efa619592045b8be65e6b789b8291a P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.181308 13670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3efa619592045b8be65e6b789b8291a" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:27.181457 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3efa619592045b8be65e6b789b8291a" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
I20260925 09:58:27.181900 13670 raft_consensus.cc:2454] T d3efa619592045b8be65e6b789b8291a P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 0.
I20260925 09:58:27.182264 13777 ts_tablet_manager.cc:1918] T dedf3e0a273d43ba84a861b36d0707ef P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:27.182602 13777 log.cc:1196] T dedf3e0a273d43ba84a861b36d0707ef P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/dedf3e0a273d43ba84a861b36d0707ef
W20260925 09:58:27.182757 13298 leader_election.cc:341] T d3efa619592045b8be65e6b789b8291a P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:27.183122 14295 leader_election.cc:288] T d3efa619592045b8be65e6b789b8291a P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:27.183349 13297 leader_election.cc:302] T d3efa619592045b8be65e6b789b8291a P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, b3082cf15dc74c0480e5d33ecf592843; no voters: c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:58:27.184360 14295 raft_consensus.cc:2781] T d3efa619592045b8be65e6b789b8291a P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:27.184732 14295 raft_consensus.cc:482] T d3efa619592045b8be65e6b789b8291a P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:27.185045 14295 raft_consensus.cc:3037] T d3efa619592045b8be65e6b789b8291a P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:27.191287 14295 raft_consensus.cc:504] T d3efa619592045b8be65e6b789b8291a P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.192560 14316 leader_election.cc:288] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b7d0c1c2f1ed4e7a9ed54e73869d53b0 (127.10.9.61:40425), 61e29afab0cf46819cbffc91248bf761 (127.10.9.60:42083)
I20260925 09:58:27.195362 13090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70718e3f6f9f489089c0d8354380ed6c" candidate_term: 5 candidate_status { last_received { term: 4 index: 195 } } ignore_live_leader: false dest_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" is_pre_election: true
I20260925 09:58:27.183256 13777 ts_tablet_manager.cc:1939] T dedf3e0a273d43ba84a861b36d0707ef P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:27.199441 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3efa619592045b8be65e6b789b8291a" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9"
W20260925 09:58:27.200608 13298 leader_election.cc:341] T d3efa619592045b8be65e6b789b8291a P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:27.205971 13772 tablet_bootstrap.cc:654] T 4b61aaac33514c999482b824e0839a72 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:27.206256 14295 leader_election.cc:288] T d3efa619592045b8be65e6b789b8291a P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Requested vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:27.206450 13670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3efa619592045b8be65e6b789b8291a" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
I20260925 09:58:27.206979 13670 raft_consensus.cc:3037] T d3efa619592045b8be65e6b789b8291a P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:27.208173 13082 catalog_manager.cc:2148] created HMS catalog entry for table [id=5a1feab31f8d4a9d9452607552d02054]
I20260925 09:58:27.208621 14274 tablet_bootstrap.cc:492] T 0e3f1d9381b4450cb6c15525699efbc9 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:27.211923 14274 ts_tablet_manager.cc:1397] T 0e3f1d9381b4450cb6c15525699efbc9 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.178s user 0.009s sys 0.003s
I20260925 09:58:27.211905 13777 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49260 (request call id 142) took 1087 ms. Trace:
I20260925 09:58:27.212199 13777 rpcz_store.cc:276] 0925 09:58:26.124102 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:26.127864 (+ 3762us) service_pool.cc:227] Handling call
0925 09:58:27.049539 (+921675us) ts_tablet_manager.cc:1145] Deleting tablet dedf3e0a273d43ba84a861b36d0707ef
0925 09:58:27.049581 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:58:27.109864 (+ 60283us) tablet_metadata.cc:641] Metadata flushed
0925 09:58:27.182042 (+ 72178us) tablet_metadata.cc:641] Metadata flushed
0925 09:58:27.211850 (+ 29808us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":921047}
I20260925 09:58:27.213254 13777 tablet_replica.cc:330] T 2e7cc97b8b594e92bf3bf74131f20456 P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:27.213918 13777 raft_consensus.cc:2229] T 2e7cc97b8b594e92bf3bf74131f20456 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:27.217386 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet dedf3e0a273d43ba84a861b36d0707ef (table default.table_f83185097e7343848b564d3d5881c9ab [id=7f3f3dd4c3d74f21878caad38c9c25b9]) successfully deleted
I20260925 09:58:27.226002 14199 raft_consensus.cc:482] T 3d18f12528cf40a49a7cd0c5c97001bf P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:27.226413 14199 raft_consensus.cc:504] T 3d18f12528cf40a49a7cd0c5c97001bf P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.227782 13777 raft_consensus.cc:2258] T 2e7cc97b8b594e92bf3bf74131f20456 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:27.229493 13777 ts_tablet_manager.cc:1905] T 2e7cc97b8b594e92bf3bf74131f20456 P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:27.229540 13668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d18f12528cf40a49a7cd0c5c97001bf" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:27.230162 13668 raft_consensus.cc:2454] T 3d18f12528cf40a49a7cd0c5c97001bf P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 0.
I20260925 09:58:27.230623 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d18f12528cf40a49a7cd0c5c97001bf" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:27.231148 13363 raft_consensus.cc:2454] T 3d18f12528cf40a49a7cd0c5c97001bf P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 0.
I20260925 09:58:27.231468 13440 leader_election.cc:302] T 3d18f12528cf40a49a7cd0c5c97001bf P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:27.232249 14302 raft_consensus.cc:2781] T 3d18f12528cf40a49a7cd0c5c97001bf P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:27.228319 14199 leader_election.cc:288] T 3d18f12528cf40a49a7cd0c5c97001bf P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:27.232576 14302 raft_consensus.cc:482] T 3d18f12528cf40a49a7cd0c5c97001bf P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:27.232859 14302 raft_consensus.cc:3037] T 3d18f12528cf40a49a7cd0c5c97001bf P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:27.233349 14295 raft_consensus.cc:482] T b9cf55829fed4c93a8eab92cee8920c9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:27.227810 14274 raft_consensus.cc:348] T 0e3f1d9381b4450cb6c15525699efbc9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:27.233721 14295 raft_consensus.cc:504] T b9cf55829fed4c93a8eab92cee8920c9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.235682 14274 raft_consensus.cc:374] T 0e3f1d9381b4450cb6c15525699efbc9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:27.235757 14295 leader_election.cc:288] T b9cf55829fed4c93a8eab92cee8920c9 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:27.237129 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9cf55829fed4c93a8eab92cee8920c9" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
W20260925 09:58:27.238375 13298 leader_election.cc:341] T b9cf55829fed4c93a8eab92cee8920c9 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:27.239192 13668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9cf55829fed4c93a8eab92cee8920c9" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:27.239009 14274 raft_consensus.cc:729] T 0e3f1d9381b4450cb6c15525699efbc9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
W20260925 09:58:27.240571 13297 leader_election.cc:341] T b9cf55829fed4c93a8eab92cee8920c9 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:27.240911 13297 leader_election.cc:302] T b9cf55829fed4c93a8eab92cee8920c9 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:58:27.240378 13781 ts_tablet_manager.cc:1918] T 2e7cc97b8b594e92bf3bf74131f20456 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:27.241860 14295 raft_consensus.cc:2726] T b9cf55829fed4c93a8eab92cee8920c9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:58:27.245208 14302 raft_consensus.cc:504] T 3d18f12528cf40a49a7cd0c5c97001bf P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.247184 14302 leader_election.cc:288] T 3d18f12528cf40a49a7cd0c5c97001bf P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Requested vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:27.247903 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d18f12528cf40a49a7cd0c5c97001bf" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843"
I20260925 09:58:27.248392 13363 raft_consensus.cc:3037] T 3d18f12528cf40a49a7cd0c5c97001bf P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:27.248262 13668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d18f12528cf40a49a7cd0c5c97001bf" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
I20260925 09:58:27.248740 13668 raft_consensus.cc:3037] T 3d18f12528cf40a49a7cd0c5c97001bf P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:27.255319 13363 raft_consensus.cc:2454] T 3d18f12528cf40a49a7cd0c5c97001bf P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:27.256511 13438 leader_election.cc:302] T 3d18f12528cf40a49a7cd0c5c97001bf P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:27.255717 14274 consensus_queue.cc:260] T 0e3f1d9381b4450cb6c15525699efbc9 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:27.257472 14199 raft_consensus.cc:2781] T 3d18f12528cf40a49a7cd0c5c97001bf P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:27.257931 14199 raft_consensus.cc:686] T 3d18f12528cf40a49a7cd0c5c97001bf P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: Becoming Leader. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Running, Role: LEADER
I20260925 09:58:27.258925 14199 consensus_queue.cc:237] T 3d18f12528cf40a49a7cd0c5c97001bf P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.260720 13780 ts_tablet_manager.cc:1918] T 8313c1ae006341bc84438085dd38f8ad P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:27.261080 13780 log.cc:1196] T 8313c1ae006341bc84438085dd38f8ad P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/8313c1ae006341bc84438085dd38f8ad
I20260925 09:58:27.261823 13780 ts_tablet_manager.cc:1939] T 8313c1ae006341bc84438085dd38f8ad P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:27.262882 13781 log.cc:1196] T 2e7cc97b8b594e92bf3bf74131f20456 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/2e7cc97b8b594e92bf3bf74131f20456
I20260925 09:58:27.262931 13668 raft_consensus.cc:2454] T 3d18f12528cf40a49a7cd0c5c97001bf P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:27.268179 13781 ts_tablet_manager.cc:1939] T 2e7cc97b8b594e92bf3bf74131f20456 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:27.271116 14274 ts_tablet_manager.cc:1428] T 0e3f1d9381b4450cb6c15525699efbc9 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.059s user 0.009s sys 0.000s
I20260925 09:58:27.273312 13670 raft_consensus.cc:2454] T d3efa619592045b8be65e6b789b8291a P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 1.
I20260925 09:58:27.274478 13297 leader_election.cc:302] T d3efa619592045b8be65e6b789b8291a P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, b3082cf15dc74c0480e5d33ecf592843; no voters: c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:58:27.282787 14199 raft_consensus.cc:482] T 0e3f1d9381b4450cb6c15525699efbc9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:27.283296 14199 raft_consensus.cc:504] T 0e3f1d9381b4450cb6c15525699efbc9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:27.288072 13780 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54548 (request call id 135) took 1759 ms. Trace:
I20260925 09:58:27.288461 13780 rpcz_store.cc:276] 0925 09:58:25.528354 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:25.528494 (+ 140us) service_pool.cc:227] Handling call
0925 09:58:26.998418 (+1469924us) ts_tablet_manager.cc:1145] Deleting tablet 8313c1ae006341bc84438085dd38f8ad
0925 09:58:26.998642 (+ 224us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:58:27.176430 (+177788us) tablet_metadata.cc:641] Metadata flushed
0925 09:58:27.260498 (+ 84068us) tablet_metadata.cc:641] Metadata flushed
0925 09:58:27.288006 (+ 27508us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1377476}
I20260925 09:58:27.289129 13670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e3f1d9381b4450cb6c15525699efbc9" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:27.289538 13780 tablet_replica.cc:330] T a8a89d2315f54a09ad5d6506f2c4797c P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:27.290284 13780 raft_consensus.cc:2229] T a8a89d2315f54a09ad5d6506f2c4797c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:27.290715 13780 raft_consensus.cc:2258] T a8a89d2315f54a09ad5d6506f2c4797c P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:27.290704 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet 8313c1ae006341bc84438085dd38f8ad (table default.table_1238f9df428247dc9b9ca7c59823bfe4 [id=5eb4ef6bf2c344bdb26e429f484c59a4]) successfully deleted
W20260925 09:58:27.291577 13440 leader_election.cc:341] T 0e3f1d9381b4450cb6c15525699efbc9 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:27.295879 13770 tablet_bootstrap.cc:492] T 0e3f1d9381b4450cb6c15525699efbc9 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:27.296244 13770 ts_tablet_manager.cc:1397] T 0e3f1d9381b4450cb6c15525699efbc9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.161s user 0.010s sys 0.007s
I20260925 09:58:27.298205 13770 raft_consensus.cc:348] T 0e3f1d9381b4450cb6c15525699efbc9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:27.298812 13770 raft_consensus.cc:374] T 0e3f1d9381b4450cb6c15525699efbc9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:27.299044 13770 raft_consensus.cc:729] T 0e3f1d9381b4450cb6c15525699efbc9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:27.300387 13770 consensus_queue.cc:260] T 0e3f1d9381b4450cb6c15525699efbc9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:27.293988 14199 leader_election.cc:288] T 0e3f1d9381b4450cb6c15525699efbc9 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731), b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161)
I20260925 09:58:27.308535 14295 raft_consensus.cc:2781] T d3efa619592045b8be65e6b789b8291a P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:27.308992 14295 raft_consensus.cc:686] T d3efa619592045b8be65e6b789b8291a P b3082cf15dc74c0480e5d33ecf592843 [term 1 LEADER]: Becoming Leader. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Running, Role: LEADER
I20260925 09:58:27.308151 14274 tablet_bootstrap.cc:492] T d3efa619592045b8be65e6b789b8291a P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:27.309741 14295 consensus_queue.cc:237] T d3efa619592045b8be65e6b789b8291a P b3082cf15dc74c0480e5d33ecf592843 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.313220 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e3f1d9381b4450cb6c15525699efbc9" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:27.314514 13363 raft_consensus.cc:2454] T 0e3f1d9381b4450cb6c15525699efbc9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 0.
I20260925 09:58:27.315980 13438 leader_election.cc:302] T 0e3f1d9381b4450cb6c15525699efbc9 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237
I20260925 09:58:27.316677 14199 raft_consensus.cc:2781] T 0e3f1d9381b4450cb6c15525699efbc9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:27.319008 14199 raft_consensus.cc:482] T 0e3f1d9381b4450cb6c15525699efbc9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:27.319491 14199 raft_consensus.cc:3037] T 0e3f1d9381b4450cb6c15525699efbc9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:27.320598 13780 ts_tablet_manager.cc:1905] T a8a89d2315f54a09ad5d6506f2c4797c P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:27.320782 13772 tablet_bootstrap.cc:492] T 4b61aaac33514c999482b824e0839a72 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:27.321175 13772 ts_tablet_manager.cc:1397] T 4b61aaac33514c999482b824e0839a72 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.158s user 0.013s sys 0.000s
I20260925 09:58:27.323374 13772 raft_consensus.cc:348] T 4b61aaac33514c999482b824e0839a72 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.324046 13772 raft_consensus.cc:374] T 4b61aaac33514c999482b824e0839a72 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:27.324286 13772 raft_consensus.cc:729] T 4b61aaac33514c999482b824e0839a72 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:27.325965 13772 consensus_queue.cc:260] T 4b61aaac33514c999482b824e0839a72 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.329221 13772 ts_tablet_manager.cc:1428] T 4b61aaac33514c999482b824e0839a72 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.008s user 0.003s sys 0.003s
I20260925 09:58:27.333140 13770 ts_tablet_manager.cc:1428] T 0e3f1d9381b4450cb6c15525699efbc9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.037s user 0.005s sys 0.001s
I20260925 09:58:27.339120 14199 raft_consensus.cc:504] T 0e3f1d9381b4450cb6c15525699efbc9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:27.342330 14274 tablet_bootstrap.cc:654] T d3efa619592045b8be65e6b789b8291a P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:27.346951 13670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e3f1d9381b4450cb6c15525699efbc9" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
I20260925 09:58:27.348115 13670 raft_consensus.cc:3037] T 0e3f1d9381b4450cb6c15525699efbc9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:27.361173 13770 tablet_bootstrap.cc:492] T 4b61aaac33514c999482b824e0839a72 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:27.366813 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e3f1d9381b4450cb6c15525699efbc9" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843"
I20260925 09:58:27.367331 13363 raft_consensus.cc:3037] T 0e3f1d9381b4450cb6c15525699efbc9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:27.373524 13363 raft_consensus.cc:2454] T 0e3f1d9381b4450cb6c15525699efbc9 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:27.375280 13438 leader_election.cc:302] T 0e3f1d9381b4450cb6c15525699efbc9 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:27.376091 14302 raft_consensus.cc:2781] T 0e3f1d9381b4450cb6c15525699efbc9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:27.380025 14302 raft_consensus.cc:686] T 0e3f1d9381b4450cb6c15525699efbc9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: Becoming Leader. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Running, Role: LEADER
I20260925 09:58:27.380944 14302 consensus_queue.cc:237] T 0e3f1d9381b4450cb6c15525699efbc9 P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:27.382678 14295 consensus_queue.cc:1035] T 90d39744f6334e42bd32c26d13e6912c P b3082cf15dc74c0480e5d33ecf592843 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:27.386206 14199 leader_election.cc:288] T 0e3f1d9381b4450cb6c15525699efbc9 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Requested vote from peers a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731), b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161)
W20260925 09:58:27.387260 14324 log.cc:924] Time spent T d3efa619592045b8be65e6b789b8291a P b3082cf15dc74c0480e5d33ecf592843: Append to log took a long time: real 0.051s user 0.001s sys 0.000s
I20260925 09:58:27.391427 13777 ts_tablet_manager.cc:1918] T 2e7cc97b8b594e92bf3bf74131f20456 P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:27.391867 13777 log.cc:1196] T 2e7cc97b8b594e92bf3bf74131f20456 P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/2e7cc97b8b594e92bf3bf74131f20456
I20260925 09:58:27.392544 13777 ts_tablet_manager.cc:1939] T 2e7cc97b8b594e92bf3bf74131f20456 P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:27.393649 14295 raft_consensus.cc:482] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:27.394101 14295 raft_consensus.cc:504] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.395594 13066 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet 2e7cc97b8b594e92bf3bf74131f20456 (table default.table_5f3286f12ae947ef8c1dfa71d8dd21d2 [id=07c6108c574e407f827bc4ab4ffdf345]) successfully deleted
I20260925 09:58:27.396674 14295 leader_election.cc:288] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
W20260925 09:58:27.397533 13298 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
W20260925 09:58:27.403888 13297 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
I20260925 09:58:27.404593 13297 leader_election.cc:302] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:58:27.405601 14295 raft_consensus.cc:2726] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:58:27.409902 14274 tablet_bootstrap.cc:492] T d3efa619592045b8be65e6b789b8291a P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:27.413120 13670 raft_consensus.cc:2454] T 0e3f1d9381b4450cb6c15525699efbc9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:27.425595 14274 ts_tablet_manager.cc:1397] T d3efa619592045b8be65e6b789b8291a P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.119s user 0.010s sys 0.005s
I20260925 09:58:27.443861 13780 ts_tablet_manager.cc:1918] T a8a89d2315f54a09ad5d6506f2c4797c P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:27.447680 13780 log.cc:1196] T a8a89d2315f54a09ad5d6506f2c4797c P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/a8a89d2315f54a09ad5d6506f2c4797c
I20260925 09:58:27.448426 13780 ts_tablet_manager.cc:1939] T a8a89d2315f54a09ad5d6506f2c4797c P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:27.450274 13780 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54548 (request call id 137) took 1605 ms. Trace:
I20260925 09:58:27.450419 14298 consensus_queue.cc:1035] T 90d39744f6334e42bd32c26d13e6912c P b3082cf15dc74c0480e5d33ecf592843 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:58:27.456717 13780 rpcz_store.cc:276] 0925 09:58:25.845324 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:25.846150 (+ 826us) service_pool.cc:227] Handling call
0925 09:58:27.289388 (+1443238us) ts_tablet_manager.cc:1145] Deleting tablet a8a89d2315f54a09ad5d6506f2c4797c
0925 09:58:27.289427 (+ 39us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:58:27.387242 (+ 97815us) tablet_metadata.cc:641] Metadata flushed
0925 09:58:27.443639 (+ 56397us) tablet_metadata.cc:641] Metadata flushed
0925 09:58:27.450219 (+ 6580us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1442613}
I20260925 09:58:27.464399 14037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70718e3f6f9f489089c0d8354380ed6c" candidate_term: 5 candidate_status { last_received { term: 4 index: 195 } } ignore_live_leader: false dest_uuid: "61e29afab0cf46819cbffc91248bf761" is_pre_election: true
I20260925 09:58:27.465294 14037 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70718e3f6f9f489089c0d8354380ed6c in term 4.
I20260925 09:58:27.466859 12992 leader_election.cc:302] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61e29afab0cf46819cbffc91248bf761, 70718e3f6f9f489089c0d8354380ed6c; no voters: b7d0c1c2f1ed4e7a9ed54e73869d53b0
I20260925 09:58:27.467710 13780 tablet_replica.cc:330] T dedf3e0a273d43ba84a861b36d0707ef P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:27.467936 14316 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260925 09:58:27.468220 14316 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 4 FOLLOWER]: Starting leader election (detected failure of leader b7d0c1c2f1ed4e7a9ed54e73869d53b0)
I20260925 09:58:27.468572 14316 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 4 FOLLOWER]: Advancing to term 5
I20260925 09:58:27.468371 13780 raft_consensus.cc:2229] T dedf3e0a273d43ba84a861b36d0707ef P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:27.472041 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet a8a89d2315f54a09ad5d6506f2c4797c (table default.table_47c0ac37d06f4f45a701b0910a42c579 [id=714d733484104c508e7ced7cb14148bf]) successfully deleted
I20260925 09:58:27.472432 13780 raft_consensus.cc:2258] T dedf3e0a273d43ba84a861b36d0707ef P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:27.416772 13770 tablet_bootstrap.cc:654] T 4b61aaac33514c999482b824e0839a72 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:27.474109 13780 ts_tablet_manager.cc:1905] T dedf3e0a273d43ba84a861b36d0707ef P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:27.480451 14316 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:27.481602 14274 raft_consensus.cc:348] T d3efa619592045b8be65e6b789b8291a P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.487442 14269 raft_consensus.cc:482] T 0090f685d0184822b41fd3e731874567 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:27.487776 14274 raft_consensus.cc:374] T d3efa619592045b8be65e6b789b8291a P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:27.488059 14274 raft_consensus.cc:729] T d3efa619592045b8be65e6b789b8291a P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:27.487903 14269 raft_consensus.cc:504] T 0090f685d0184822b41fd3e731874567 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.492848 13670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0090f685d0184822b41fd3e731874567" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
W20260925 09:58:27.494031 13297 leader_election.cc:341] T 0090f685d0184822b41fd3e731874567 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:27.499881 14274 consensus_queue.cc:260] T d3efa619592045b8be65e6b789b8291a P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.500591 14269 leader_election.cc:288] T 0090f685d0184822b41fd3e731874567 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:27.502003 13770 tablet_bootstrap.cc:492] T 4b61aaac33514c999482b824e0839a72 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:27.502544 13770 ts_tablet_manager.cc:1397] T 4b61aaac33514c999482b824e0839a72 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.142s user 0.009s sys 0.002s
I20260925 09:58:27.506078 14274 ts_tablet_manager.cc:1428] T d3efa619592045b8be65e6b789b8291a P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.079s user 0.002s sys 0.005s
W20260925 09:58:27.506181 13298 leader_election.cc:341] T 0090f685d0184822b41fd3e731874567 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: 0090f685d0184822b41fd3e731874567
I20260925 09:58:27.506584 13298 leader_election.cc:302] T 0090f685d0184822b41fd3e731874567 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:58:27.506928 14274 tablet_bootstrap.cc:492] T 4b61aaac33514c999482b824e0839a72 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:27.507622 14269 raft_consensus.cc:2726] T 0090f685d0184822b41fd3e731874567 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:58:27.508986 13770 raft_consensus.cc:348] T 4b61aaac33514c999482b824e0839a72 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.509680 13770 raft_consensus.cc:374] T 4b61aaac33514c999482b824e0839a72 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:27.511080 13090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70718e3f6f9f489089c0d8354380ed6c" candidate_term: 5 candidate_status { last_received { term: 4 index: 195 } } ignore_live_leader: false dest_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0"
I20260925 09:58:27.511492 13770 raft_consensus.cc:729] T 4b61aaac33514c999482b824e0839a72 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:27.511330 14316 leader_election.cc:288] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [CANDIDATE]: Term 5 election: Requested vote from peers b7d0c1c2f1ed4e7a9ed54e73869d53b0 (127.10.9.61:40425), 61e29afab0cf46819cbffc91248bf761 (127.10.9.60:42083)
I20260925 09:58:27.512090 14037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70718e3f6f9f489089c0d8354380ed6c" candidate_term: 5 candidate_status { last_received { term: 4 index: 195 } } ignore_live_leader: false dest_uuid: "61e29afab0cf46819cbffc91248bf761"
I20260925 09:58:27.512097 13770 consensus_queue.cc:260] T 4b61aaac33514c999482b824e0839a72 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.512604 14037 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 4 FOLLOWER]: Advancing to term 5
I20260925 09:58:27.514561 14274 tablet_bootstrap.cc:654] T 4b61aaac33514c999482b824e0839a72 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:27.526726 14037 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70718e3f6f9f489089c0d8354380ed6c in term 5.
I20260925 09:58:27.527992 13770 ts_tablet_manager.cc:1428] T 4b61aaac33514c999482b824e0839a72 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.025s user 0.006s sys 0.000s
I20260925 09:58:27.531810 13770 tablet_bootstrap.cc:492] T 0090f685d0184822b41fd3e731874567 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:27.531880 13780 ts_tablet_manager.cc:1918] T dedf3e0a273d43ba84a861b36d0707ef P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:27.533842 13780 log.cc:1196] T dedf3e0a273d43ba84a861b36d0707ef P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/dedf3e0a273d43ba84a861b36d0707ef
I20260925 09:58:27.534479 13780 ts_tablet_manager.cc:1939] T dedf3e0a273d43ba84a861b36d0707ef P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:27.537672 12992 leader_election.cc:302] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61e29afab0cf46819cbffc91248bf761, 70718e3f6f9f489089c0d8354380ed6c; no voters: b7d0c1c2f1ed4e7a9ed54e73869d53b0
I20260925 09:58:27.538713 14316 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 5 FOLLOWER]: Leader election won for term 5
I20260925 09:58:27.548764 14274 tablet_bootstrap.cc:492] T 4b61aaac33514c999482b824e0839a72 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:27.549758 14274 ts_tablet_manager.cc:1397] T 4b61aaac33514c999482b824e0839a72 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.043s user 0.003s sys 0.007s
I20260925 09:58:27.552788 13780 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54548 (request call id 138) took 1630 ms. Trace:
I20260925 09:58:27.558005 14316 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 5 LEADER]: Becoming Leader. State: Replica: 70718e3f6f9f489089c0d8354380ed6c, State: Running, Role: LEADER
I20260925 09:58:27.559002 14316 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 195, Committed index: 195, Last appended: 4.195, Last appended by leader: 195, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:27.556200 13780 rpcz_store.cc:276] 0925 09:58:25.922529 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:25.922682 (+ 153us) service_pool.cc:227] Handling call
0925 09:58:27.467551 (+1544869us) ts_tablet_manager.cc:1145] Deleting tablet dedf3e0a273d43ba84a861b36d0707ef
0925 09:58:27.467613 (+ 62us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:58:27.524259 (+ 56646us) tablet_metadata.cc:641] Metadata flushed
0925 09:58:27.531686 (+ 7427us) tablet_metadata.cc:641] Metadata flushed
0925 09:58:27.552723 (+ 21037us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1544345}
I20260925 09:58:27.564576 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet dedf3e0a273d43ba84a861b36d0707ef (table default.table_f83185097e7343848b564d3d5881c9ab [id=7f3f3dd4c3d74f21878caad38c9c25b9]) successfully deleted
I20260925 09:58:27.561038 14274 raft_consensus.cc:348] T 4b61aaac33514c999482b824e0839a72 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.565472 14274 raft_consensus.cc:374] T 4b61aaac33514c999482b824e0839a72 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:27.566824 13770 tablet_bootstrap.cc:654] T 0090f685d0184822b41fd3e731874567 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:27.566984 14274 raft_consensus.cc:729] T 4b61aaac33514c999482b824e0839a72 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:27.571556 13780 tablet_replica.cc:330] T 2e7cc97b8b594e92bf3bf74131f20456 P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:27.572402 13780 raft_consensus.cc:2229] T 2e7cc97b8b594e92bf3bf74131f20456 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:27.572288 14335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70718e3f6f9f489089c0d8354380ed6c. Latest consensus state: current_term: 5 leader_uuid: "70718e3f6f9f489089c0d8354380ed6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } } }
I20260925 09:58:27.572887 13780 raft_consensus.cc:2258] T 2e7cc97b8b594e92bf3bf74131f20456 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:27.573143 14335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [sys.catalog]: This master's current role is: LEADER
I20260925 09:58:27.570897 14274 consensus_queue.cc:260] T 4b61aaac33514c999482b824e0839a72 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.574568 13780 ts_tablet_manager.cc:1905] T 2e7cc97b8b594e92bf3bf74131f20456 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:27.586766 14336 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:58:27.589986 14274 ts_tablet_manager.cc:1428] T 4b61aaac33514c999482b824e0839a72 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.039s user 0.009s sys 0.000s
I20260925 09:58:27.593904 14274 tablet_bootstrap.cc:492] T b9cf55829fed4c93a8eab92cee8920c9 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:27.613778 13770 tablet_bootstrap.cc:492] T 0090f685d0184822b41fd3e731874567 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:27.614483 13770 ts_tablet_manager.cc:1397] T 0090f685d0184822b41fd3e731874567 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.083s user 0.005s sys 0.012s
I20260925 09:58:27.624913 13770 raft_consensus.cc:348] T 0090f685d0184822b41fd3e731874567 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.628418 14274 tablet_bootstrap.cc:654] T b9cf55829fed4c93a8eab92cee8920c9 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:27.628937 13770 raft_consensus.cc:374] T 0090f685d0184822b41fd3e731874567 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:27.630652 13770 raft_consensus.cc:729] T 0090f685d0184822b41fd3e731874567 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:27.634370 13770 consensus_queue.cc:260] T 0090f685d0184822b41fd3e731874567 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.639715 14269 consensus_queue.cc:1035] T 48ffc9cb12e346e7ae20c7f916ac33d4 P b3082cf15dc74c0480e5d33ecf592843 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:27.640694 14336 catalog_manager.cc:669] Loaded metadata for table default.table_5f3286f12ae947ef8c1dfa71d8dd21d2 [id=07c6108c574e407f827bc4ab4ffdf345]
I20260925 09:58:27.652110 14336 catalog_manager.cc:669] Loaded metadata for table default.table_8f71132b5a004ad89115f7a2278b95d6 [id=0cdc8d8fef5445668c0b0107243e0ae9]
I20260925 09:58:27.656641 14336 catalog_manager.cc:669] Loaded metadata for table default.table_1238f9df428247dc9b9ca7c59823bfe4 [id=5eb4ef6bf2c344bdb26e429f484c59a4]
I20260925 09:58:27.661615 14336 catalog_manager.cc:669] Loaded metadata for table default.table_47c0ac37d06f4f45a701b0910a42c579 [id=714d733484104c508e7ced7cb14148bf]
I20260925 09:58:27.663494 13770 ts_tablet_manager.cc:1428] T 0090f685d0184822b41fd3e731874567 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.049s user 0.007s sys 0.000s
I20260925 09:58:27.665185 14336 catalog_manager.cc:669] Loaded metadata for table default.table_41304460d5a345c88866a7429d816aca [id=77d343c035fa4fd2b3faadcfc4fec52f]
I20260925 09:58:27.666322 14269 consensus_queue.cc:1035] T 48ffc9cb12e346e7ae20c7f916ac33d4 P b3082cf15dc74c0480e5d33ecf592843 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:27.666853 14336 catalog_manager.cc:669] Loaded metadata for table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]
I20260925 09:58:27.668409 14336 catalog_manager.cc:669] Loaded metadata for table default.table_f83185097e7343848b564d3d5881c9ab [id=7f3f3dd4c3d74f21878caad38c9c25b9]
I20260925 09:58:27.674926 13770 tablet_bootstrap.cc:492] T fafc2b657ace4a4381452ce43fad39b6 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:27.677917 13780 ts_tablet_manager.cc:1918] T 2e7cc97b8b594e92bf3bf74131f20456 P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260925 09:58:24.444653 14038 rpcz_store.cc:276] 0925 09:58:22.322244 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:22.322396 (+ 152us) service_pool.cc:227] Handling call
0925 09:58:23.263731 (+941335us) raft_consensus.cc:1457] Updating replica for 124 ops
0925 09:58:23.340985 (+ 77254us) raft_consensus.cc:1520] Early marking committed up to index 6
0925 09:58:23.462504 (+121519us) log.cc:842] Serialized 14 byte log entry
0925 09:58:23.485092 (+ 22588us) raft_consensus.cc:1525] Triggering prepare for 123 ops
0925 09:58:23.485653 (+ 561us) write_op.cc:270] Start()
0925 09:58:23.485772 (+ 119us) write_op.cc:276] Timestamp: P: 1790330293195242 usec, L: 0
0925 09:58:23.493646 (+ 7874us) write_op.cc:270] Start()
0925 09:58:23.493736 (+ 90us) write_op.cc:276] Timestamp: P: 1790330293235345 usec, L: 0
0925 09:58:23.494236 (+ 500us) write_op.cc:270] Start()
0925 09:58:23.494345 (+ 109us) write_op.cc:276] Timestamp: P: 1790330293244848 usec, L: 0
0925 09:58:23.494828 (+ 483us) write_op.cc:270] Start()
0925 09:58:23.494910 (+ 82us) write_op.cc:276] Timestamp: P: 1790330293293836 usec, L: 0
0925 09:58:23.495381 (+ 471us) write_op.cc:270] Start()
0925 09:58:23.495483 (+ 102us) write_op.cc:276] Timestamp: P: 1790330293305576 usec, L: 0
0925 09:58:23.520196 (+ 24713us) write_op.cc:270] Start()
0925 09:58:23.520386 (+ 190us) write_op.cc:276] Timestamp: P: 1790330293360066 usec, L: 0
0925 09:58:23.520906 (+ 520us) write_op.cc:270] Start()
0925 09:58:23.520985 (+ 79us) write_op.cc:276] Timestamp: P: 1790330293502823 usec, L: 0
0925 09:58:23.521383 (+ 398us) write_op.cc:270] Start()
0925 09:58:23.521455 (+ 72us) write_op.cc:276] Timestamp: P: 1790330293542225 usec, L: 0
0925 09:58:23.541003 (+ 19548us) write_op.cc:270] Start()
0925 09:58:23.541112 (+ 109us) write_op.cc:276] Timestamp: P: 1790330293608512 usec, L: 0
0925 09:58:23.541614 (+ 502us) write_op.cc:270] Start()
0925 09:58:23.541686 (+ 72us) write_op.cc:276] Timestamp: P: 1790330293639157 usec, L: 0
0925 09:58:23.542155 (+ 469us) write_op.cc:270] Start()
0925 09:58:23.542227 (+ 72us) write_op.cc:276] Timestamp: P: 1790330293710102 usec, L: 0
0925 09:58:23.542628 (+ 401us) write_op.cc:270] Start()
0925 09:58:23.542720 (+ 92us) write_op.cc:276] Timestamp: P: 1790330293850079 usec, L: 0
0925 09:58:23.543148 (+ 428us) write_op.cc:270] Start()
0925 09:58:23.543213 (+ 65us) write_op.cc:276] Timestamp: P: 1790330293956037 usec, L: 0
0925 09:58:23.546895 (+ 3682us) write_op.cc:270] Start()
0925 09:58:23.546975 (+ 80us) write_op.cc:276] Timestamp: P: 1790330294084942 usec, L: 0
0925 09:58:23.547485 (+ 510us) write_op.cc:270] Start()
0925 09:58:23.547564 (+ 79us) write_op.cc:276] Timestamp: P: 1790330294135975 usec, L: 0
0925 09:58:23.548027 (+ 463us) write_op.cc:270] Start()
0925 09:58:23.548115 (+ 88us) write_op.cc:276] Timestamp: P: 1790330294248802 usec, L: 0
0925 09:58:23.548587 (+ 472us) write_op.cc:270] Start()
0925 09:58:23.548661 (+ 74us) write_op.cc:276] Timestamp: P: 1790330294322048 usec, L: 0
0925 09:58:23.549103 (+ 442us) write_op.cc:270] Start()
0925 09:58:23.549177 (+ 74us) write_op.cc:276] Timestamp: P: 1790330294455409 usec, L: 0
0925 09:58:23.549585 (+ 408us) write_op.cc:270] Start()
0925 09:58:23.549658 (+ 73us) write_op.cc:276] Timestamp: P: 1790330294603932 usec, L: 0
0925 09:58:23.550104 (+ 446us) write_op.cc:270] Start()
0925 09:58:23.550174 (+ 70us) write_op.cc:276] Timestamp: P: 1790330294656402 usec, L: 0
0925 09:58:23.550582 (+ 408us) write_op.cc:270] Start()
0925 09:58:23.550663 (+ 81us) write_op.cc:276] Timestamp: P: 1790330294717577 usec, L: 0
0925 09:58:23.551073 (+ 410us) write_op.cc:270] Start()
0925 09:58:23.551147 (+ 74us) write_op.cc:276] Timestamp: P: 1790330294814655 usec, L: 0
0925 09:58:23.551567 (+ 420us) write_op.cc:270] Start()
0925 09:58:23.551641 (+ 74us) write_op.cc:276] Timestamp: P: 1790330294884877 usec, L: 0
0925 09:58:23.552120 (+ 479us) write_op.cc:270] Start()
0925 09:58:23.552220 (+ 100us) write_op.cc:276] Timestamp: P: 1790330294938015 usec, L: 0
0925 09:58:23.552647 (+ 427us) write_op.cc:270] Start()
0925 09:58:23.552716 (+ 69us) write_op.cc:276] Timestamp: P: 1790330295018617 usec, L: 0
0925 09:58:23.557286 (+ 4570us) write_op.cc:270] Start()
0925 09:58:23.557368 (+ 82us) write_op.cc:276] Timestamp: P: 1790330295022794 usec, L: 0
0925 09:58:23.566108 (+ 8740us) write_op.cc:270] Start()
0925 09:58:23.566205 (+ 97us) write_op.cc:276] Timestamp: P: 1790330295052503 usec, L: 0
0925 09:58:23.566720 (+ 515us) write_op.cc:270] Start()
0925 09:58:23.566798 (+ 78us) write_op.cc:276] Timestamp: P: 1790330295217112 usec, L: 0
0925 09:58:23.567247 (+ 449us) write_op.cc:270] Start()
0925 09:58:23.567316 (+ 69us) write_op.cc:276] Timestamp: P: 1790330295295027 usec, L: 0
0925 09:58:23.567769 (+ 453us) write_op.cc:270] Start()
0925 09:58:23.567852 (+ 83us) write_op.cc:276] Timestamp: P: 1790330295387091 usec, L: 0
0925 09:58:23.568263 (+ 411us) write_op.cc:270] Start()
0925 09:58:23.568333 (+ 70us) write_op.cc:276] Timestamp: P: 1790330295568611 usec, L: 0
0925 09:58:23.568763 (+ 430us) write_op.cc:270] Start()
0925 09:58:23.568846 (+ 83us) write_op.cc:276] Timestamp: P: 1790330295667383 usec, L: 0
0925 09:58:23.569363 (+ 517us) write_op.cc:270] Start()
0925 09:58:23.569444 (+ 81us) write_op.cc:276] Timestamp: P: 1790330295694598 usec, L: 0
0925 09:58:23.569915 (+ 471us) write_op.cc:270] Start()
0925 09:58:23.569992 (+ 77us) write_op.cc:276] Timestamp: P: 1790330295760302 usec, L: 0
0925 09:58:23.570428 (+ 436us) write_op.cc:270] Start()
0925 09:58:23.570503 (+ 75us) write_op.cc:276] Timestamp: P: 1790330295856509 usec, L: 0
0925 09:58:23.570931 (+ 428us) write_op.cc:270] Start()
0925 09:58:23.571005 (+ 74us) write_op.cc:276] Timestamp: P: 1790330295980265 usec, L: 0
0925 09:58:23.586503 (+ 15498us) write_op.cc:270] Start()
0925 09:58:23.586599 (+ 96us) write_op.cc:276] Timestamp: P: 1790330296030453 usec, L: 0
0925 09:58:23.587083 (+ 484us) write_op.cc:270] Start()
0925 09:58:23.587161 (+ 78us) write_op.cc:276] Timestamp: P: 1790330296041207 usec, L: 0
0925 09:58:23.587620 (+ 459us) write_op.cc:270] Start()
0925 09:58:23.587696 (+ 76us) write_op.cc:276] Timestamp: P: 1790330296109222 usec, L: 0
0925 09:58:23.588193 (+ 497us) write_op.cc:270] Start()
0925 09:58:23.588275 (+ 82us) write_op.cc:276] Timestamp: P: 1790330296231780 usec, L: 0
0925 09:58:23.588718 (+ 443us) write_op.cc:270] Start()
0925 09:58:23.588795 (+ 77us) write_op.cc:276] Timestamp: P: 1790330296236660 usec, L: 0
0925 09:58:23.589238 (+ 443us) write_op.cc:270] Start()
0925 09:58:23.589312 (+ 74us) write_op.cc:276] Timestamp: P: 1790330296302264 usec, L: 0
0925 09:58:23.589766 (+ 454us) write_op.cc:270] Start()
0925 09:58:23.589848 (+ 82us) write_op.cc:276] Timestamp: P: 1790330296337544 usec, L: 0
0925 09:58:23.590304 (+ 456us) write_op.cc:270] Start()
0925 09:58:23.590379 (+ 75us) write_op.cc:276] Timestamp: P: 1790330296410823 usec, L: 0
0925 09:58:23.590821 (+ 442us) write_op.cc:270] Start()
0925 09:58:23.590915 (+ 94us) write_op.cc:276] Timestamp: P: 1790330296488366 usec, L: 0
0925 09:58:23.596674 (+ 5759us) write_op.cc:270] Start()
0925 09:58:23.596754 (+ 80us) write_op.cc:276] Timestamp: P: 1790330296622233 usec, L: 0
0925 09:58:23.597202 (+ 448us) write_op.cc:270] Start()
0925 09:58:23.597276 (+ 74us) write_op.cc:276] Timestamp: P: 1790330296691983 usec, L: 0
0925 09:58:23.597719 (+ 443us) write_op.cc:270] Start()
0925 09:58:23.597876 (+ 157us) write_op.cc:276] Timestamp: P: 1790330296751511 usec, L: 0
0925 09:58:23.598378 (+ 502us) write_op.cc:270] Start()
0925 09:58:23.598455 (+ 77us) write_op.cc:276] Timestamp: P: 1790330296803711 usec, L: 0
0925 09:58:23.598879 (+ 424us) write_op.cc:270] Start()
0925 09:58:23.598956 (+ 77us) write_op.cc:276] Timestamp: P: 1790330297095827 usec, L: 0
0925 09:58:23.599383 (+ 427us) write_op.cc:270] Start()
0925 09:58:23.599482 (+ 99us) write_op.cc:276] Timestamp: P: 1790330297174383 usec, L: 0
0925 09:58:23.599973 (+ 491us) write_op.cc:270] Start()
0925 09:58:23.600051 (+ 78us) write_op.cc:276] Timestamp: P: 1790330297256740 usec, L: 0
0925 09:58:23.600477 (+ 426us) write_op.cc:270] Start()
0925 09:58:23.600561 (+ 84us) write_op.cc:276] Timestamp: P: 1790330297284506 usec, L: 0
0925 09:58:23.600999 (+ 438us) write_op.cc:270] Start()
0925 09:58:23.601093 (+ 94us) write_op.cc:276] Timestamp: P: 1790330297385531 usec, L: 0
0925 09:58:23.611050 (+ 9957us) write_op.cc:270] Start()
0925 09:58:23.611142 (+ 92us) write_op.cc:276] Timestamp: P: 1790330297424791 usec, L: 0
0925 09:58:23.611662 (+ 520us) write_op.cc:270] Start()
0925 09:58:23.611736 (+ 74us) write_op.cc:276] Timestamp: P: 1790330297586196 usec, L: 0
0925 09:58:23.614311 (+ 2575us) write_op.cc:270] Start()
0925 09:58:23.614398 (+ 87us) write_op.cc:276] Timestamp: P: 1790330297611192 usec, L: 0
0925 09:58:23.616297 (+ 1899us) write_op.cc:270] Start()
0925 09:58:23.616389 (+ 92us) write_op.cc:276] Timestamp: P: 1790330297791879 usec, L: 0
0925 09:58:23.617867 (+ 1478us) write_op.cc:270] Start()
0925 09:58:23.618043 (+ 176us) write_op.cc:276] Timestamp: P: 1790330297816470 usec, L: 0
0925 09:58:23.618488 (+ 445us) write_op.cc:270] Start()
0925 09:58:23.618559 (+ 71us) write_op.cc:276] Timestamp: P: 1790330297925257 usec, L: 0
0925 09:58:23.618982 (+ 423us) write_op.cc:270] Start()
0925 09:58:23.619053 (+ 71us) write_op.cc:276] Timestamp: P: 1790330298028712 usec, L: 0
0925 09:58:23.619522 (+ 469us) write_op.cc:270] Start()
0925 09:58:23.619599 (+ 77us) write_op.cc:276] Timestamp: P: 1790330298084702 usec, L: 0
0925 09:58:23.620051 (+ 452us) write_op.cc:270] Start()
0925 09:58:23.620140 (+ 89us) write_op.cc:276] Timestamp: P: 1790330298086523 usec, L: 0
0925 09:58:23.620628 (+ 488us) write_op.cc:270] Start()
0925 09:58:23.620703 (+ 75us) write_op.cc:276] Timestamp: P: 1790330298219590 usec, L: 0
0925 09:58:23.621270 (+ 567us) write_op.cc:270] Start()
0925 09:58:23.621349 (+ 79us) write_op.cc:276] Timestamp: P: 1790330298221720 usec, L: 0
0925 09:58:23.621762 (+ 413us) write_op.cc:270] Start()
0925 09:58:23.621835 (+ 73us) write_op.cc:276] Timestamp: P: 1790330298355233 usec, L: 0
0925 09:58:23.622269 (+ 434us) write_op.cc:270] Start()
0925 09:58:23.622344 (+ 75us) write_op.cc:276] Timestamp: P: 1790330298421653 usec, L: 0
0925 09:58:23.622755 (+ 411us) write_op.cc:270] Start()
0925 09:58:23.622825 (+ 70us) write_op.cc:276] Timestamp: P: 1790330298428917 usec, L: 0
0925 09:58:23.625254 (+ 2429us) write_op.cc:270] Start()
0925 09:58:23.625347 (+ 93us) write_op.cc:276] Timestamp: P: 1790330298538415 usec, L: 0
0925 09:58:23.631174 (+ 5827us) write_op.cc:270] Start()
0925 09:58:23.631261 (+ 87us) write_op.cc:276] Timestamp: P: 1790330298586405 usec, L: 0
0925 09:58:23.631832 (+ 571us) write_op.cc:270] Start()
0925 09:58:23.631926 (+ 94us) write_op.cc:276] Timestamp: P: 1790330298646203 usec, L: 0
0925 09:58:23.632364 (+ 438us) write_op.cc:270] Start()
0925 09:58:23.632442 (+ 78us) write_op.cc:276] Timestamp: P: 1790330298704915 usec, L: 0
0925 09:58:23.633066 (+ 624us) write_op.cc:270] Start()
0925 09:58:23.633258 (+ 192us) write_op.cc:276] Timestamp: P: 1790330298736549 usec, L: 0
0925 09:58:23.633707 (+ 449us) write_op.cc:270] Start()
0925 09:58:23.633787 (+ 80us) write_op.cc:276] Timestamp: P: 1790330298800288 usec, L: 0
0925 09:58:23.635240 (+ 1453us) write_op.cc:270] Start()
0925 09:58:23.635315 (+ 75us) write_op.cc:276] Timestamp: P: 1790330298842323 usec, L: 0
0925 09:58:23.635768 (+ 453us) write_op.cc:270] Start()
0925 09:58:23.635855 (+ 87us) write_op.cc:276] Timestamp: P: 1790330298861293 usec, L: 0
0925 09:58:23.644088 (+ 8233us) write_op.cc:270] Start()
0925 09:58:23.644176 (+ 88us) write_op.cc:276] Timestamp: P: 1790330298882310 usec, L: 0
0925 09:58:23.650771 (+ 6595us) write_op.cc:270] Start()
0925 09:58:23.650872 (+ 101us) write_op.cc:276] Timestamp: P: 1790330298975726 usec, L: 0
0925 09:58:23.651321 (+ 449us) write_op.cc:270] Start()
0925 09:58:23.651413 (+ 92us) write_op.cc:276] Timestamp: P: 1790330299050819 usec, L: 0
0925 09:58:23.651824 (+ 411us) write_op.cc:270] Start()
0925 09:58:23.651910 (+ 86us) write_op.cc:276] Timestamp: P: 1790330299089458 usec, L: 0
0925 09:58:23.684305 (+ 32395us) write_op.cc:270] Start()
0925 09:58:23.684402 (+ 97us) write_op.cc:276] Timestamp: P: 1790330299094940 usec, L: 0
0925 09:58:23.684908 (+ 506us) write_op.cc:270] Start()
0925 09:58:23.684986 (+ 78us) write_op.cc:276] Timestamp: P: 1790330299116067 usec, L: 0
0925 09:58:23.691479 (+ 6493us) write_op.cc:270] Start()
0925 09:58:23.691578 (+ 99us) write_op.cc:276] Timestamp: P: 1790330299163342 usec, L: 0
0925 09:58:23.692182 (+ 604us) write_op.cc:270] Start()
0925 09:58:23.692268 (+ 86us) write_op.cc:276] Timestamp: P: 1790330299207316 usec, L: 0
0925 09:58:23.692695 (+ 427us) write_op.cc:270] Start()
0925 09:58:23.692772 (+ 77us) write_op.cc:276] Timestamp: P: 1790330299378140 usec, L: 0
0925 09:58:23.693274 (+ 502us) write_op.cc:270] Start()
0925 09:58:23.693351 (+ 77us) write_op.cc:276] Timestamp: P: 1790330299508828 usec, L: 0
0925 09:58:23.693779 (+ 428us) write_op.cc:270] Start()
0925 09:58:23.693858 (+ 79us) write_op.cc:276] Timestamp: P: 1790330299633132 usec, L: 0
0925 09:58:23.694279 (+ 421us) write_op.cc:270] Start()
0925 09:58:23.694386 (+ 107us) write_op.cc:276] Timestamp: P: 1790330299700611 usec, L: 0
0925 09:58:23.694870 (+ 484us) write_op.cc:270] Start()
0925 09:58:23.694951 (+ 81us) write_op.cc:276] Timestamp: P: 1790330299732775 usec, L: 0
0925 09:58:23.701810 (+ 6859us) write_op.cc:270] Start()
0925 09:58:23.701892 (+ 82us) write_op.cc:276] Timestamp: P: 1790330299771029 usec, L: 0
0925 09:58:23.702827 (+ 935us) write_op.cc:270] Start()
0925 09:58:23.702917 (+ 90us) write_op.cc:276] Timestamp: P: 1790330299877970 usec, L: 0
0925 09:58:23.703357 (+ 440us) write_op.cc:270] Start()
0925 09:58:23.703455 (+ 98us) write_op.cc:276] Timestamp: P: 1790330299984835 usec, L: 0
0925 09:58:23.705153 (+ 1698us) write_op.cc:270] Start()
0925 09:58:23.705235 (+ 82us) write_op.cc:276] Timestamp: P: 1790330300054246 usec, L: 0
0925 09:58:23.705685 (+ 450us) write_op.cc:270] Start()
0925 09:58:23.705758 (+ 73us) write_op.cc:276] Timestamp: P: 1790330300122924 usec, L: 0
0925 09:58:23.706313 (+ 555us) write_op.cc:270] Start()
0925 09:58:23.706425 (+ 112us) write_op.cc:276] Timestamp: P: 1790330300185371 usec, L: 0
0925 09:58:23.712467 (+ 6042us) write_op.cc:270] Start()
0925 09:58:23.712550 (+ 83us) write_op.cc:276] Timestamp: P: 1790330300217069 usec, L: 0
0925 09:58:23.713006 (+ 456us) write_op.cc:270] Start()
0925 09:58:23.713100 (+ 94us) write_op.cc:276] Timestamp: P: 1790330300312911 usec, L: 0
0925 09:58:23.713513 (+ 413us) write_op.cc:270] Start()
0925 09:58:23.713686 (+ 173us) write_op.cc:276] Timestamp: P: 1790330300416659 usec, L: 0
0925 09:58:23.714217 (+ 531us) write_op.cc:270] Start()
0925 09:58:23.714295 (+ 78us) write_op.cc:276] Timestamp: P: 1790330300421127 usec, L: 0
0925 09:58:23.714709 (+ 414us) write_op.cc:270] Start()
0925 09:58:23.714780 (+ 71us) write_op.cc:276] Timestamp: P: 1790330300521588 usec, L: 0
0925 09:58:23.715219 (+ 439us) write_op.cc:270] Start()
0925 09:58:23.715297 (+ 78us) write_op.cc:276] Timestamp: P: 1790330300525681 usec, L: 0
0925 09:58:23.715757 (+ 460us) write_op.cc:270] Start()
0925 09:58:23.715833 (+ 76us) write_op.cc:276] Timestamp: P: 1790330300592277 usec, L: 0
0925 09:58:23.716285 (+ 452us) write_op.cc:270] Start()
0925 09:58:23.754568 (+ 38283us) write_op.cc:276] Timestamp: P: 1790330300602524 usec, L: 0
0925 09:58:23.755160 (+ 592us) write_op.cc:270] Start()
0925 09:58:23.755244 (+ 84us) write_op.cc:276] Timestamp: P: 1790330300633288 usec, L: 0
0925 09:58:23.755781 (+ 537us) write_op.cc:270] Start()
0925 09:58:23.755847 (+ 66us) write_op.cc:276] Timestamp: P: 1790330300820803 usec, L: 0
0925 09:58:23.756268 (+ 421us) write_op.cc:270] Start()
0925 09:58:23.756345 (+ 77us) write_op.cc:276] Timestamp: P: 1790330300824617 usec, L: 0
0925 09:58:23.756758 (+ 413us) write_op.cc:270] Start()
0925 09:58:23.756842 (+ 84us) write_op.cc:276] Timestamp: P: 1790330300889998 usec, L: 0
0925 09:58:23.757277 (+ 435us) write_op.cc:270] Start()
0925 09:58:23.757357 (+ 80us) write_op.cc:276] Timestamp: P: 1790330300946684 usec, L: 0
0925 09:58:23.762551 (+ 5194us) write_op.cc:270] Start()
0925 09:58:23.762641 (+ 90us) write_op.cc:276] Timestamp: P: 1790330301034202 usec, L: 0
0925 09:58:23.763142 (+ 501us) write_op.cc:270] Start()
0925 09:58:23.763217 (+ 75us) write_op.cc:276] Timestamp: P: 1790330301131243 usec, L: 0
0925 09:58:23.781447 (+ 18230us) write_op.cc:270] Start()
0925 09:58:23.781550 (+ 103us) write_op.cc:276] Timestamp: P: 1790330301145066 usec, L: 0
0925 09:58:23.782115 (+ 565us) write_op.cc:270] Start()
0925 09:58:23.782195 (+ 80us) write_op.cc:276] Timestamp: P: 1790330301214349 usec, L: 0
0925 09:58:23.782652 (+ 457us) write_op.cc:270] Start()
0925 09:58:23.782724 (+ 72us) write_op.cc:276] Timestamp: P: 1790330301237356 usec, L: 0
0925 09:58:23.783139 (+ 415us) write_op.cc:270] Start()
0925 09:58:23.783213 (+ 74us) write_op.cc:276] Timestamp: P: 1790330301393681 usec, L: 0
0925 09:58:23.791770 (+ 8557us) write_op.cc:270] Start()
0925 09:58:23.791863 (+ 93us) write_op.cc:276] Timestamp: P: 1790330301521771 usec, L: 0
0925 09:58:23.792342 (+ 479us) write_op.cc:270] Start()
0925 09:58:23.792416 (+ 74us) write_op.cc:276] Timestamp: P: 1790330301659769 usec, L: 0
0925 09:58:23.792866 (+ 450us) write_op.cc:270] Start()
0925 09:58:23.792942 (+ 76us) write_op.cc:276] Timestamp: P: 1790330301708673 usec, L: 0
0925 09:58:23.793381 (+ 439us) write_op.cc:270] Start()
0925 09:58:23.793454 (+ 73us) write_op.cc:276] Timestamp: P: 1790330301801473 usec, L: 0
0925 09:58:23.793883 (+ 429us) write_op.cc:270] Start()
0925 09:58:23.793962 (+ 79us) write_op.cc:276] Timestamp: P: 1790330301884863 usec, L: 0
0925 09:58:23.794460 (+ 498us) write_op.cc:270] Start()
0925 09:58:23.794529 (+ 69us) write_op.cc:276] Timestamp: P: 1790330301887307 usec, L: 0
0925 09:58:23.794939 (+ 410us) write_op.cc:270] Start()
0925 09:58:23.795027 (+ 88us) write_op.cc:276] Timestamp: P: 1790330301959071 usec, L: 0
0925 09:58:23.810510 (+ 15483us) write_op.cc:270] Start()
0925 09:58:23.810618 (+ 108us) write_op.cc:276] Timestamp: P: 1790330301996475 usec, L: 0
0925 09:58:23.811192 (+ 574us) write_op.cc:270] Start()
0925 09:58:23.811274 (+ 82us) write_op.cc:276] Timestamp: P: 1790330302008029 usec, L: 0
0925 09:58:23.894290 (+ 83016us) spinlock_profiling.cc:243] Waited 58.5 ms on lock 0x7b2c00004908. stack: 00007f46800e9033 00007f46800e8f30 00007f467fc6a534 00000000004db2b4 00000000004db249 00007f4686364e11 00007f4686374442 00007f46870abc11 00007f468707058a 00007f468706f99e 00007f468804c2e9 00007f4686822684 00007f4686822612 00007f4686822541 00007f46868224bc 00007f4686821772
0925 09:58:23.965820 (+ 71530us) log.cc:842] Serialized 64959 byte log entry
0925 09:58:24.059181 (+ 93361us) raft_consensus.cc:1631] Marking committed up to 127
0925 09:58:24.305092 (+245911us) raft_consensus.cc:1683] Filling consensus response to leader.
0925 09:58:24.307150 (+ 2058us) spinlock_profiling.cc:243] Waited 820 ms on lock 0x7b5400010754. stack: 00007f46800e9033 00007f46800e8f30 00007f467fc6a534 00000000004db2b4 00000000004db249 00007f4686ff5af0 00007f4687078f2a 00007f468907030c 00007f468907872c 00007f46890786d8 00007f4689078658 00007f4689078613 00007f46890778bc 00007f46874a90e6 00007f46874a9078 00007f468708f2fa
0925 09:58:24.311970 (+ 4820us) spinlock_profiling.cc:243] Waited 970 ms on lock 0x7b5400010754. stack: 00007f46800e9033 00007f46800e8f30 00007f467fc6a534 00000000004db2b4 00000000004db249 00007f4686ff5af0 00007f4687078f2a 00007f468907030c 00007f468907872c 00007f46890786d8 00007f4689078658 00007f4689078613 00007f46890778bc 00007f46874a90e6 00007f46874a9078 00007f468708f2fa
0925 09:58:24.443645 (+131675us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0925 09:58:24.443719 (+ 74us) raft_consensus.cc:1671] finished
0925 09:58:24.443739 (+ 20us) raft_consensus.cc:1677] UpdateReplicas() finished
0925 09:58:24.444186 (+ 447us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:58:23.493417 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:58:23.493957 (+ 540us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:58:23.493987 (+ 30us) write_op.cc:435] Acquired schema lock
0925 09:58:23.494004 (+ 17us) tablet.cc:660] Decoding operations
0925 09:58:23.494317 (+ 313us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:58:23.494398 (+ 81us) write_op.cc:641] Partition lock acquired for write op
0925 09:58:23.494425 (+ 27us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:58:23.494638 (+ 213us) tablet.cc:699] Row locks acquired
0925 09:58:23.494654 (+ 16us) write_op.cc:260] PREPARE: finished
0925 09:58:24.059378 (+564724us) op_driver.cc:444] REPLICATION: finished
0925 09:58:24.114064 (+ 54686us) write_op.cc:301] APPLY: starting
0925 09:58:24.114128 (+ 64us) tablet.cc:1365] starting BulkCheckPresence
0925 09:58:24.114438 (+ 310us) tablet.cc:1368] finished BulkCheckPresence
0925 09:58:24.114458 (+ 20us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:58:24.114946 (+ 488us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:58:24.115189 (+ 243us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 09:58:24.115318 (+ 129us) write_op.cc:312] APPLY: finished
0925 09:58:24.115575 (+ 257us) log.cc:842] Serialized 28 byte log entry
0925 09:58:24.115909 (+ 334us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:24.142536 (+ 26627us) write_op.cc:454] Released schema lock
0925 09:58:24.142586 (+ 50us) write_op.cc:341] FINISH: Updating metrics
0925 09:58:24.142811 (+ 225us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:24.142834 (+ 23us) write_op.cc:454] Released schema lock
Related trace 'op':
0925 09:58:23.494942 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:58:23.495336 (+ 394us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:58:23.495359 (+ 23us) write_op.cc:435] Acquired schema lock
0925 09:58:23.495374 (+ 15us) tablet.cc:660] Decoding operations
0925 09:58:23.495848 (+ 474us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:58:23.495949 (+ 101us) write_op.cc:641] Partition lock acquired for write op
0925 09:58:23.495992 (+ 43us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:58:24.116014 (+620022us) lock_manager.cc:599] Waited 619717us for lock on <redacted> (key hash 8607424894063028186)
0925 09:58:24.116059 (+ 45us) tablet.cc:699] Row locks acquired
0925 09:58:24.116073 (+ 14us) write_op.cc:260] PREPARE: finished
0925 09:58:24.129512 (+ 13439us) op_driver.cc:444] REPLICATION: finished
0925 09:58:24.135506 (+ 5994us) write_op.cc:301] APPLY: starting
0925 09:58:24.136881 (+ 1375us) tablet.cc:1365] starting BulkCheckPresence
0925 09:58:24.136996 (+ 115us) tablet.cc:1368] finished BulkCheckPresence
0925 09:58:24.137016 (+ 20us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:58:24.137237 (+ 221us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:58:24.137433 (+ 196us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:58:24.137478 (+ 45us) write_op.cc:312] APPLY: finished
0925 09:58:24.137698 (+ 220us) log.cc:842] Serialized 22 byte log entry
0925 09:58:24.179677 (+ 41979us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:24.179735 (+ 58us) write_op.cc:454] Released schema lock
0925 09:58:24.179764 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0925 09:58:24.179888 (+ 124us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:24.179908 (+ 20us) write_op.cc:454] Released schema lock
Related trace 'op':
0925 09:58:24.116314 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:58:24.116725 (+ 411us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:58:24.116748 (+ 23us) write_op.cc:435] Acquired schema lock
0925 09:58:24.116763 (+ 15us) tablet.cc:660] Decoding operations
0925 09:58:24.117055 (+ 292us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:58:24.117108 (+ 53us) write_op.cc:641] Partition lock acquired for write op
0925 09:58:24.117135 (+ 27us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:58:24.117333 (+ 198us) tablet.cc:699] Row locks acquired
0925 09:58:24.117349 (+ 16us) write_op.cc:260] PREPARE: finished
0925 09:58:24.143238 (+ 25889us) op_driver.cc:444] REPLICATION: finished
0925 09:58:24.146745 (+ 3507us) write_op.cc:301] APPLY: starting
0925 09:58:24.146804 (+ 59us) tablet.cc:1365] starting BulkCheckPresence
0925 09:58:24.146901 (+ 97us) tablet.cc:1368] finished BulkCheckPresence
0925 09:58:24.146934 (+ 33us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:58:24.147263 (+ 329us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:58:24.147422 (+ 159us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 09:58:24.147535 (+ 113us) write_op.cc:312] APPLY: finished
0925 09:58:24.147718 (+ 183us) log.cc:842] Serialized 28 byte log entry
0925 09:58:24.148327 (+ 609us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:24.149751 (+ 1424us) write_op.cc:454] Released schema lock
0925 09:58:24.149781 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0925 09:58:24.149917 (+ 136us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:24.149953 (+ 36us) write_op.cc:454] Released schema lock
Related trace 'op':
0925 09:58:24.117565 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:58:24.117996 (+ 431us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:58:24.118023 (+ 27us) write_op.cc:435] Acquired schema lock
0925 09:58:24.118036 (+ 13us) tablet.cc:660] Decoding operations
0925 09:58:24.118325 (+ 289us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:58:24.118392 (+ 67us) write_op.cc:641] Partition lock acquired for write op
0925 09:58:24.118413 (+ 21us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:58:24.118598 (+ 185us) tablet.cc:699] Row locks acquired
0925 09:58:24.118614 (+ 16us) write_op.cc:260] PREPARE: finished
0925 09:58:24.143448 (+ 24834us) op_driver.cc:444] REPLICATION: finished
0925 09:58:24.150200 (+ 6752us) write_op.cc:301] APPLY: starting
0925 09:58:24.150246 (+ 46us) tablet.cc:1365] starting BulkCheckPresence
0925 09:58:24.150335 (+ 89us) tablet.cc:1368] finished BulkCheckPresence
0925 09:58:24.150354 (+ 19us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:58:24.150643 (+ 289us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:58:24.150760 (+ 117us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0925 09:58:24.150868 (+ 108us) write_op.cc:312] APPLY: finished
0925 09:58:24.151042 (+ 174us) log.cc:842] Serialized 28 byte log entry
0925 09:58:24.219698 (+ 68656us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:24.278478 (+ 58780us) write_op.cc:454] Released schema lock
0925 09:58:24.278512 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0925 09:58:24.278645 (+ 133us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:24.278671 (+ 26us) write_op.cc:454] Released schema lock
Related trace 'op':
0925 09:58:24.118818 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:58:24.119171 (+ 353us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:58:24.119192 (+ 21us) write_op.cc:435] Acquired schema lock
0925 09:58:24.119210 (+ 18us) tablet.cc:660] Decoding operations
0925 09:58:24.119481 (+ 271us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:58:24.119530 (+ 49us) write_op.cc:641] Partition lock acquired for write op
0925 09:58:24.119554 (+ 24us) tablet.cc:683] Acquiring locks for 1 operations
0925 09:58:24.148406 (+ 28852us) tablet.cc:699] Row locks acquired
0925 09:58:24.148423 (+ 17us) write_op.cc:260] PREPARE: finished
0925 09:58:24.171554 (+ 23131us) op_driver.cc:444] REPLICATION: finished
0925 09:58:24.180189 (+ 8635us) write_op.cc:301] APPLY: starting
0925 09:58:24.180232 (+ 43us) tablet.cc:1365] starting BulkCheckPresence
0925 09:58:24.180317 (+ 85us) tablet.cc:1368] finished BulkCheckPresence
0925 09:58:24.212430 (+ 32113us) tablet.cc:1370] starting ApplyRowOperation cycle
0925 09:58:24.212741 (+ 311us) tablet.cc:1381] finished ApplyRowOperation cycle
0925 09:58:24.212880 (+ 139us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0925 09:58:24.213016 (+ 136us) write_op.cc:312] APPLY: finished
0925 09:58:24.213194 (+ 178us) log.cc:842] Serialized 22 byte log entry
0925 09:58:24.213974 (+ 780us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:24.214038 (+ 64us) write_op.
I20260925 09:58:27.678267 13780 log.cc:1196] T 2e7cc97b8b594e92bf3bf74131f20456 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/2e7cc97b8b594e92bf3bf74131f20456
I20260925 09:58:27.678887 13780 ts_tablet_manager.cc:1939] T 2e7cc97b8b594e92bf3bf74131f20456 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
I20260925 09:58:27.680609 13780 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54548 (request call id 144) took 1104 ms. Trace:
I20260925 09:58:27.681348 14336 catalog_manager.cc:669] Loaded metadata for table default.table_840c9ea5fcc04f1b8811754058184452 [id=84e4b25904ae485fbf3816bd06643b42]
I20260925 09:58:27.683228 14336 catalog_manager.cc:669] Loaded metadata for table default.table_76fed9ed8d204d969883e54e5c7330c7 [id=929b4a72ba9b4357929f4354783971d6]
I20260925 09:58:27.684823 14336 catalog_manager.cc:669] Loaded metadata for table default.table_afc7cdc28ab84dff9a8782192aa34919 [id=a74c28d6faf4436385410132d67371eb]
I20260925 09:58:27.685140 13780 rpcz_store.cc:276] 0925 09:58:26.576378 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:26.578688 (+ 2310us) service_pool.cc:227] Handling call
0925 09:58:27.571375 (+992687us) ts_tablet_manager.cc:1145] Deleting tablet 2e7cc97b8b594e92bf3bf74131f20456
0925 09:58:27.571442 (+ 67us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0925 09:58:27.589002 (+ 17560us) tablet_metadata.cc:641] Metadata flushed
0925 09:58:27.677715 (+ 88713us) tablet_metadata.cc:641] Metadata flushed
0925 09:58:27.680560 (+ 2845us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":985341}
I20260925 09:58:27.686470 13068 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet 2e7cc97b8b594e92bf3bf74131f20456 (table default.table_5f3286f12ae947ef8c1dfa71d8dd21d2 [id=07c6108c574e407f827bc4ab4ffdf345]) successfully deleted
I20260925 09:58:27.687371 14336 catalog_manager.cc:669] Loaded metadata for table default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]
I20260925 09:58:27.689044 14336 catalog_manager.cc:669] Loaded metadata for table default.table_ca5cce29b409479f9191e1b06eb5bac0 [id=db33e4ea5b1d4f0abedf9b9dde8614e2]
I20260925 09:58:27.690716 14336 catalog_manager.cc:669] Loaded metadata for table default.table_1310f782dc864b068b28842f01c96b9c [id=dc9c88f862a64597aab300baf32f6f76]
I20260925 09:58:27.690848 14274 tablet_bootstrap.cc:492] T b9cf55829fed4c93a8eab92cee8920c9 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:27.692341 14336 catalog_manager.cc:669] Loaded metadata for table default.table_d4409ab5331140e9a4bb995d57be4ec4 [id=f196d9d3750b4857905c1332d3ac763e]
I20260925 09:58:27.691583 14274 ts_tablet_manager.cc:1397] T b9cf55829fed4c93a8eab92cee8920c9 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.098s user 0.014s sys 0.000s
I20260925 09:58:27.712548 14274 raft_consensus.cc:348] T b9cf55829fed4c93a8eab92cee8920c9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.713188 14274 raft_consensus.cc:374] T b9cf55829fed4c93a8eab92cee8920c9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:27.713403 14274 raft_consensus.cc:729] T b9cf55829fed4c93a8eab92cee8920c9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:27.715222 14336 tablet_loader.cc:96] loaded metadata for tablet 0090f685d0184822b41fd3e731874567 (table default.table_76fed9ed8d204d969883e54e5c7330c7 [id=929b4a72ba9b4357929f4354783971d6])
I20260925 09:58:27.716507 14336 tablet_loader.cc:96] loaded metadata for tablet 0e3f1d9381b4450cb6c15525699efbc9 (table default.table_afc7cdc28ab84dff9a8782192aa34919 [id=a74c28d6faf4436385410132d67371eb])
I20260925 09:58:27.718416 14336 tablet_loader.cc:96] loaded metadata for tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7])
I20260925 09:58:27.719619 14336 tablet_loader.cc:96] loaded metadata for tablet 3d18f12528cf40a49a7cd0c5c97001bf (table default.table_1238f9df428247dc9b9ca7c59823bfe4 [id=5eb4ef6bf2c344bdb26e429f484c59a4])
I20260925 09:58:27.719652 14274 consensus_queue.cc:260] T b9cf55829fed4c93a8eab92cee8920c9 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.720956 14336 tablet_loader.cc:96] loaded metadata for tablet 48ffc9cb12e346e7ae20c7f916ac33d4 (table default.table_8f71132b5a004ad89115f7a2278b95d6 [id=0cdc8d8fef5445668c0b0107243e0ae9])
I20260925 09:58:27.722218 14336 tablet_loader.cc:96] loaded metadata for tablet 4b61aaac33514c999482b824e0839a72 (table default.table_f83185097e7343848b564d3d5881c9ab [id=7f3f3dd4c3d74f21878caad38c9c25b9])
I20260925 09:58:27.722978 13770 tablet_bootstrap.cc:654] T fafc2b657ace4a4381452ce43fad39b6 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:27.723567 14336 tablet_loader.cc:96] loaded metadata for tablet 62d6d3750baa41c193c39ca39c4486a1 (table default.table_d4409ab5331140e9a4bb995d57be4ec4 [id=f196d9d3750b4857905c1332d3ac763e])
I20260925 09:58:27.724715 14336 tablet_loader.cc:96] loaded metadata for tablet 90d39744f6334e42bd32c26d13e6912c (table default.table_ca5cce29b409479f9191e1b06eb5bac0 [id=db33e4ea5b1d4f0abedf9b9dde8614e2])
I20260925 09:58:27.725343 14274 ts_tablet_manager.cc:1428] T b9cf55829fed4c93a8eab92cee8920c9 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.020s user 0.004s sys 0.000s
I20260925 09:58:27.726770 14336 tablet_loader.cc:96] loaded metadata for tablet ace8260d4ac94cafbb9147ebeb01ed85 (table default.table_41304460d5a345c88866a7429d816aca [id=77d343c035fa4fd2b3faadcfc4fec52f])
I20260925 09:58:27.728106 14336 tablet_loader.cc:96] loaded metadata for tablet b9cf55829fed4c93a8eab92cee8920c9 (table default.table_840c9ea5fcc04f1b8811754058184452 [id=84e4b25904ae485fbf3816bd06643b42])
I20260925 09:58:27.729362 14336 tablet_loader.cc:96] loaded metadata for tablet d3efa619592045b8be65e6b789b8291a (table default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477])
I20260925 09:58:27.730343 14336 tablet_loader.cc:96] loaded metadata for tablet e370ffe0dec24f40a3b446ed4cdc831b (table default.table_5f3286f12ae947ef8c1dfa71d8dd21d2 [id=07c6108c574e407f827bc4ab4ffdf345])
I20260925 09:58:27.731529 14336 tablet_loader.cc:96] loaded metadata for tablet e505eea45c414e82a69d7d965de6c814 (table default.table_1310f782dc864b068b28842f01c96b9c [id=dc9c88f862a64597aab300baf32f6f76])
I20260925 09:58:27.732702 14336 tablet_loader.cc:96] loaded metadata for tablet fafc2b657ace4a4381452ce43fad39b6 (table default.table_47c0ac37d06f4f45a701b0910a42c579 [id=714d733484104c508e7ced7cb14148bf])
I20260925 09:58:27.736236 14336 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:58:27.744894 13770 tablet_bootstrap.cc:492] T fafc2b657ace4a4381452ce43fad39b6 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:27.745301 13770 ts_tablet_manager.cc:1397] T fafc2b657ace4a4381452ce43fad39b6 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.071s user 0.011s sys 0.001s
I20260925 09:58:27.747455 13770 raft_consensus.cc:348] T fafc2b657ace4a4381452ce43fad39b6 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.748090 13770 raft_consensus.cc:374] T fafc2b657ace4a4381452ce43fad39b6 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:27.748343 13770 raft_consensus.cc:729] T fafc2b657ace4a4381452ce43fad39b6 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:27.748936 13770 consensus_queue.cc:260] T fafc2b657ace4a4381452ce43fad39b6 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.753175 14336 catalog_manager.cc:1259] Loaded cluster ID: 85565ad29e6546888a40444865b055de
I20260925 09:58:27.753464 14336 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:58:27.766264 13770 ts_tablet_manager.cc:1428] T fafc2b657ace4a4381452ce43fad39b6 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.021s user 0.001s sys 0.005s
W20260925 09:58:27.768190 13067 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 consensus state has no leader
I20260925 09:58:27.770785 13770 tablet_bootstrap.cc:492] T b9cf55829fed4c93a8eab92cee8920c9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:27.771337 14336 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:58:27.776367 14269 raft_consensus.cc:482] T fafc2b657ace4a4381452ce43fad39b6 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:27.776800 14269 raft_consensus.cc:504] T fafc2b657ace4a4381452ce43fad39b6 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.778600 14269 leader_election.cc:288] T fafc2b657ace4a4381452ce43fad39b6 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:27.780927 13670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fafc2b657ace4a4381452ce43fad39b6" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:27.781525 13670 raft_consensus.cc:2454] T fafc2b657ace4a4381452ce43fad39b6 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 0.
I20260925 09:58:27.785211 14336 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c: Loaded TSK: 0
I20260925 09:58:27.786361 14336 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:58:27.787737 13297 leader_election.cc:302] T fafc2b657ace4a4381452ce43fad39b6 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, b3082cf15dc74c0480e5d33ecf592843; no voters:
W20260925 09:58:27.788901 13298 leader_election.cc:341] T fafc2b657ace4a4381452ce43fad39b6 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: fafc2b657ace4a4381452ce43fad39b6
I20260925 09:58:27.789637 14269 raft_consensus.cc:2781] T fafc2b657ace4a4381452ce43fad39b6 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:27.789994 14269 raft_consensus.cc:482] T fafc2b657ace4a4381452ce43fad39b6 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:27.790330 14269 raft_consensus.cc:3037] T fafc2b657ace4a4381452ce43fad39b6 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:27.798992 14336 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260925 09:58:27.800179 13770 tablet_bootstrap.cc:654] T b9cf55829fed4c93a8eab92cee8920c9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:27.804791 14269 raft_consensus.cc:504] T fafc2b657ace4a4381452ce43fad39b6 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
W20260925 09:58:27.808207 13298 leader_election.cc:341] T fafc2b657ace4a4381452ce43fad39b6 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: fafc2b657ace4a4381452ce43fad39b6
I20260925 09:58:27.809965 13670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fafc2b657ace4a4381452ce43fad39b6" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
I20260925 09:58:27.810541 13670 raft_consensus.cc:3037] T fafc2b657ace4a4381452ce43fad39b6 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:27.811851 14336 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: 30
I20260925 09:58:27.816987 13670 raft_consensus.cc:2454] T fafc2b657ace4a4381452ce43fad39b6 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 1.
I20260925 09:58:27.818105 13297 leader_election.cc:302] T fafc2b657ace4a4381452ce43fad39b6 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, b3082cf15dc74c0480e5d33ecf592843; no voters: c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:58:27.818863 14295 raft_consensus.cc:2781] T fafc2b657ace4a4381452ce43fad39b6 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:27.819275 14295 raft_consensus.cc:686] T fafc2b657ace4a4381452ce43fad39b6 P b3082cf15dc74c0480e5d33ecf592843 [term 1 LEADER]: Becoming Leader. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Running, Role: LEADER
I20260925 09:58:27.820034 14295 consensus_queue.cc:237] T fafc2b657ace4a4381452ce43fad39b6 P b3082cf15dc74c0480e5d33ecf592843 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.827540 13770 tablet_bootstrap.cc:492] T b9cf55829fed4c93a8eab92cee8920c9 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:27.827960 13770 ts_tablet_manager.cc:1397] T b9cf55829fed4c93a8eab92cee8920c9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.061s user 0.009s sys 0.002s
I20260925 09:58:27.828552 14309 consensus_queue.cc:1035] T ace8260d4ac94cafbb9147ebeb01ed85 P a06c3ef7a6a94f7cbcc03e7cd232f237 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:27.829180 14302 raft_consensus.cc:482] T 4b61aaac33514c999482b824e0839a72 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:27.829586 14302 raft_consensus.cc:504] T 4b61aaac33514c999482b824e0839a72 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.832247 14302 leader_election.cc:288] T 4b61aaac33514c999482b824e0839a72 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:27.833406 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b61aaac33514c999482b824e0839a72" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:27.833941 13363 raft_consensus.cc:2454] T 4b61aaac33514c999482b824e0839a72 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 0.
I20260925 09:58:27.835253 13669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b61aaac33514c999482b824e0839a72" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:27.835909 13669 raft_consensus.cc:2454] T 4b61aaac33514c999482b824e0839a72 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 0.
I20260925 09:58:27.837182 13440 leader_election.cc:302] T 4b61aaac33514c999482b824e0839a72 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:27.838085 13770 raft_consensus.cc:348] T b9cf55829fed4c93a8eab92cee8920c9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.839191 14302 raft_consensus.cc:2781] T 4b61aaac33514c999482b824e0839a72 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:27.839269 13770 raft_consensus.cc:374] T b9cf55829fed4c93a8eab92cee8920c9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:27.839516 14302 raft_consensus.cc:482] T 4b61aaac33514c999482b824e0839a72 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:27.839798 14302 raft_consensus.cc:3037] T 4b61aaac33514c999482b824e0839a72 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:27.839704 13770 raft_consensus.cc:729] T b9cf55829fed4c93a8eab92cee8920c9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:27.846509 14038 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 5 FOLLOWER]: Refusing update from remote peer 70718e3f6f9f489089c0d8354380ed6c: Log matching property violated. Preceding OpId in replica: term: 4 index: 129. Preceding OpId from leader: term: 5 index: 197. (index mismatch)
I20260925 09:58:27.847990 14335 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 196, Last known committed idx: 127, Time since last communication: 0.000s
I20260925 09:58:27.842720 13770 consensus_queue.cc:260] T b9cf55829fed4c93a8eab92cee8920c9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.864230 13770 ts_tablet_manager.cc:1428] T b9cf55829fed4c93a8eab92cee8920c9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.036s user 0.005s sys 0.003s
I20260925 09:58:27.895620 14309 raft_consensus.cc:482] T 0090f685d0184822b41fd3e731874567 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:27.896195 14309 raft_consensus.cc:504] T 0090f685d0184822b41fd3e731874567 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.899564 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0090f685d0184822b41fd3e731874567" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:27.900130 13363 raft_consensus.cc:2454] T 0090f685d0184822b41fd3e731874567 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 0.
I20260925 09:58:27.901147 13603 leader_election.cc:302] T 0090f685d0184822b41fd3e731874567 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, b3082cf15dc74c0480e5d33ecf592843; no voters:
W20260925 09:58:27.903612 13606 leader_election.cc:341] T 0090f685d0184822b41fd3e731874567 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: 0090f685d0184822b41fd3e731874567
I20260925 09:58:27.904438 14309 leader_election.cc:288] T 0090f685d0184822b41fd3e731874567 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161)
I20260925 09:58:27.905211 14309 raft_consensus.cc:2781] T 0090f685d0184822b41fd3e731874567 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:27.905982 14309 raft_consensus.cc:482] T 0090f685d0184822b41fd3e731874567 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:27.910265 14309 raft_consensus.cc:3037] T 0090f685d0184822b41fd3e731874567 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:27.914458 14199 raft_consensus.cc:482] T b9cf55829fed4c93a8eab92cee8920c9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:27.914885 14199 raft_consensus.cc:504] T b9cf55829fed4c93a8eab92cee8920c9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.916761 14199 leader_election.cc:288] T b9cf55829fed4c93a8eab92cee8920c9 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:27.918573 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9cf55829fed4c93a8eab92cee8920c9" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:27.919119 13363 raft_consensus.cc:2454] T b9cf55829fed4c93a8eab92cee8920c9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 0.
I20260925 09:58:27.920125 13438 leader_election.cc:302] T b9cf55829fed4c93a8eab92cee8920c9 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:27.920827 14199 raft_consensus.cc:2781] T b9cf55829fed4c93a8eab92cee8920c9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:27.921147 14199 raft_consensus.cc:482] T b9cf55829fed4c93a8eab92cee8920c9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:27.921424 14199 raft_consensus.cc:3037] T b9cf55829fed4c93a8eab92cee8920c9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:27.924269 13669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9cf55829fed4c93a8eab92cee8920c9" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:27.924857 13669 raft_consensus.cc:2454] T b9cf55829fed4c93a8eab92cee8920c9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 0.
I20260925 09:58:27.931588 14199 raft_consensus.cc:504] T b9cf55829fed4c93a8eab92cee8920c9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:27.950692 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9cf55829fed4c93a8eab92cee8920c9" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843"
I20260925 09:58:27.951491 13363 raft_consensus.cc:3037] T b9cf55829fed4c93a8eab92cee8920c9 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:27.952263 13090 raft_consensus.cc:3032] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 4 LEADER]: Stepping down as leader of term 4
I20260925 09:58:27.952592 13090 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b7d0c1c2f1ed4e7a9ed54e73869d53b0, State: Running, Role: LEADER
I20260925 09:58:27.953269 13090 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 195, Committed index: 195, Last appended: 4.198, Last appended by leader: 198, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:27.954963 13090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 4 FOLLOWER]: Advancing to term 5
I20260925 09:58:27.958496 13363 raft_consensus.cc:2454] T b9cf55829fed4c93a8eab92cee8920c9 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:27.959772 13438 leader_election.cc:302] T b9cf55829fed4c93a8eab92cee8920c9 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:27.977442 14351 raft_consensus.cc:2781] T b9cf55829fed4c93a8eab92cee8920c9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:27.993422 13090 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 5 FOLLOWER]: Refusing update from remote peer 70718e3f6f9f489089c0d8354380ed6c: Log matching property violated. Preceding OpId in replica: term: 4 index: 198. Preceding OpId from leader: term: 5 index: 197. (term mismatch)
I20260925 09:58:28.001726 14348 consensus_queue.cc:1035] T ace8260d4ac94cafbb9147ebeb01ed85 P a06c3ef7a6a94f7cbcc03e7cd232f237 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:58:27.996438 13090 pending_rounds.cc:76] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Aborting all ops after (but not including) 196
I20260925 09:58:28.014513 13090 pending_rounds.cc:97] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Aborting uncommitted WRITE_OP operation due to leader change: 4.197
I20260925 09:58:27.937136 14302 raft_consensus.cc:504] T 4b61aaac33514c999482b824e0839a72 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:28.027220 14199 leader_election.cc:288] T b9cf55829fed4c93a8eab92cee8920c9 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Requested vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
W20260925 09:58:28.027925 13269 catalog_manager.cc:3810] Aborted: an error occurred while updating the sys-catalog: Op aborted by new leader
I20260925 09:58:28.027834 14269 leader_election.cc:288] T fafc2b657ace4a4381452ce43fad39b6 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Requested vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
W20260925 09:58:28.028553 13269 hms_notification_log_listener.cc:294] Failed to handle Hive Metastore notification: 33 ALTER_TABLE default.table_eda42a4c592f447b8084b26441b81b7c: Aborted: an error occurred while updating the sys-catalog: Op aborted by new leader
W20260925 09:58:28.028818 14345 log.cc:924] Time spent T fafc2b657ace4a4381452ce43fad39b6 P b3082cf15dc74c0480e5d33ecf592843: Append to log took a long time: real 0.057s user 0.000s sys 0.000s
I20260925 09:58:28.029627 13090 pending_rounds.cc:97] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Aborting uncommitted WRITE_OP operation due to leader change: 4.198
I20260925 09:58:28.035137 14302 leader_election.cc:288] T 4b61aaac33514c999482b824e0839a72 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Requested vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
W20260925 09:58:28.041395 13082 catalog_manager.cc:2176] Aborted: an error occurred while writing to the sys-catalog: Op aborted by new leader
W20260925 09:58:28.045512 13269 hms_notification_log_listener.cc:134] Hive Metastore notification log listener poll failed: Service unavailable: lost leadership while handling Hive Metastore notification log events: an error occurred while updating the sys-catalog: Op aborted by new leader
I20260925 09:58:28.046931 13075 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:50556 (request call id 107) took 2805 ms. Trace:
I20260925 09:58:28.059710 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b61aaac33514c999482b824e0839a72" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843"
I20260925 09:58:28.047240 13075 rpcz_store.cc:276] 0925 09:58:25.241792 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:25.241934 (+ 142us) service_pool.cc:227] Handling call
0925 09:58:28.046867 (+2804933us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1271168}
W20260925 09:58:28.072038 13728 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.10.9.61:40425)
I20260925 09:58:28.060408 13363 raft_consensus.cc:3037] T 4b61aaac33514c999482b824e0839a72 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:58:28.074084 13074 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_1b826b3631274aca84923893f7383ee3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:58:28.076865 13073 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:50556 (request call id 104) took 3309 ms. Trace:
I20260925 09:58:28.077181 13073 rpcz_store.cc:276] 0925 09:58:24.767598 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:25.042143 (+274545us) service_pool.cc:227] Handling call
0925 09:58:25.476493 (+434350us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:58:28.049529 (+2573036us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:58:28.076063 (+ 26534us) catalog_manager.cc:3616] Apply alter schema
0925 09:58:28.076805 (+ 742us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":715101,"Hive Metastore.run_cpu_time_us":1756,"Hive Metastore.run_wall_time_us":901168}
W20260925 09:58:28.077840 13603 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.1:42161, user_credentials={real_user=slave}} blocked reactor thread for 77131.8us
I20260925 09:58:28.079358 13670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9cf55829fed4c93a8eab92cee8920c9" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
I20260925 09:58:28.079970 13670 raft_consensus.cc:3037] T b9cf55829fed4c93a8eab92cee8920c9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
W20260925 09:58:28.080530 13079 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_489bd8fcb4594ceabc28fdbc0a951dd0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:58:28.084908 13669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b61aaac33514c999482b824e0839a72" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
I20260925 09:58:28.085453 13669 raft_consensus.cc:3037] T 4b61aaac33514c999482b824e0839a72 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:28.086508 13004 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49450:
name: "default.Table_aa3ad1f6adf841ae9a2efad3eed2f203"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:58:28.096342 13669 raft_consensus.cc:2454] T 4b61aaac33514c999482b824e0839a72 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:28.101550 14298 consensus_queue.cc:1035] T d3efa619592045b8be65e6b789b8291a P b3082cf15dc74c0480e5d33ecf592843 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:28.104605 14350 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 5 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } } }
I20260925 09:58:28.107735 14335 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 196, Last known committed idx: 195, Time since last communication: 0.001s
I20260925 09:58:28.108632 14335 consensus_queue.cc:1225] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [LEADER]: Peer b7d0c1c2f1ed4e7a9ed54e73869d53b0 log is divergent from this leader: its last log entry 4.196 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 195
I20260925 09:58:28.115643 13440 leader_election.cc:302] T 4b61aaac33514c999482b824e0839a72 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:28.111891 13091 pending_rounds.cc:76] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Aborting all ops after (but not including) 195
I20260925 09:58:28.118988 13091 pending_rounds.cc:97] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Aborting uncommitted WRITE_OP operation due to leader change: 4.196
W20260925 09:58:28.119946 13077 catalog_manager.cc:6530] Aborted: an error occurred while writing to the sys-catalog: Op aborted by new leader
I20260925 09:58:28.120532 13077 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:50556 (request call id 119) took 1601 ms. Trace:
I20260925 09:58:28.120811 13077 rpcz_store.cc:276] 0925 09:58:26.518745 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:26.592885 (+ 74140us) service_pool.cc:227] Handling call
0925 09:58:28.120475 (+1527590us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:58:26.704824 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:58:26.705171 (+ 347us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:58:26.705193 (+ 22us) write_op.cc:435] Acquired schema lock
0925 09:58:26.705207 (+ 14us) tablet.cc:660] Decoding operations
0925 09:58:26.705491 (+ 284us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:58:26.705530 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0925 09:58:26.705551 (+ 21us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:58:26.705737 (+ 186us) tablet.cc:699] Row locks acquired
0925 09:58:26.705749 (+ 12us) write_op.cc:260] PREPARE: finished
0925 09:58:26.705863 (+ 114us) write_op.cc:270] Start()
0925 09:58:26.705944 (+ 81us) write_op.cc:276] Timestamp: P: 1790330306705795 usec, L: 0
0925 09:58:26.705960 (+ 16us) op_driver.cc:331] REPLICATION: starting
0925 09:58:26.706449 (+ 489us) log.cc:842] Serialized 619 byte log entry
0925 09:58:28.119436 (+1412987us) op_driver.cc:444] REPLICATION: finished
0925 09:58:28.119487 (+ 51us) op_driver.cc:372] HandleFailure(Aborted: Op aborted by new leader)
0925 09:58:28.119512 (+ 25us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:28.119671 (+ 159us) write_op.cc:454] Released schema lock
0925 09:58:28.119713 (+ 42us) write_op.cc:335] FINISH: Op aborted
0925 09:58:28.119882 (+ 169us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:28.119915 (+ 33us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"num_ops":2,"prepare.queue_time_us":108261,"prepare.run_cpu_time_us":1884,"prepare.run_wall_time_us":1884,"raft.queue_time_us":1257,"raft.run_cpu_time_us":544,"raft.run_wall_time_us":543,"replication_time_us":1413288}]]}
I20260925 09:58:28.124739 14356 raft_consensus.cc:2781] T 4b61aaac33514c999482b824e0839a72 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:28.125264 13076 catalog_manager.cc:5587] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161) does not have the latest schema for tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7]). Expected version 1 got 0
I20260925 09:58:28.125283 14356 raft_consensus.cc:686] T 4b61aaac33514c999482b824e0839a72 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: Becoming Leader. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Running, Role: LEADER
I20260925 09:58:28.125849 14350 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:58:28.125921 13076 catalog_manager.cc:5495] T 90d39744f6334e42bd32c26d13e6912c P b3082cf15dc74c0480e5d33ecf592843 reported cstate change: term changed from 0 to 1, leader changed from <none> to b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b3082cf15dc74c0480e5d33ecf592843" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: UNKNOWN } } }
W20260925 09:58:28.131685 13728 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.10.9.62:43437)
E20260925 09:58:28.135782 13076 catalog_manager.cc:5650] Error updating tablets from b3082cf15dc74c0480e5d33ecf592843: Illegal state: replica b7d0c1c2f1ed4e7a9ed54e73869d53b0 is not leader of this config: current role FOLLOWER. Tablet report was: is_incremental: true updated_tablets { tablet_id: "90d39744f6334e42bd32c26d13e6912c" state: RUNNING consensus_state { current_term: 1 leader_uuid: "b3082cf15dc74c0480e5d33ecf592843" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: UNKNOWN } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY stats { } } updated_tablets { tablet_id: "3970efec7a0d4e7c9275c1f6b88ff4a9" state: RUNNING consensus_state { current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } } } schema_version: 0 tablet_data_state: TABLET_DATA_READY } sequence_number: 112
I20260925 09:58:28.142550 14309 raft_consensus.cc:504] T 0090f685d0184822b41fd3e731874567 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:28.145985 14343 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70718e3f6f9f489089c0d8354380ed6c. Latest consensus state: current_term: 5 leader_uuid: "70718e3f6f9f489089c0d8354380ed6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } } }
I20260925 09:58:28.146382 13003 master_service.cc:946] ReplaceTablet: received request to replace tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 from {username='slave'} at 127.0.0.1:49450
I20260925 09:58:28.146737 14343 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [sys.catalog]: This master's current role is: FOLLOWER
E20260925 09:58:28.147014 13272 catalog_manager.cc:816] Error processing pending assignments: Illegal state: error persisting updated tablet metadata: replica b7d0c1c2f1ed4e7a9ed54e73869d53b0 is not leader of this config: current role FOLLOWER
W20260925 09:58:28.147706 13076 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.10.9.1:53777 (request call id 113) took 1524 ms (client timeout 1000 ms). Trace:
I20260925 09:58:28.157079 14351 raft_consensus.cc:686] T b9cf55829fed4c93a8eab92cee8920c9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: Becoming Leader. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Running, Role: LEADER
I20260925 09:58:28.158214 14351 consensus_queue.cc:237] T b9cf55829fed4c93a8eab92cee8920c9 P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
W20260925 09:58:28.150347 13272 catalog_manager.cc:6293] Illegal state: an error occurred while writing to the sys-catalog: replica b7d0c1c2f1ed4e7a9ed54e73869d53b0 is not leader of this config: current role FOLLOWER
W20260925 09:58:28.160138 13606 leader_election.cc:341] T 0090f685d0184822b41fd3e731874567 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: 0090f685d0184822b41fd3e731874567
E20260925 09:58:28.161661 13272 catalog_manager.cc:836] Error processing tables/tablets deletions: Illegal state: an error occurred while writing to the sys-catalog: replica b7d0c1c2f1ed4e7a9ed54e73869d53b0 is not leader of this config: current role FOLLOWER
W20260925 09:58:28.151854 13076 rpcz_store.cc:269] 0925 09:58:26.622913 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:26.731440 (+108527us) service_pool.cc:227] Handling call
0925 09:58:28.135571 (+1404131us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:28.135592 (+ 21us) write_op.cc:454] Released schema lock
0925 09:58:28.147643 (+ 12051us) inbound_call.cc:173] Queueing failure response
Related trace 'op':
0925 09:58:28.135346 (+ 0us) op_driver.cc:372] HandleFailure(Illegal state: replica b7d0c1c2f1ed4e7a9ed54e73869d53b0 is not leader of this config: current role FOLLOWER)
0925 09:58:28.135414 (+ 68us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:28.135433 (+ 19us) write_op.cc:454] Released schema lock
0925 09:58:28.135452 (+ 19us) write_op.cc:335] FINISH: Op aborted
Metrics: {"reported_tablets":2,"child_traces":[["op",{}]]}
I20260925 09:58:28.161993 13362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0090f685d0184822b41fd3e731874567" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843"
I20260925 09:58:28.162590 13362 raft_consensus.cc:3037] T 0090f685d0184822b41fd3e731874567 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:28.169487 13362 raft_consensus.cc:2454] T 0090f685d0184822b41fd3e731874567 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 1.
I20260925 09:58:28.170786 13603 leader_election.cc:302] T 0090f685d0184822b41fd3e731874567 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, b3082cf15dc74c0480e5d33ecf592843; no voters: c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:58:28.171741 14357 raft_consensus.cc:2781] T 0090f685d0184822b41fd3e731874567 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:28.172233 14357 raft_consensus.cc:686] T 0090f685d0184822b41fd3e731874567 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 LEADER]: Becoming Leader. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Running, Role: LEADER
I20260925 09:58:28.173120 14357 consensus_queue.cc:237] T 0090f685d0184822b41fd3e731874567 P a06c3ef7a6a94f7cbcc03e7cd232f237 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:28.174571 14364 mvcc.cc:205] Tried to move back new op lower bound from 7333192940926304256 to 7333192939776667648. Current Snapshot: MvccSnapshot[applied={T|T < 7333192940926304256}]
I20260925 09:58:28.180568 13504 raft_consensus.cc:3037] T d3efa619592045b8be65e6b789b8291a P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:28.157104 14353 consensus_queue.cc:1035] T 3d18f12528cf40a49a7cd0c5c97001bf P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.033s
I20260925 09:58:28.193981 13006 catalog_manager.cc:5495] T ace8260d4ac94cafbb9147ebeb01ed85 P c4d724d4b83e4d1e817a3a505dc24eb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3). New cstate: current_term: 1 leader_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } }
I20260925 09:58:28.194857 14269 consensus_queue.cc:1035] T d3efa619592045b8be65e6b789b8291a P b3082cf15dc74c0480e5d33ecf592843 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:28.199046 13005 catalog_manager.cc:5495] T fafc2b657ace4a4381452ce43fad39b6 P b3082cf15dc74c0480e5d33ecf592843 reported cstate change: term changed from 0 to 1, leader changed from <none> to b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b3082cf15dc74c0480e5d33ecf592843" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: UNKNOWN } } }
I20260925 09:58:28.201510 13670 raft_consensus.cc:2454] T b9cf55829fed4c93a8eab92cee8920c9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:28.207523 14309 leader_election.cc:288] T 0090f685d0184822b41fd3e731874567 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 election: Requested vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161)
I20260925 09:58:28.213372 14355 consensus_queue.cc:1035] T 3d18f12528cf40a49a7cd0c5c97001bf P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:28.126121 14356 consensus_queue.cc:237] T 4b61aaac33514c999482b824e0839a72 P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:28.219744 14343 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "70718e3f6f9f489089c0d8354380ed6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } } }
I20260925 09:58:28.220487 14343 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:58:28.241080 13363 raft_consensus.cc:2454] T 4b61aaac33514c999482b824e0839a72 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:28.259851 14335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 5 leader_uuid: "70718e3f6f9f489089c0d8354380ed6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } } }
I20260925 09:58:28.260675 14335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [sys.catalog]: This master's current role is: LEADER
I20260925 09:58:28.275987 14356 consensus_queue.cc:1035] T 0e3f1d9381b4450cb6c15525699efbc9 P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:28.282747 14269 consensus_queue.cc:1035] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P b3082cf15dc74c0480e5d33ecf592843 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:28.285641 13718 heartbeater.cc:503] Master 127.10.9.62:43437 was elected leader, sending a full tablet report...
I20260925 09:58:28.315749 13003 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c: Sending DeleteTablet for 3 replicas of tablet 3970efec7a0d4e7c9275c1f6b88ff4a9
I20260925 09:58:28.317579 13003 catalog_manager.cc:6565] ReplaceTablet: tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 deleted and replaced by tablet 11f8c7b5918a431aa9c4975f28ea89cb
I20260925 09:58:28.326972 13411 heartbeater.cc:503] Master 127.10.9.62:43437 was elected leader, sending a full tablet report...
I20260925 09:58:28.344341 13483 tablet_service.cc:1514] Processing DeleteTablet for tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 with delete_type TABLET_DATA_DELETED (replaced by tablet 11f8c7b5918a431aa9c4975f28ea89cb) from {username='slave'} at 127.0.0.1:39032
I20260925 09:58:28.347304 13482 tablet_service.cc:1467] Processing CreateTablet for tablet e370ffe0dec24f40a3b446ed4cdc831b (DEFAULT_TABLE table=default.table_5f3286f12ae947ef8c1dfa71d8dd21d2 [id=07c6108c574e407f827bc4ab4ffdf345]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:28.348417 13482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e370ffe0dec24f40a3b446ed4cdc831b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:28.361704 14269 consensus_queue.cc:1035] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P b3082cf15dc74c0480e5d33ecf592843 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:28.370090 13082 catalog_manager.cc:2158] deleted orphaned HMS catalog entry for table [id=5a1feab31f8d4a9d9452607552d02054]
I20260925 09:58:28.371013 13082 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:50556 (request call id 105) took 3593 ms. Trace:
I20260925 09:58:28.371335 13082 rpcz_store.cc:276] 0925 09:58:24.777975 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:25.086203 (+308228us) service_pool.cc:227] Handling call
0925 09:58:25.478665 (+392462us) catalog_manager.cc:2075] Acquired catalog manager lock
0925 09:58:25.480897 (+ 2232us) catalog_manager.cc:2121] Created new table and tablet info
0925 09:58:27.208132 (+1727235us) catalog_manager.cc:2147] Created new table in HMS catalog
0925 09:58:28.041713 (+833581us) catalog_manager.cc:2155] Rolling back HMS table creation
0925 09:58:28.370934 (+329221us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:58:27.210263 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0925 09:58:27.210650 (+ 387us) write_op.cc:432] Acquiring schema lock in shared mode
0925 09:58:27.210671 (+ 21us) write_op.cc:435] Acquired schema lock
0925 09:58:27.210683 (+ 12us) tablet.cc:660] Decoding operations
0925 09:58:27.210945 (+ 262us) write_op.cc:620] Acquiring the partition lock for write op
0925 09:58:27.210978 (+ 33us) write_op.cc:641] Partition lock acquired for write op
0925 09:58:27.210999 (+ 21us) tablet.cc:683] Acquiring locks for 2 operations
0925 09:58:27.211174 (+ 175us) tablet.cc:699] Row locks acquired
0925 09:58:27.211187 (+ 13us) write_op.cc:260] PREPARE: finished
0925 09:58:27.211286 (+ 99us) write_op.cc:270] Start()
0925 09:58:27.211364 (+ 78us) write_op.cc:276] Timestamp: P: 1790330307211230 usec, L: 0
0925 09:58:27.211380 (+ 16us) op_driver.cc:331] REPLICATION: starting
0925 09:58:27.211881 (+ 501us) log.cc:842] Serialized 433 byte log entry
0925 09:58:28.034294 (+822413us) op_driver.cc:444] REPLICATION: finished
0925 09:58:28.039541 (+ 5247us) op_driver.cc:372] HandleFailure(Aborted: Op aborted by new leader)
0925 09:58:28.039599 (+ 58us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:28.039723 (+ 124us) write_op.cc:454] Released schema lock
0925 09:58:28.039761 (+ 38us) write_op.cc:335] FINISH: Op aborted
0925 09:58:28.042698 (+ 2937us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:28.042747 (+ 49us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":1617208,"Hive Metastore.run_cpu_time_us":943,"Hive Metastore.run_wall_time_us":437279,"child_traces":[["op",{"num_ops":2,"prepare.queue_time_us":284,"prepare.run_cpu_time_us":1815,"prepare.run_wall_time_us":1815,"replication_time_us":821625}]]}
I20260925 09:58:28.386639 13338 tablet_service.cc:1514] Processing DeleteTablet for tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 with delete_type TABLET_DATA_DELETED (replaced by tablet 11f8c7b5918a431aa9c4975f28ea89cb) from {username='slave'} at 127.0.0.1:58732
I20260925 09:58:28.386866 13340 tablet_service.cc:1467] Processing CreateTablet for tablet e370ffe0dec24f40a3b446ed4cdc831b (DEFAULT_TABLE table=default.table_5f3286f12ae947ef8c1dfa71d8dd21d2 [id=07c6108c574e407f827bc4ab4ffdf345]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:28.388389 13340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e370ffe0dec24f40a3b446ed4cdc831b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:28.401610 14302 consensus_queue.cc:1035] T 0e3f1d9381b4450cb6c15525699efbc9 P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:28.408035 13552 heartbeater.cc:503] Master 127.10.9.62:43437 was elected leader, sending a full tablet report...
I20260925 09:58:28.410290 13649 tablet_service.cc:1514] Processing DeleteTablet for tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 with delete_type TABLET_DATA_DELETED (replaced by tablet 11f8c7b5918a431aa9c4975f28ea89cb) from {username='slave'} at 127.0.0.1:60700
I20260925 09:58:28.410594 13646 tablet_service.cc:1467] Processing CreateTablet for tablet e370ffe0dec24f40a3b446ed4cdc831b (DEFAULT_TABLE table=default.table_5f3286f12ae947ef8c1dfa71d8dd21d2 [id=07c6108c574e407f827bc4ab4ffdf345]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:28.411715 13646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e370ffe0dec24f40a3b446ed4cdc831b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:28.414278 14390 tablet_replica.cc:330] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P a06c3ef7a6a94f7cbcc03e7cd232f237: stopping tablet replica
I20260925 09:58:28.415181 14390 raft_consensus.cc:2229] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:28.415455 14385 tablet_replica.cc:330] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P c4d724d4b83e4d1e817a3a505dc24eb9: stopping tablet replica
I20260925 09:58:28.415680 14390 raft_consensus.cc:2258] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 09:58:28.421996 13223 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 11f8c7b5918a431aa9c4975f28ea89cb has no consensus state
I20260925 09:58:28.430898 14385 raft_consensus.cc:2229] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 09:58:28.432142 14385 raft_consensus.cc:2258] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:28.434214 14390 ts_tablet_manager.cc:1905] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 09:58:28.442296 13005 catalog_manager.cc:5328] Ignoring report from unknown tablet af6ace3011c84603b32d0a409d5b8c1c
W20260925 09:58:28.450882 13003 catalog_manager.cc:5328] Ignoring report from unknown tablet b103b022a84040ea97247d6503002f47
I20260925 09:58:28.455865 14037 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 5 FOLLOWER]: Deduplicated request from leader. Original: 4.129->[4.130-4.198] Dedup: 4.195->[4.196-4.198]
I20260925 09:58:28.456429 14037 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 5 FOLLOWER]: Rejecting Update request from peer b7d0c1c2f1ed4e7a9ed54e73869d53b0 for earlier term 4. Current term is 5. Ops: [4.130-4.198]
I20260925 09:58:28.460260 14349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70718e3f6f9f489089c0d8354380ed6c. Latest consensus state: current_term: 5 leader_uuid: "70718e3f6f9f489089c0d8354380ed6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } } }
I20260925 09:58:28.461061 14349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:58:28.479924 14349 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 70718e3f6f9f489089c0d8354380ed6c)
I20260925 09:58:28.480392 14349 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:28.482939 13017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "61e29afab0cf46819cbffc91248bf761" candidate_term: 6 candidate_status { last_received { term: 5 index: 197 } } ignore_live_leader: false dest_uuid: "70718e3f6f9f489089c0d8354380ed6c" is_pre_election: true
I20260925 09:58:28.483179 13091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "61e29afab0cf46819cbffc91248bf761" candidate_term: 6 candidate_status { last_received { term: 5 index: 197 } } ignore_live_leader: false dest_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" is_pre_election: true
I20260925 09:58:28.483940 14349 leader_election.cc:288] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 70718e3f6f9f489089c0d8354380ed6c (127.10.9.62:43437), b7d0c1c2f1ed4e7a9ed54e73869d53b0 (127.10.9.61:40425)
I20260925 09:58:28.484725 13957 leader_election.cc:302] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61e29afab0cf46819cbffc91248bf761; no voters: 70718e3f6f9f489089c0d8354380ed6c, b7d0c1c2f1ed4e7a9ed54e73869d53b0
I20260925 09:58:28.487839 14385 ts_tablet_manager.cc:1905] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 09:58:28.488718 12993 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 11f8c7b5918a431aa9c4975f28ea89cb (table default.table_eda42a4c592f447b8084b26441b81b7c [id=7e0502ffa5544a5bb252a72020649cd7], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 11f8c7b5918a431aa9c4975f28ea89cb has no consensus state
I20260925 09:58:28.497174 14349 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260925 09:58:28.502698 14390 ts_tablet_manager.cc:1918] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P a06c3ef7a6a94f7cbcc03e7cd232f237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:28.507603 13007 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49450:
name: "default.Table_688d7c9730444646b6d7a901d5537807"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:58:28.512398 14393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 5 leader_uuid: "70718e3f6f9f489089c0d8354380ed6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } } }
I20260925 09:58:28.516722 14349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "70718e3f6f9f489089c0d8354380ed6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } } }
I20260925 09:58:28.517578 14349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:58:28.519649 14390 log.cc:1196] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-2/wal/wals/3970efec7a0d4e7c9275c1f6b88ff4a9
I20260925 09:58:28.520565 14390 ts_tablet_manager.cc:1939] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P a06c3ef7a6a94f7cbcc03e7cd232f237: Deleting consensus metadata
W20260925 09:58:28.531275 13004 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_aa3ad1f6adf841ae9a2efad3eed2f203 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:58:28.537866 13223 catalog_manager.cc:2709] Deleting table default.table_688d7c9730444646b6d7a901d5537807 [id=5a1feab31f8d4a9d9452607552d02054] in response to Hive Metastore notification log event 35
I20260925 09:58:28.537923 14391 tablet_bootstrap.cc:492] T e370ffe0dec24f40a3b446ed4cdc831b P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
W20260925 09:58:28.538774 13223 hms_notification_log_listener.cc:294] Failed to handle Hive Metastore notification: 35 DROP_TABLE default.table_688d7c9730444646b6d7a901d5537807: Not found: the table does not exist: table_id: "5a1feab31f8d4a9d9452607552d02054" table_name: "default.table_688d7c9730444646b6d7a901d5537807"
I20260925 09:58:28.547797 14393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [sys.catalog]: This master's current role is: LEADER
I20260925 09:58:28.561573 14387 tablet_replica.cc:330] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P b3082cf15dc74c0480e5d33ecf592843: stopping tablet replica
I20260925 09:58:28.581002 13338 tablet_service.cc:1467] Processing CreateTablet for tablet 11f8c7b5918a431aa9c4975f28ea89cb (DEFAULT_TABLE table=default.table_eda42a4c592f447b8084b26441b81b7c [id=7e0502ffa5544a5bb252a72020649cd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:28.582207 13338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11f8c7b5918a431aa9c4975f28ea89cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:28.585501 13482 tablet_service.cc:1467] Processing CreateTablet for tablet 11f8c7b5918a431aa9c4975f28ea89cb (DEFAULT_TABLE table=default.table_eda42a4c592f447b8084b26441b81b7c [id=7e0502ffa5544a5bb252a72020649cd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:28.586723 13482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11f8c7b5918a431aa9c4975f28ea89cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:28.592490 14387 raft_consensus.cc:2229] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P b3082cf15dc74c0480e5d33ecf592843 [term 1 LEADER]: Raft consensus shutting down.
I20260925 09:58:28.593489 14387 raft_consensus.cc:2258] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 09:58:28.593109 13005 catalog_manager.cc:5495] T d3efa619592045b8be65e6b789b8291a P b3082cf15dc74c0480e5d33ecf592843 reported cstate change: term changed from 0 to 1, leader changed from <none> to b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b3082cf15dc74c0480e5d33ecf592843" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: HEALTHY } } }
I20260925 09:58:28.594502 13005 catalog_manager.cc:5587] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161) does not have the latest schema for tablet d3efa619592045b8be65e6b789b8291a (table default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]). Expected version 1 got 0
I20260925 09:58:28.595531 13005 catalog_manager.cc:5495] T 48ffc9cb12e346e7ae20c7f916ac33d4 P b3082cf15dc74c0480e5d33ecf592843 reported cstate change: term changed from 0 to 1, leader changed from <none> to b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b3082cf15dc74c0480e5d33ecf592843" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: HEALTHY } } }
I20260925 09:58:28.597582 13005 catalog_manager.cc:5495] T 90d39744f6334e42bd32c26d13e6912c P b3082cf15dc74c0480e5d33ecf592843 reported cstate change: term changed from 0 to 1, leader changed from <none> to b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b3082cf15dc74c0480e5d33ecf592843" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: HEALTHY } } }
I20260925 09:58:28.599139 13005 catalog_manager.cc:5495] T 3d18f12528cf40a49a7cd0c5c97001bf P b3082cf15dc74c0480e5d33ecf592843 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2). New cstate: current_term: 1 leader_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } }
W20260925 09:58:28.618871 13007 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_688d7c9730444646b6d7a901d5537807 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 09:58:28.611141 14385 ts_tablet_manager.cc:1918] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:28.629992 14385 log.cc:1196] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-1/wal/wals/3970efec7a0d4e7c9275c1f6b88ff4a9
I20260925 09:58:28.631677 14385 ts_tablet_manager.cc:1939] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P c4d724d4b83e4d1e817a3a505dc24eb9: Deleting consensus metadata
I20260925 09:58:28.635211 12994 catalog_manager.cc:4841] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 (table default.table_eda42a4c592f447b8084b26441b81b7c [id=7e0502ffa5544a5bb252a72020649cd7]) successfully deleted
W20260925 09:58:28.636037 12993 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.61:40425, user_credentials={real_user=slave}} blocked reactor thread for 91192.5us
I20260925 09:58:28.642848 13649 tablet_service.cc:1467] Processing CreateTablet for tablet 11f8c7b5918a431aa9c4975f28ea89cb (DEFAULT_TABLE table=default.table_eda42a4c592f447b8084b26441b81b7c [id=7e0502ffa5544a5bb252a72020649cd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:28.597301 14387 ts_tablet_manager.cc:1905] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P b3082cf15dc74c0480e5d33ecf592843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 09:58:28.643888 13649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11f8c7b5918a431aa9c4975f28ea89cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:28.656723 12993 catalog_manager.cc:4841] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 (table default.table_eda42a4c592f447b8084b26441b81b7c [id=7e0502ffa5544a5bb252a72020649cd7]) successfully deleted
I20260925 09:58:28.659067 14391 tablet_bootstrap.cc:654] T e370ffe0dec24f40a3b446ed4cdc831b P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:28.664098 13006 catalog_manager.cc:5587] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327) does not have the latest schema for tablet d3efa619592045b8be65e6b789b8291a (table default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]). Expected version 1 got 0
I20260925 09:58:28.665050 13006 catalog_manager.cc:5495] T 4b61aaac33514c999482b824e0839a72 P c4d724d4b83e4d1e817a3a505dc24eb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2). New cstate: current_term: 1 leader_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: UNKNOWN } } }
I20260925 09:58:28.666733 13006 catalog_manager.cc:5495] T 0e3f1d9381b4450cb6c15525699efbc9 P c4d724d4b83e4d1e817a3a505dc24eb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2). New cstate: current_term: 1 leader_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: UNKNOWN } } }
I20260925 09:58:28.668306 13006 catalog_manager.cc:5495] T b9cf55829fed4c93a8eab92cee8920c9 P c4d724d4b83e4d1e817a3a505dc24eb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2). New cstate: current_term: 1 leader_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: UNKNOWN } } }
I20260925 09:58:28.685343 14396 tablet_bootstrap.cc:492] T e370ffe0dec24f40a3b446ed4cdc831b P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:28.692111 14391 tablet_bootstrap.cc:492] T e370ffe0dec24f40a3b446ed4cdc831b P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:28.697062 14391 ts_tablet_manager.cc:1397] T e370ffe0dec24f40a3b446ed4cdc831b P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.159s user 0.007s sys 0.005s
I20260925 09:58:28.723469 14395 tablet_bootstrap.cc:492] T e370ffe0dec24f40a3b446ed4cdc831b P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:28.754755 14361 mvcc.cc:205] Tried to move back new op lower bound from 7333192940926304256 to 7333192939776667648. Current Snapshot: MvccSnapshot[applied={T|T < 7333192940926304256 or (T in {7333192940926304256})}]
I20260925 09:58:28.755844 13078 catalog_manager.cc:2685] deleted HMS catalog entry for table default.table_41304460d5a345c88866a7429d816aca [id=77d343c035fa4fd2b3faadcfc4fec52f]
I20260925 09:58:28.757706 13078 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:50556 (request call id 110) took 3079 ms. Trace:
I20260925 09:58:28.758054 13078 rpcz_store.cc:276] 0925 09:58:25.678037 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:25.690105 (+ 12068us) service_pool.cc:227] Handling call
0925 09:58:28.046579 (+2356474us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:58:28.757645 (+711066us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":322805,"Hive Metastore.run_cpu_time_us":346,"Hive Metastore.run_wall_time_us":385319}
I20260925 09:58:28.766280 13004 catalog_manager.cc:2148] created HMS catalog entry for table [id=7c9c081163934b5197aaeedda0de597c]
I20260925 09:58:28.820979 14395 tablet_bootstrap.cc:654] T e370ffe0dec24f40a3b446ed4cdc831b P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
W20260925 09:58:28.821729 13067 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 consensus state has no leader
I20260925 09:58:28.847178 13003 catalog_manager.cc:5587] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731) does not have the latest schema for tablet d3efa619592045b8be65e6b789b8291a (table default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]). Expected version 1 got 0
I20260925 09:58:28.848568 13003 catalog_manager.cc:5495] T 0090f685d0184822b41fd3e731874567 P a06c3ef7a6a94f7cbcc03e7cd232f237 reported cstate change: term changed from 0 to 1, leader changed from <none> to a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3). New cstate: current_term: 1 leader_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: HEALTHY } } }
I20260925 09:58:28.893280 13005 catalog_manager.cc:5587] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161) does not have the latest schema for tablet d3efa619592045b8be65e6b789b8291a (table default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]). Expected version 1 got 0
I20260925 09:58:28.893630 14399 tablet.cc:1722] T d3efa619592045b8be65e6b789b8291a P b3082cf15dc74c0480e5d33ecf592843: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:58:28.898501 14400 tablet.cc:1722] T d3efa619592045b8be65e6b789b8291a P a06c3ef7a6a94f7cbcc03e7cd232f237: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:58:28.902568 14401 tablet.cc:1722] T d3efa619592045b8be65e6b789b8291a P c4d724d4b83e4d1e817a3a505dc24eb9: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260925 09:58:28.917140 14391 raft_consensus.cc:348] T e370ffe0dec24f40a3b446ed4cdc831b P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:28.919807 14391 raft_consensus.cc:374] T e370ffe0dec24f40a3b446ed4cdc831b P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:28.920120 14391 raft_consensus.cc:729] T e370ffe0dec24f40a3b446ed4cdc831b P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:28.920719 14391 consensus_queue.cc:260] T e370ffe0dec24f40a3b446ed4cdc831b P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:28.922858 14391 ts_tablet_manager.cc:1428] T e370ffe0dec24f40a3b446ed4cdc831b P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.225s user 0.006s sys 0.000s
I20260925 09:58:28.924486 14387 ts_tablet_manager.cc:1918] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P b3082cf15dc74c0480e5d33ecf592843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260925 09:58:28.925781 14396 tablet_bootstrap.cc:654] T e370ffe0dec24f40a3b446ed4cdc831b P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:28.926167 14387 log.cc:1196] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P b3082cf15dc74c0480e5d33ecf592843: Deleting WAL directory at /tmp/dist-test-taskdR3P3Z/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790330210960882-10276-0/minicluster-data/ts-0/wal/wals/3970efec7a0d4e7c9275c1f6b88ff4a9
I20260925 09:58:28.929050 14391 tablet_bootstrap.cc:492] T 11f8c7b5918a431aa9c4975f28ea89cb P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:28.925267 14395 tablet_bootstrap.cc:492] T e370ffe0dec24f40a3b446ed4cdc831b P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:28.931861 14395 ts_tablet_manager.cc:1397] T e370ffe0dec24f40a3b446ed4cdc831b P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.209s user 0.005s sys 0.007s
I20260925 09:58:28.932098 14387 ts_tablet_manager.cc:1939] T 3970efec7a0d4e7c9275c1f6b88ff4a9 P b3082cf15dc74c0480e5d33ecf592843: Deleting consensus metadata
I20260925 09:58:28.934666 14395 raft_consensus.cc:348] T e370ffe0dec24f40a3b446ed4cdc831b P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:28.936223 12991 catalog_manager.cc:4841] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 (table default.table_eda42a4c592f447b8084b26441b81b7c [id=7e0502ffa5544a5bb252a72020649cd7]) successfully deleted
I20260925 09:58:28.955569 14395 raft_consensus.cc:374] T e370ffe0dec24f40a3b446ed4cdc831b P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:28.962972 13649 tablet_service.cc:1514] Processing DeleteTablet for tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 with delete_type TABLET_DATA_DELETED (replaced by tablet 11f8c7b5918a431aa9c4975f28ea89cb) from {username='slave'} at 127.0.0.1:60700
I20260925 09:58:28.964689 14395 raft_consensus.cc:729] T e370ffe0dec24f40a3b446ed4cdc831b P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
W20260925 09:58:28.964892 12993 catalog_manager.cc:4808] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): delete failed for tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 because the tablet was not found. No further retry: Not found: Tablet not found: 3970efec7a0d4e7c9275c1f6b88ff4a9
I20260925 09:58:28.966914 13482 tablet_service.cc:1514] Processing DeleteTablet for tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 with delete_type TABLET_DATA_DELETED (replaced by tablet 11f8c7b5918a431aa9c4975f28ea89cb) from {username='slave'} at 127.0.0.1:39032
W20260925 09:58:28.968554 12994 catalog_manager.cc:4808] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): delete failed for tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 because the tablet was not found. No further retry: Not found: Tablet not found: 3970efec7a0d4e7c9275c1f6b88ff4a9
I20260925 09:58:28.969460 14395 consensus_queue.cc:260] T e370ffe0dec24f40a3b446ed4cdc831b P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:28.973459 14395 ts_tablet_manager.cc:1428] T e370ffe0dec24f40a3b446ed4cdc831b P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.041s user 0.003s sys 0.003s
I20260925 09:58:28.982530 14395 tablet_bootstrap.cc:492] T 11f8c7b5918a431aa9c4975f28ea89cb P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:28.998507 13007 catalog_manager.cc:2148] created HMS catalog entry for table [id=c06b5e37a4f14eb5b653ada7e7f73b4f]
I20260925 09:58:29.001257 14391 tablet_bootstrap.cc:654] T 11f8c7b5918a431aa9c4975f28ea89cb P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:29.021973 13482 tablet_service.cc:1467] Processing CreateTablet for tablet 733492d93592485e920628c47d5c4251 (DEFAULT_TABLE table=default.table_aa3ad1f6adf841ae9a2efad3eed2f203 [id=7c9c081163934b5197aaeedda0de597c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:29.023548 13482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 733492d93592485e920628c47d5c4251. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 09:58:29.048703 13298 consensus_peers.cc:487] T fafc2b657ace4a4381452ce43fad39b6 P b3082cf15dc74c0480e5d33ecf592843 -> Peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Couldn't send request to peer c4d724d4b83e4d1e817a3a505dc24eb9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fafc2b657ace4a4381452ce43fad39b6. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:58:29.053005 14295 consensus_queue.cc:1035] T fafc2b657ace4a4381452ce43fad39b6 P b3082cf15dc74c0480e5d33ecf592843 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:58:29.054823 14391 tablet_bootstrap.cc:492] T 11f8c7b5918a431aa9c4975f28ea89cb P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:29.055413 14391 ts_tablet_manager.cc:1397] T 11f8c7b5918a431aa9c4975f28ea89cb P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.127s user 0.014s sys 0.002s
I20260925 09:58:29.068938 14391 raft_consensus.cc:348] T 11f8c7b5918a431aa9c4975f28ea89cb P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:29.074602 14391 raft_consensus.cc:374] T 11f8c7b5918a431aa9c4975f28ea89cb P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:29.077268 14391 raft_consensus.cc:729] T 11f8c7b5918a431aa9c4975f28ea89cb P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:29.083830 13340 tablet_service.cc:1467] Processing CreateTablet for tablet 733492d93592485e920628c47d5c4251 (DEFAULT_TABLE table=default.table_aa3ad1f6adf841ae9a2efad3eed2f203 [id=7c9c081163934b5197aaeedda0de597c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:29.085326 13340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 733492d93592485e920628c47d5c4251. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:29.085762 14391 consensus_queue.cc:260] T 11f8c7b5918a431aa9c4975f28ea89cb P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:29.097026 13649 tablet_service.cc:1467] Processing CreateTablet for tablet 733492d93592485e920628c47d5c4251 (DEFAULT_TABLE table=default.table_aa3ad1f6adf841ae9a2efad3eed2f203 [id=7c9c081163934b5197aaeedda0de597c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:29.097034 14351 consensus_queue.cc:1035] T 4b61aaac33514c999482b824e0839a72 P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:29.098223 13649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 733492d93592485e920628c47d5c4251. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:29.127763 14351 consensus_queue.cc:1035] T b9cf55829fed4c93a8eab92cee8920c9 P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:29.133399 10276 master-stress-test.cc:580] Killed test thread 1/10
I20260925 09:58:29.133936 10276 master-stress-test.cc:578] Killing test thread 2/10
I20260925 09:58:29.124221 14391 ts_tablet_manager.cc:1428] T 11f8c7b5918a431aa9c4975f28ea89cb P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.068s user 0.007s sys 0.002s
I20260925 09:58:29.177035 14391 tablet_bootstrap.cc:492] T 733492d93592485e920628c47d5c4251 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:29.177989 13080 catalog_manager.cc:3411] renamed table default.table_5f3286f12ae947ef8c1dfa71d8dd21d2 [id=07c6108c574e407f827bc4ab4ffdf345] in HMS: new name default.table_49a233eb03e6426081975d230536b222
I20260925 09:58:29.205111 13080 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:50556 (request call id 111) took 3522 ms. Trace:
I20260925 09:58:29.205470 13080 rpcz_store.cc:276] 0925 09:58:25.682669 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:25.682795 (+ 126us) service_pool.cc:227] Handling call
0925 09:58:28.047897 (+2365102us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:58:29.202476 (+1154579us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0925 09:58:29.204288 (+ 1812us) catalog_manager.cc:3616] Apply alter schema
0925 09:58:29.205049 (+ 761us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":706897,"Hive Metastore.run_cpu_time_us":1947,"Hive Metastore.run_wall_time_us":421763}
W20260925 09:58:29.234661 13440 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.3:40731, user_credentials={real_user=slave}} blocked reactor thread for 46384.2us
I20260925 09:58:29.245004 14414 consensus_queue.cc:1035] T 4b61aaac33514c999482b824e0839a72 P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:29.265808 14409 consensus_queue.cc:1035] T 0090f685d0184822b41fd3e731874567 P a06c3ef7a6a94f7cbcc03e7cd232f237 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:58:29.267083 14416 consensus_queue.cc:1035] T b9cf55829fed4c93a8eab92cee8920c9 P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260925 09:58:29.300247 13074 catalog_manager.cc:2148] created HMS catalog entry for table [id=5c00787090a0439e8cbb872359f800f2]
W20260925 09:58:29.302416 13074 catalog_manager.cc:2176] Illegal state: an error occurred while writing to the sys-catalog: replica b7d0c1c2f1ed4e7a9ed54e73869d53b0 is not leader of this config: current role FOLLOWER
I20260925 09:58:29.328766 14395 tablet_bootstrap.cc:654] T 11f8c7b5918a431aa9c4975f28ea89cb P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:29.376516 14396 tablet_bootstrap.cc:492] T e370ffe0dec24f40a3b446ed4cdc831b P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
W20260925 09:58:29.376611 13606 consensus_peers.cc:487] T 0090f685d0184822b41fd3e731874567 P a06c3ef7a6a94f7cbcc03e7cd232f237 -> Peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Couldn't send request to peer c4d724d4b83e4d1e817a3a505dc24eb9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0090f685d0184822b41fd3e731874567. This is attempt 1: this message will repeat every 5th retry.
I20260925 09:58:29.376940 14396 ts_tablet_manager.cc:1397] T e370ffe0dec24f40a3b446ed4cdc831b P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.692s user 0.081s sys 0.133s
I20260925 09:58:29.379853 14396 raft_consensus.cc:348] T e370ffe0dec24f40a3b446ed4cdc831b P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:29.380540 14396 raft_consensus.cc:374] T e370ffe0dec24f40a3b446ed4cdc831b P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:29.380810 14396 raft_consensus.cc:729] T e370ffe0dec24f40a3b446ed4cdc831b P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:29.381560 14396 consensus_queue.cc:260] T e370ffe0dec24f40a3b446ed4cdc831b P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:29.383754 14396 ts_tablet_manager.cc:1428] T e370ffe0dec24f40a3b446ed4cdc831b P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.007s user 0.006s sys 0.001s
I20260925 09:58:29.384598 14396 tablet_bootstrap.cc:492] T 11f8c7b5918a431aa9c4975f28ea89cb P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:29.398741 14395 tablet_bootstrap.cc:492] T 11f8c7b5918a431aa9c4975f28ea89cb P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:29.402225 14395 ts_tablet_manager.cc:1397] T 11f8c7b5918a431aa9c4975f28ea89cb P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.420s user 0.013s sys 0.004s
I20260925 09:58:29.406059 14351 raft_consensus.cc:482] T 11f8c7b5918a431aa9c4975f28ea89cb P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:29.406384 13079 catalog_manager.cc:2148] created HMS catalog entry for table [id=14d87f883414443880d145ad7ef9354b]
I20260925 09:58:29.407248 14351 raft_consensus.cc:504] T 11f8c7b5918a431aa9c4975f28ea89cb P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:29.409556 14395 raft_consensus.cc:348] T 11f8c7b5918a431aa9c4975f28ea89cb P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:29.410221 14395 raft_consensus.cc:374] T 11f8c7b5918a431aa9c4975f28ea89cb P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:29.410486 14395 raft_consensus.cc:729] T 11f8c7b5918a431aa9c4975f28ea89cb P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:29.411116 14395 consensus_queue.cc:260] T 11f8c7b5918a431aa9c4975f28ea89cb P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:29.411645 13670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11f8c7b5918a431aa9c4975f28ea89cb" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
W20260925 09:58:29.413017 13440 leader_election.cc:341] T 11f8c7b5918a431aa9c4975f28ea89cb P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Illegal state: must be running to vote when last-logged opid is not known
I20260925 09:58:29.413318 14395 ts_tablet_manager.cc:1428] T 11f8c7b5918a431aa9c4975f28ea89cb P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.010s user 0.004s sys 0.000s
I20260925 09:58:29.414215 14395 tablet_bootstrap.cc:492] T 733492d93592485e920628c47d5c4251 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:29.419111 14351 leader_election.cc:288] T 11f8c7b5918a431aa9c4975f28ea89cb P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:29.424762 13362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11f8c7b5918a431aa9c4975f28ea89cb" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:29.425527 13362 raft_consensus.cc:2454] T 11f8c7b5918a431aa9c4975f28ea89cb P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 0.
I20260925 09:58:29.427182 14391 tablet_bootstrap.cc:654] T 733492d93592485e920628c47d5c4251 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:29.432946 14395 tablet_bootstrap.cc:654] T 733492d93592485e920628c47d5c4251 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
W20260925 09:58:29.448700 13079 catalog_manager.cc:2176] Illegal state: an error occurred while writing to the sys-catalog: replica b7d0c1c2f1ed4e7a9ed54e73869d53b0 is not leader of this config: current role FOLLOWER
I20260925 09:58:29.450160 14396 tablet_bootstrap.cc:654] T 11f8c7b5918a431aa9c4975f28ea89cb P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:29.450330 13438 leader_election.cc:302] T 11f8c7b5918a431aa9c4975f28ea89cb P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237
I20260925 09:58:29.452064 14414 raft_consensus.cc:2781] T 11f8c7b5918a431aa9c4975f28ea89cb P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:29.452280 14395 tablet_bootstrap.cc:492] T 733492d93592485e920628c47d5c4251 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:29.452441 14414 raft_consensus.cc:482] T 11f8c7b5918a431aa9c4975f28ea89cb P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:29.452689 14395 ts_tablet_manager.cc:1397] T 733492d93592485e920628c47d5c4251 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.039s user 0.016s sys 0.000s
I20260925 09:58:29.452710 14414 raft_consensus.cc:3037] T 11f8c7b5918a431aa9c4975f28ea89cb P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:29.453217 14391 tablet_bootstrap.cc:492] T 733492d93592485e920628c47d5c4251 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:29.453790 14391 ts_tablet_manager.cc:1397] T 733492d93592485e920628c47d5c4251 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.277s user 0.007s sys 0.008s
I20260925 09:58:29.455255 14395 raft_consensus.cc:348] T 733492d93592485e920628c47d5c4251 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:29.455729 14357 raft_consensus.cc:482] T e370ffe0dec24f40a3b446ed4cdc831b P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:29.456084 14395 raft_consensus.cc:374] T 733492d93592485e920628c47d5c4251 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:29.456331 14395 raft_consensus.cc:729] T 733492d93592485e920628c47d5c4251 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:29.456183 14357 raft_consensus.cc:504] T e370ffe0dec24f40a3b446ed4cdc831b P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:29.456990 14395 consensus_queue.cc:260] T 733492d93592485e920628c47d5c4251 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:29.457640 14391 raft_consensus.cc:348] T 733492d93592485e920628c47d5c4251 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:29.459792 14357 leader_election.cc:288] T e370ffe0dec24f40a3b446ed4cdc831b P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161)
I20260925 09:58:29.459663 14395 ts_tablet_manager.cc:1428] T 733492d93592485e920628c47d5c4251 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260925 09:58:29.460940 13361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e370ffe0dec24f40a3b446ed4cdc831b" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:29.461498 13361 raft_consensus.cc:2454] T e370ffe0dec24f40a3b446ed4cdc831b P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 0.
I20260925 09:58:29.462173 14391 raft_consensus.cc:374] T 733492d93592485e920628c47d5c4251 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:29.462471 14391 raft_consensus.cc:729] T 733492d93592485e920628c47d5c4251 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:29.466326 14391 consensus_queue.cc:260] T 733492d93592485e920628c47d5c4251 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:29.470271 13603 leader_election.cc:302] T e370ffe0dec24f40a3b446ed4cdc831b P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, b3082cf15dc74c0480e5d33ecf592843; no voters:
I20260925 09:58:29.472391 13649 tablet_service.cc:1467] Processing CreateTablet for tablet fe39e8e4c3ad47f393565dde2cded405 (DEFAULT_TABLE table=default.table_688d7c9730444646b6d7a901d5537807 [id=c06b5e37a4f14eb5b653ada7e7f73b4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:29.472769 14414 raft_consensus.cc:504] T 11f8c7b5918a431aa9c4975f28ea89cb P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:29.473379 13649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe39e8e4c3ad47f393565dde2cded405. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:29.479242 13504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e370ffe0dec24f40a3b446ed4cdc831b" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
I20260925 09:58:29.480130 13504 raft_consensus.cc:2454] T e370ffe0dec24f40a3b446ed4cdc831b P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 0.
I20260925 09:58:29.487234 14391 ts_tablet_manager.cc:1428] T 733492d93592485e920628c47d5c4251 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.033s user 0.004s sys 0.006s
I20260925 09:58:29.488165 13340 tablet_service.cc:1467] Processing CreateTablet for tablet fe39e8e4c3ad47f393565dde2cded405 (DEFAULT_TABLE table=default.table_688d7c9730444646b6d7a901d5537807 [id=c06b5e37a4f14eb5b653ada7e7f73b4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:29.489266 13340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe39e8e4c3ad47f393565dde2cded405. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:29.491151 14396 tablet_bootstrap.cc:492] T 11f8c7b5918a431aa9c4975f28ea89cb P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:29.494712 14396 ts_tablet_manager.cc:1397] T 11f8c7b5918a431aa9c4975f28ea89cb P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.110s user 0.005s sys 0.008s
I20260925 09:58:29.504401 13482 tablet_service.cc:1467] Processing CreateTablet for tablet fe39e8e4c3ad47f393565dde2cded405 (DEFAULT_TABLE table=default.table_688d7c9730444646b6d7a901d5537807 [id=c06b5e37a4f14eb5b653ada7e7f73b4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 09:58:29.505545 13482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe39e8e4c3ad47f393565dde2cded405. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:29.506472 14357 raft_consensus.cc:2781] T e370ffe0dec24f40a3b446ed4cdc831b P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:29.506803 14357 raft_consensus.cc:482] T e370ffe0dec24f40a3b446ed4cdc831b P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:29.507170 14357 raft_consensus.cc:3037] T e370ffe0dec24f40a3b446ed4cdc831b P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:29.506955 13006 catalog_manager.cc:5587] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327) does not have the latest schema for tablet 11f8c7b5918a431aa9c4975f28ea89cb (table default.table_eda42a4c592f447b8084b26441b81b7c [id=7e0502ffa5544a5bb252a72020649cd7]). Expected version 2 got 0
I20260925 09:58:29.531186 14396 raft_consensus.cc:348] T 11f8c7b5918a431aa9c4975f28ea89cb P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:29.546947 14396 raft_consensus.cc:374] T 11f8c7b5918a431aa9c4975f28ea89cb P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:29.547515 14396 raft_consensus.cc:729] T 11f8c7b5918a431aa9c4975f28ea89cb P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:29.549340 14391 tablet_bootstrap.cc:492] T fe39e8e4c3ad47f393565dde2cded405 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:29.543236 14357 raft_consensus.cc:504] T e370ffe0dec24f40a3b446ed4cdc831b P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:29.550621 14396 consensus_queue.cc:260] T 11f8c7b5918a431aa9c4975f28ea89cb P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:29.555534 14391 tablet_bootstrap.cc:654] T fe39e8e4c3ad47f393565dde2cded405 P c4d724d4b83e4d1e817a3a505dc24eb9: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:29.570083 13006 catalog_manager.cc:5587] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731) does not have the latest schema for tablet 11f8c7b5918a431aa9c4975f28ea89cb (table default.table_eda42a4c592f447b8084b26441b81b7c [id=7e0502ffa5544a5bb252a72020649cd7]). Expected version 2 got 0
I20260925 09:58:29.570384 14396 ts_tablet_manager.cc:1428] T 11f8c7b5918a431aa9c4975f28ea89cb P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.074s user 0.002s sys 0.007s
I20260925 09:58:29.581494 14396 tablet_bootstrap.cc:492] T 733492d93592485e920628c47d5c4251 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:29.582469 13003 catalog_manager.cc:5587] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731) does not have the latest schema for tablet 11f8c7b5918a431aa9c4975f28ea89cb (table default.table_eda42a4c592f447b8084b26441b81b7c [id=7e0502ffa5544a5bb252a72020649cd7]). Expected version 2 got 0
I20260925 09:58:29.584249 14391 tablet_bootstrap.cc:492] T fe39e8e4c3ad47f393565dde2cded405 P c4d724d4b83e4d1e817a3a505dc24eb9: No bootstrap required, opened a new log
I20260925 09:58:29.584717 14391 ts_tablet_manager.cc:1397] T fe39e8e4c3ad47f393565dde2cded405 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.036s user 0.008s sys 0.004s
I20260925 09:58:29.587307 14391 raft_consensus.cc:348] T fe39e8e4c3ad47f393565dde2cded405 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:29.593484 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11f8c7b5918a431aa9c4975f28ea89cb" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843"
I20260925 09:58:29.594009 13363 raft_consensus.cc:3037] T 11f8c7b5918a431aa9c4975f28ea89cb P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:29.595065 13361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e370ffe0dec24f40a3b446ed4cdc831b" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843"
I20260925 09:58:29.595275 13668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11f8c7b5918a431aa9c4975f28ea89cb" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
I20260925 09:58:29.595674 13361 raft_consensus.cc:3037] T e370ffe0dec24f40a3b446ed4cdc831b P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:29.595834 13668 raft_consensus.cc:3037] T 11f8c7b5918a431aa9c4975f28ea89cb P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:29.593986 13504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e370ffe0dec24f40a3b446ed4cdc831b" candidate_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9"
I20260925 09:58:29.599123 13504 raft_consensus.cc:3037] T e370ffe0dec24f40a3b446ed4cdc831b P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:29.599999 13363 raft_consensus.cc:2454] T 11f8c7b5918a431aa9c4975f28ea89cb P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:29.601053 13438 leader_election.cc:302] T 11f8c7b5918a431aa9c4975f28ea89cb P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:29.601873 14416 raft_consensus.cc:2781] T 11f8c7b5918a431aa9c4975f28ea89cb P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:29.602056 13668 raft_consensus.cc:2454] T 11f8c7b5918a431aa9c4975f28ea89cb P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:29.602280 14416 raft_consensus.cc:686] T 11f8c7b5918a431aa9c4975f28ea89cb P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: Becoming Leader. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Running, Role: LEADER
I20260925 09:58:29.602793 13361 raft_consensus.cc:2454] T e370ffe0dec24f40a3b446ed4cdc831b P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 1.
I20260925 09:58:29.603283 14414 leader_election.cc:288] T 11f8c7b5918a431aa9c4975f28ea89cb P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Requested vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:29.603065 14416 consensus_queue.cc:237] T 11f8c7b5918a431aa9c4975f28ea89cb P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:29.604851 14395 tablet_bootstrap.cc:492] T fe39e8e4c3ad47f393565dde2cded405 P b3082cf15dc74c0480e5d33ecf592843: Bootstrap starting.
I20260925 09:58:29.605984 13504 raft_consensus.cc:2454] T e370ffe0dec24f40a3b446ed4cdc831b P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a06c3ef7a6a94f7cbcc03e7cd232f237 in term 1.
I20260925 09:58:29.606796 14391 raft_consensus.cc:374] T fe39e8e4c3ad47f393565dde2cded405 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:29.607146 14391 raft_consensus.cc:729] T fe39e8e4c3ad47f393565dde2cded405 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:29.607146 13606 leader_election.cc:302] T e370ffe0dec24f40a3b446ed4cdc831b P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:29.607993 14409 raft_consensus.cc:2781] T e370ffe0dec24f40a3b446ed4cdc831b P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:29.607836 14391 consensus_queue.cc:260] T fe39e8e4c3ad47f393565dde2cded405 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:29.608404 14409 raft_consensus.cc:686] T e370ffe0dec24f40a3b446ed4cdc831b P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 LEADER]: Becoming Leader. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Running, Role: LEADER
I20260925 09:58:29.609131 14409 consensus_queue.cc:237] T e370ffe0dec24f40a3b446ed4cdc831b P a06c3ef7a6a94f7cbcc03e7cd232f237 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:29.609979 14391 ts_tablet_manager.cc:1428] T fe39e8e4c3ad47f393565dde2cded405 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.025s user 0.005s sys 0.000s
I20260925 09:58:29.612175 14395 tablet_bootstrap.cc:654] T fe39e8e4c3ad47f393565dde2cded405 P b3082cf15dc74c0480e5d33ecf592843: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:29.612268 14357 leader_election.cc:288] T e370ffe0dec24f40a3b446ed4cdc831b P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 election: Requested vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161)
I20260925 09:58:29.619071 13005 catalog_manager.cc:5587] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161) does not have the latest schema for tablet 11f8c7b5918a431aa9c4975f28ea89cb (table default.table_eda42a4c592f447b8084b26441b81b7c [id=7e0502ffa5544a5bb252a72020649cd7]). Expected version 2 got 0
F20260925 09:58:29.628461 13223 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 36): 36 DROP_TABLE default.table_41304460d5a345c88866a7429d816aca
*** Check failure stack trace: ***
I20260925 09:58:29.641146 13074 catalog_manager.cc:2158] deleted orphaned HMS catalog entry for table [id=5c00787090a0439e8cbb872359f800f2]
I20260925 09:58:29.642035 13074 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:50556 (request call id 114) took 3642 ms. Trace:
I20260925 09:58:29.642387 13074 rpcz_store.cc:276] 0925 09:58:25.999781 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:26.001260 (+ 1479us) service_pool.cc:227] Handling call
0925 09:58:28.085598 (+2084338us) catalog_manager.cc:2075] Acquired catalog manager lock
0925 09:58:28.090082 (+ 4484us) catalog_manager.cc:2121] Created new table and tablet info
0925 09:58:29.300199 (+1210117us) catalog_manager.cc:2147] Created new table in HMS catalog
0925 09:58:29.302166 (+ 1967us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:29.302193 (+ 27us) write_op.cc:454] Released schema lock
0925 09:58:29.302724 (+ 531us) catalog_manager.cc:2155] Rolling back HMS table creation
0925 09:58:29.641972 (+339248us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:58:29.301911 (+ 0us) op_driver.cc:372] HandleFailure(Illegal state: replica b7d0c1c2f1ed4e7a9ed54e73869d53b0 is not leader of this config: current role FOLLOWER)
0925 09:58:29.301953 (+ 42us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:29.301976 (+ 23us) write_op.cc:454] Released schema lock
0925 09:58:29.302008 (+ 32us) write_op.cc:335] FINISH: Op aborted
Metrics: {"Hive Metastore.queue_time_us":1189625,"Hive Metastore.run_cpu_time_us":932,"Hive Metastore.run_wall_time_us":357143,"child_traces":[["op",{}]]}
W20260925 09:58:29.646090 13728 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.10.9.62:43437)
I20260925 09:58:29.652294 14357 raft_consensus.cc:482] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:29.652748 14357 raft_consensus.cc:504] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:29.654542 14357 leader_election.cc:288] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
W20260925 09:58:29.658778 13606 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
W20260925 09:58:29.660048 13603 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
I20260925 09:58:29.660400 13603 leader_election.cc:302] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237; no voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:58:29.661306 14357 raft_consensus.cc:2726] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:58:29.663800 13003 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49450:
name: "default.Table_1b826b3631274aca84923893f7383ee3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:58:29.674064 13005 catalog_manager.cc:5495] T 11f8c7b5918a431aa9c4975f28ea89cb P c4d724d4b83e4d1e817a3a505dc24eb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2). New cstate: current_term: 1 leader_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: UNKNOWN } } }
I20260925 09:58:29.675899 13005 catalog_manager.cc:5587] TS c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327) does not have the latest schema for tablet 11f8c7b5918a431aa9c4975f28ea89cb (table default.table_eda42a4c592f447b8084b26441b81b7c [id=7e0502ffa5544a5bb252a72020649cd7]). Expected version 2 got 0
I20260925 09:58:29.698022 14396 tablet_bootstrap.cc:654] T 733492d93592485e920628c47d5c4251 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:29.699716 13006 catalog_manager.cc:5495] T e370ffe0dec24f40a3b446ed4cdc831b P a06c3ef7a6a94f7cbcc03e7cd232f237 reported cstate change: term changed from 0 to 1, leader changed from <none> to a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3). New cstate: current_term: 1 leader_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: HEALTHY } } }
I20260925 09:58:29.720568 14396 tablet_bootstrap.cc:492] T 733492d93592485e920628c47d5c4251 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:29.721084 14396 ts_tablet_manager.cc:1397] T 733492d93592485e920628c47d5c4251 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.141s user 0.011s sys 0.000s
I20260925 09:58:29.724043 14396 raft_consensus.cc:348] T 733492d93592485e920628c47d5c4251 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:29.724902 14396 raft_consensus.cc:374] T 733492d93592485e920628c47d5c4251 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:29.725214 14396 raft_consensus.cc:729] T 733492d93592485e920628c47d5c4251 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:29.731710 14396 consensus_queue.cc:260] T 733492d93592485e920628c47d5c4251 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:29.744781 14396 ts_tablet_manager.cc:1428] T 733492d93592485e920628c47d5c4251 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.023s user 0.007s sys 0.001s
I20260925 09:58:29.745836 14396 tablet_bootstrap.cc:492] T fe39e8e4c3ad47f393565dde2cded405 P a06c3ef7a6a94f7cbcc03e7cd232f237: Bootstrap starting.
I20260925 09:58:29.748863 14395 tablet_bootstrap.cc:492] T fe39e8e4c3ad47f393565dde2cded405 P b3082cf15dc74c0480e5d33ecf592843: No bootstrap required, opened a new log
I20260925 09:58:29.752072 14396 tablet_bootstrap.cc:654] T fe39e8e4c3ad47f393565dde2cded405 P a06c3ef7a6a94f7cbcc03e7cd232f237: Neither blocks nor log segments found. Creating new log.
I20260925 09:58:29.753496 14395 ts_tablet_manager.cc:1397] T fe39e8e4c3ad47f393565dde2cded405 P b3082cf15dc74c0480e5d33ecf592843: Time spent bootstrapping tablet: real 0.149s user 0.012s sys 0.007s
I20260925 09:58:29.762375 14416 raft_consensus.cc:482] T 733492d93592485e920628c47d5c4251 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:29.762847 14416 raft_consensus.cc:504] T 733492d93592485e920628c47d5c4251 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:29.764930 14416 leader_election.cc:288] T 733492d93592485e920628c47d5c4251 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:29.766664 13668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "733492d93592485e920628c47d5c4251" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:29.767380 13668 raft_consensus.cc:2454] T 733492d93592485e920628c47d5c4251 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 0.
I20260925 09:58:29.773342 13363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "733492d93592485e920628c47d5c4251" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843" is_pre_election: true
I20260925 09:58:29.774152 13363 raft_consensus.cc:2454] T 733492d93592485e920628c47d5c4251 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 0.
I20260925 09:58:29.775444 13440 leader_election.cc:302] T 733492d93592485e920628c47d5c4251 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:29.777217 14416 raft_consensus.cc:2781] T 733492d93592485e920628c47d5c4251 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:29.777631 14416 raft_consensus.cc:482] T 733492d93592485e920628c47d5c4251 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:29.778026 14416 raft_consensus.cc:3037] T 733492d93592485e920628c47d5c4251 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:29.785702 13668 raft_consensus.cc:1261] T 11f8c7b5918a431aa9c4975f28ea89cb P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Refusing update from remote peer c4d724d4b83e4d1e817a3a505dc24eb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:58:29.787521 14421 consensus_queue.cc:1035] T 11f8c7b5918a431aa9c4975f28ea89cb P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260925 09:58:29.799281 13363 raft_consensus.cc:1261] T 11f8c7b5918a431aa9c4975f28ea89cb P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Refusing update from remote peer c4d724d4b83e4d1e817a3a505dc24eb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 09:58:29.801198 14421 consensus_queue.cc:1035] T 11f8c7b5918a431aa9c4975f28ea89cb P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
@ 0x7f0a1d1939a0 google::LogMessage::Flush() at ??:0
I20260925 09:58:29.825073 14395 raft_consensus.cc:348] T fe39e8e4c3ad47f393565dde2cded405 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:29.825765 14395 raft_consensus.cc:374] T fe39e8e4c3ad47f393565dde2cded405 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:29.826025 14395 raft_consensus.cc:729] T fe39e8e4c3ad47f393565dde2cded405 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Initialized, Role: FOLLOWER
I20260925 09:58:29.826963 14395 consensus_queue.cc:260] T fe39e8e4c3ad47f393565dde2cded405 P b3082cf15dc74c0480e5d33ecf592843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:29.830564 14416 raft_consensus.cc:504] T 733492d93592485e920628c47d5c4251 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:29.834729 13361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "733492d93592485e920628c47d5c4251" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3082cf15dc74c0480e5d33ecf592843"
I20260925 09:58:29.835253 13361 raft_consensus.cc:3037] T 733492d93592485e920628c47d5c4251 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:29.842381 13361 raft_consensus.cc:2454] T 733492d93592485e920628c47d5c4251 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:29.846478 13005 catalog_manager.cc:5587] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731) does not have the latest schema for tablet 11f8c7b5918a431aa9c4975f28ea89cb (table default.table_eda42a4c592f447b8084b26441b81b7c [id=7e0502ffa5544a5bb252a72020649cd7]). Expected version 2 got 0
I20260925 09:58:29.851086 13438 leader_election.cc:302] T 733492d93592485e920628c47d5c4251 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:29.851903 14421 raft_consensus.cc:2781] T 733492d93592485e920628c47d5c4251 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:29.852346 14421 raft_consensus.cc:686] T 733492d93592485e920628c47d5c4251 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 LEADER]: Becoming Leader. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Running, Role: LEADER
I20260925 09:58:29.853109 14421 consensus_queue.cc:237] T 733492d93592485e920628c47d5c4251 P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:29.859951 13006 catalog_manager.cc:5587] TS b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161) does not have the latest schema for tablet 11f8c7b5918a431aa9c4975f28ea89cb (table default.table_eda42a4c592f447b8084b26441b81b7c [id=7e0502ffa5544a5bb252a72020649cd7]). Expected version 2 got 0
I20260925 09:58:29.876610 14395 ts_tablet_manager.cc:1428] T fe39e8e4c3ad47f393565dde2cded405 P b3082cf15dc74c0480e5d33ecf592843: Time spent starting tablet: real 0.118s user 0.006s sys 0.002s
I20260925 09:58:29.879324 13079 catalog_manager.cc:2158] deleted orphaned HMS catalog entry for table [id=14d87f883414443880d145ad7ef9354b]
I20260925 09:58:29.880158 13079 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:50556 (request call id 117) took 3443 ms. Trace:
I20260925 09:58:29.880424 13079 rpcz_store.cc:276] 0925 09:58:26.436535 (+ 0us) service_pool.cc:170] Inserting onto call queue
0925 09:58:26.436663 (+ 128us) service_pool.cc:227] Handling call
0925 09:58:28.087169 (+1650506us) catalog_manager.cc:2075] Acquired catalog manager lock
0925 09:58:28.091930 (+ 4761us) catalog_manager.cc:2121] Created new table and tablet info
0925 09:58:29.406345 (+1314415us) catalog_manager.cc:2147] Created new table in HMS catalog
0925 09:58:29.448410 (+ 42065us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:29.448440 (+ 30us) write_op.cc:454] Released schema lock
0925 09:58:29.449079 (+ 639us) catalog_manager.cc:2155] Rolling back HMS table creation
0925 09:58:29.880099 (+431020us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0925 09:58:29.448158 (+ 0us) op_driver.cc:372] HandleFailure(Illegal state: replica b7d0c1c2f1ed4e7a9ed54e73869d53b0 is not leader of this config: current role FOLLOWER)
0925 09:58:29.448205 (+ 47us) write_op.cc:489] Releasing partition, row and schema locks
0925 09:58:29.448235 (+ 30us) write_op.cc:454] Released schema lock
0925 09:58:29.448264 (+ 29us) write_op.cc:335] FINISH: Op aborted
Metrics: {"Hive Metastore.queue_time_us":1398887,"Hive Metastore.run_cpu_time_us":897,"Hive Metastore.run_wall_time_us":344317,"child_traces":[["op",{}]]}
I20260925 09:58:29.891884 14447 tablet.cc:1722] T 11f8c7b5918a431aa9c4975f28ea89cb P b3082cf15dc74c0480e5d33ecf592843: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 09:58:29.893309 13005 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49450:
name: "default.Table_489bd8fcb4594ceabc28fdbc0a951dd0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 09:58:29.894608 14416 leader_election.cc:288] T 733492d93592485e920628c47d5c4251 P c4d724d4b83e4d1e817a3a505dc24eb9 [CANDIDATE]: Term 1 election: Requested vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:29.895650 13668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "733492d93592485e920628c47d5c4251" candidate_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
I20260925 09:58:29.896229 13668 raft_consensus.cc:3037] T 733492d93592485e920628c47d5c4251 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:29.914172 14396 tablet_bootstrap.cc:492] T fe39e8e4c3ad47f393565dde2cded405 P a06c3ef7a6a94f7cbcc03e7cd232f237: No bootstrap required, opened a new log
I20260925 09:58:29.914688 14396 ts_tablet_manager.cc:1397] T fe39e8e4c3ad47f393565dde2cded405 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent bootstrapping tablet: real 0.169s user 0.009s sys 0.014s
I20260925 09:58:29.917502 14396 raft_consensus.cc:348] T fe39e8e4c3ad47f393565dde2cded405 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:29.918223 14396 raft_consensus.cc:374] T fe39e8e4c3ad47f393565dde2cded405 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 09:58:29.918511 14396 raft_consensus.cc:729] T fe39e8e4c3ad47f393565dde2cded405 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06c3ef7a6a94f7cbcc03e7cd232f237, State: Initialized, Role: FOLLOWER
I20260925 09:58:29.920562 14446 tablet.cc:1722] T 11f8c7b5918a431aa9c4975f28ea89cb P a06c3ef7a6a94f7cbcc03e7cd232f237: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 09:58:29.925002 14445 tablet.cc:1722] T 11f8c7b5918a431aa9c4975f28ea89cb P c4d724d4b83e4d1e817a3a505dc24eb9: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260925 09:58:29.932920 13007 catalog_manager.cc:5495] T 733492d93592485e920628c47d5c4251 P c4d724d4b83e4d1e817a3a505dc24eb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2). New cstate: current_term: 1 leader_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: UNKNOWN } } }
I20260925 09:58:29.903169 13668 raft_consensus.cc:2454] T 733492d93592485e920628c47d5c4251 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d724d4b83e4d1e817a3a505dc24eb9 in term 1.
I20260925 09:58:29.919158 14396 consensus_queue.cc:260] T fe39e8e4c3ad47f393565dde2cded405 P a06c3ef7a6a94f7cbcc03e7cd232f237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:29.964452 13004 catalog_manager.cc:5587] TS a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731) does not have the latest schema for tablet 11f8c7b5918a431aa9c4975f28ea89cb (table default.table_eda42a4c592f447b8084b26441b81b7c [id=7e0502ffa5544a5bb252a72020649cd7]). Expected version 2 got 0
I20260925 09:58:29.965116 14396 ts_tablet_manager.cc:1428] T fe39e8e4c3ad47f393565dde2cded405 P a06c3ef7a6a94f7cbcc03e7cd232f237: Time spent starting tablet: real 0.050s user 0.004s sys 0.004s
I20260925 09:58:30.002430 13006 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c: default.table_eda42a4c592f447b8084b26441b81b7c [id=7e0502ffa5544a5bb252a72020649cd7] alter complete (version 2)
@ 0x7f0a1d19370f google::LogMessage::~LogMessage() at ??:0
I20260925 09:58:30.040081 14269 raft_consensus.cc:482] T fe39e8e4c3ad47f393565dde2cded405 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:30.040524 14269 raft_consensus.cc:504] T fe39e8e4c3ad47f393565dde2cded405 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:30.042898 14269 leader_election.cc:288] T fe39e8e4c3ad47f393565dde2cded405 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:30.044085 13668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe39e8e4c3ad47f393565dde2cded405" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" is_pre_election: true
I20260925 09:58:30.044673 13668 raft_consensus.cc:2454] T fe39e8e4c3ad47f393565dde2cded405 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 0.
I20260925 09:58:30.045603 13297 leader_election.cc:302] T fe39e8e4c3ad47f393565dde2cded405 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237, b3082cf15dc74c0480e5d33ecf592843; no voters:
I20260925 09:58:30.045639 13504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe39e8e4c3ad47f393565dde2cded405" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" is_pre_election: true
I20260925 09:58:30.046195 13504 raft_consensus.cc:2454] T fe39e8e4c3ad47f393565dde2cded405 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 0.
I20260925 09:58:30.050729 14269 raft_consensus.cc:2781] T fe39e8e4c3ad47f393565dde2cded405 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 09:58:30.051477 14269 raft_consensus.cc:482] T fe39e8e4c3ad47f393565dde2cded405 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 09:58:30.052121 14269 raft_consensus.cc:3037] T fe39e8e4c3ad47f393565dde2cded405 P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:30.078574 14269 raft_consensus.cc:504] T fe39e8e4c3ad47f393565dde2cded405 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:30.086663 14449 ts_tablet_manager.cc:927] T fafc2b657ace4a4381452ce43fad39b6 P c4d724d4b83e4d1e817a3a505dc24eb9: Initiating tablet copy from peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161)
I20260925 09:58:30.090083 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe39e8e4c3ad47f393565dde2cded405" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9"
I20260925 09:58:30.090749 13505 raft_consensus.cc:3037] T fe39e8e4c3ad47f393565dde2cded405 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:30.097842 14269 leader_election.cc:288] T fe39e8e4c3ad47f393565dde2cded405 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Requested vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:30.098748 13668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe39e8e4c3ad47f393565dde2cded405" candidate_uuid: "b3082cf15dc74c0480e5d33ecf592843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237"
I20260925 09:58:30.099272 13668 raft_consensus.cc:3037] T fe39e8e4c3ad47f393565dde2cded405 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Advancing to term 1
I20260925 09:58:30.104025 13505 raft_consensus.cc:2454] T fe39e8e4c3ad47f393565dde2cded405 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 1.
I20260925 09:58:30.105021 13298 leader_election.cc:302] T fe39e8e4c3ad47f393565dde2cded405 P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9; no voters:
I20260925 09:58:30.105804 14269 raft_consensus.cc:2781] T fe39e8e4c3ad47f393565dde2cded405 P b3082cf15dc74c0480e5d33ecf592843 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 09:58:30.106281 14269 raft_consensus.cc:686] T fe39e8e4c3ad47f393565dde2cded405 P b3082cf15dc74c0480e5d33ecf592843 [term 1 LEADER]: Becoming Leader. State: Replica: b3082cf15dc74c0480e5d33ecf592843, State: Running, Role: LEADER
I20260925 09:58:30.109370 13668 raft_consensus.cc:2454] T fe39e8e4c3ad47f393565dde2cded405 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3082cf15dc74c0480e5d33ecf592843 in term 1.
I20260925 09:58:30.110183 14269 consensus_queue.cc:237] T fe39e8e4c3ad47f393565dde2cded405 P b3082cf15dc74c0480e5d33ecf592843 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } }
I20260925 09:58:30.114039 14449 tablet_copy_client.cc:316] T fafc2b657ace4a4381452ce43fad39b6 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.9.1:42161
I20260925 09:58:30.116158 13383 tablet_copy_service.cc:140] P b3082cf15dc74c0480e5d33ecf592843: Received BeginTabletCopySession request for tablet fafc2b657ace4a4381452ce43fad39b6 from peer c4d724d4b83e4d1e817a3a505dc24eb9 ({username='slave'} at 127.10.9.2:37295)
I20260925 09:58:30.116634 13383 tablet_copy_service.cc:161] P b3082cf15dc74c0480e5d33ecf592843: Beginning new tablet copy session on tablet fafc2b657ace4a4381452ce43fad39b6 from peer c4d724d4b83e4d1e817a3a505dc24eb9 at {username='slave'} at 127.10.9.2:37295: session id = c4d724d4b83e4d1e817a3a505dc24eb9-fafc2b657ace4a4381452ce43fad39b6
I20260925 09:58:30.126844 13383 tablet_copy_source_session.cc:205] T fafc2b657ace4a4381452ce43fad39b6 P b3082cf15dc74c0480e5d33ecf592843: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 09:58:30.134069 14449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fafc2b657ace4a4381452ce43fad39b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:30.154627 13006 catalog_manager.cc:5495] T fe39e8e4c3ad47f393565dde2cded405 P b3082cf15dc74c0480e5d33ecf592843 reported cstate change: term changed from 0 to 1, leader changed from <none> to b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1). New cstate: current_term: 1 leader_uuid: "b3082cf15dc74c0480e5d33ecf592843" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } health_report { overall_health: HEALTHY } } }
I20260925 09:58:30.177376 14449 tablet_copy_client.cc:791] T fafc2b657ace4a4381452ce43fad39b6 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet copy: Starting download of 0 data blocks...
I20260925 09:58:30.178020 14449 tablet_copy_client.cc:655] T fafc2b657ace4a4381452ce43fad39b6 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet copy: Starting download of 1 WAL segments...
I20260925 09:58:30.182406 14449 tablet_copy_client.cc:531] T fafc2b657ace4a4381452ce43fad39b6 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
@ 0x7f0a2822eaea kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260925 09:58:30.208174 14449 tablet_bootstrap.cc:492] T fafc2b657ace4a4381452ce43fad39b6 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:30.239279 14449 tablet_bootstrap.cc:492] T fafc2b657ace4a4381452ce43fad39b6 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 09:58:30.240408 14449 tablet_bootstrap.cc:492] T fafc2b657ace4a4381452ce43fad39b6 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap complete.
I20260925 09:58:30.246186 14449 ts_tablet_manager.cc:1397] T fafc2b657ace4a4381452ce43fad39b6 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.038s user 0.014s sys 0.004s
I20260925 09:58:30.248796 14449 raft_consensus.cc:348] T fafc2b657ace4a4381452ce43fad39b6 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:30.249557 14449 raft_consensus.cc:729] T fafc2b657ace4a4381452ce43fad39b6 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:30.250334 14449 consensus_queue.cc:260] T fafc2b657ace4a4381452ce43fad39b6 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:30.261191 14449 ts_tablet_manager.cc:1428] T fafc2b657ace4a4381452ce43fad39b6 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.015s user 0.004s sys 0.000s
I20260925 09:58:30.263655 13383 tablet_copy_service.cc:342] P b3082cf15dc74c0480e5d33ecf592843: Request end of tablet copy session c4d724d4b83e4d1e817a3a505dc24eb9-fafc2b657ace4a4381452ce43fad39b6 received from {username='slave'} at 127.10.9.2:37295
I20260925 09:58:30.264032 13383 tablet_copy_service.cc:433] P b3082cf15dc74c0480e5d33ecf592843: ending tablet copy session c4d724d4b83e4d1e817a3a505dc24eb9-fafc2b657ace4a4381452ce43fad39b6 on tablet fafc2b657ace4a4381452ce43fad39b6 with peer c4d724d4b83e4d1e817a3a505dc24eb9
@ 0x7f0a2822e33c kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f0a28231fd2 kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260925 09:58:30.409855 14409 consensus_queue.cc:1035] T e370ffe0dec24f40a3b446ed4cdc831b P a06c3ef7a6a94f7cbcc03e7cd232f237 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:30.451653 14457 consensus_queue.cc:1035] T e370ffe0dec24f40a3b446ed4cdc831b P a06c3ef7a6a94f7cbcc03e7cd232f237 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:30.527891 14449 ts_tablet_manager.cc:927] T 0090f685d0184822b41fd3e731874567 P c4d724d4b83e4d1e817a3a505dc24eb9: Initiating tablet copy from peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
I20260925 09:58:30.534765 14449 tablet_copy_client.cc:316] T 0090f685d0184822b41fd3e731874567 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet copy: Beginning tablet copy session from remote peer at address 127.10.9.3:40731
I20260925 09:58:30.536600 13691 tablet_copy_service.cc:140] P a06c3ef7a6a94f7cbcc03e7cd232f237: Received BeginTabletCopySession request for tablet 0090f685d0184822b41fd3e731874567 from peer c4d724d4b83e4d1e817a3a505dc24eb9 ({username='slave'} at 127.10.9.2:45549)
I20260925 09:58:30.537127 13691 tablet_copy_service.cc:161] P a06c3ef7a6a94f7cbcc03e7cd232f237: Beginning new tablet copy session on tablet 0090f685d0184822b41fd3e731874567 from peer c4d724d4b83e4d1e817a3a505dc24eb9 at {username='slave'} at 127.10.9.2:45549: session id = c4d724d4b83e4d1e817a3a505dc24eb9-0090f685d0184822b41fd3e731874567
I20260925 09:58:30.546404 13691 tablet_copy_source_session.cc:205] T 0090f685d0184822b41fd3e731874567 P a06c3ef7a6a94f7cbcc03e7cd232f237: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 09:58:30.558931 14449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0090f685d0184822b41fd3e731874567. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 09:58:30.573961 14449 tablet_copy_client.cc:791] T 0090f685d0184822b41fd3e731874567 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet copy: Starting download of 0 data blocks...
I20260925 09:58:30.574563 14449 tablet_copy_client.cc:655] T 0090f685d0184822b41fd3e731874567 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet copy: Starting download of 1 WAL segments...
I20260925 09:58:30.578253 14449 tablet_copy_client.cc:531] T 0090f685d0184822b41fd3e731874567 P c4d724d4b83e4d1e817a3a505dc24eb9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 09:58:30.586498 14449 tablet_bootstrap.cc:492] T 0090f685d0184822b41fd3e731874567 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap starting.
I20260925 09:58:30.607081 14449 tablet_bootstrap.cc:492] T 0090f685d0184822b41fd3e731874567 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 09:58:30.607942 14449 tablet_bootstrap.cc:492] T 0090f685d0184822b41fd3e731874567 P c4d724d4b83e4d1e817a3a505dc24eb9: Bootstrap complete.
I20260925 09:58:30.608598 14449 ts_tablet_manager.cc:1397] T 0090f685d0184822b41fd3e731874567 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent bootstrapping tablet: real 0.022s user 0.020s sys 0.004s
@ 0x7f0a28231f8a _ZNSt3__18__invokeIRZN4kudu6master30HmsNotificationLogListenerTask4InitEvE3$_0JEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOS6_DpOS7_ at ??:0
I20260925 09:58:30.610931 14449 raft_consensus.cc:348] T 0090f685d0184822b41fd3e731874567 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:30.611897 14449 raft_consensus.cc:729] T 0090f685d0184822b41fd3e731874567 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d724d4b83e4d1e817a3a505dc24eb9, State: Initialized, Role: FOLLOWER
I20260925 09:58:30.612558 14449 consensus_queue.cc:260] T 0090f685d0184822b41fd3e731874567 P c4d724d4b83e4d1e817a3a505dc24eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:30.657608 14449 ts_tablet_manager.cc:1428] T 0090f685d0184822b41fd3e731874567 P c4d724d4b83e4d1e817a3a505dc24eb9: Time spent starting tablet: real 0.049s user 0.004s sys 0.000s
I20260925 09:58:30.663044 13691 tablet_copy_service.cc:342] P a06c3ef7a6a94f7cbcc03e7cd232f237: Request end of tablet copy session c4d724d4b83e4d1e817a3a505dc24eb9-0090f685d0184822b41fd3e731874567 received from {username='slave'} at 127.10.9.2:45549
I20260925 09:58:30.663465 13691 tablet_copy_service.cc:433] P a06c3ef7a6a94f7cbcc03e7cd232f237: ending tablet copy session c4d724d4b83e4d1e817a3a505dc24eb9-0090f685d0184822b41fd3e731874567 on tablet 0090f685d0184822b41fd3e731874567 with peer c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:58:30.692094 14467 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 70718e3f6f9f489089c0d8354380ed6c)
I20260925 09:58:30.692587 14467 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:30.694641 14467 leader_election.cc:288] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 70718e3f6f9f489089c0d8354380ed6c (127.10.9.62:43437), b7d0c1c2f1ed4e7a9ed54e73869d53b0 (127.10.9.61:40425)
I20260925 09:58:30.695209 13017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "61e29afab0cf46819cbffc91248bf761" candidate_term: 6 candidate_status { last_received { term: 5 index: 218 } } ignore_live_leader: false dest_uuid: "70718e3f6f9f489089c0d8354380ed6c" is_pre_election: true
I20260925 09:58:30.696236 13091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "61e29afab0cf46819cbffc91248bf761" candidate_term: 6 candidate_status { last_received { term: 5 index: 218 } } ignore_live_leader: false dest_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" is_pre_election: true
I20260925 09:58:30.696835 13091 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61e29afab0cf46819cbffc91248bf761 in term 5.
I20260925 09:58:30.697890 13955 leader_election.cc:302] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61e29afab0cf46819cbffc91248bf761, b7d0c1c2f1ed4e7a9ed54e73869d53b0; no voters: 70718e3f6f9f489089c0d8354380ed6c
I20260925 09:58:30.698796 14467 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260925 09:58:30.699132 14467 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 70718e3f6f9f489089c0d8354380ed6c)
I20260925 09:58:30.699489 14467 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 5 FOLLOWER]: Advancing to term 6
I20260925 09:58:30.709972 14467 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:30.715791 14467 leader_election.cc:288] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [CANDIDATE]: Term 6 election: Requested vote from peers 70718e3f6f9f489089c0d8354380ed6c (127.10.9.62:43437), b7d0c1c2f1ed4e7a9ed54e73869d53b0 (127.10.9.61:40425)
I20260925 09:58:30.716213 13017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "61e29afab0cf46819cbffc91248bf761" candidate_term: 6 candidate_status { last_received { term: 5 index: 218 } } ignore_live_leader: false dest_uuid: "70718e3f6f9f489089c0d8354380ed6c"
I20260925 09:58:30.717602 13091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "61e29afab0cf46819cbffc91248bf761" candidate_term: 6 candidate_status { last_received { term: 5 index: 218 } } ignore_live_leader: false dest_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0"
I20260925 09:58:30.718135 13091 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 5 FOLLOWER]: Advancing to term 6
I20260925 09:58:30.722802 14416 consensus_queue.cc:1035] T 733492d93592485e920628c47d5c4251 P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:30.721400 14038 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 6 FOLLOWER]: Rejecting Update request from peer 70718e3f6f9f489089c0d8354380ed6c for earlier term 5. Current term is 6. Ops: []
I20260925 09:58:30.731276 13091 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61e29afab0cf46819cbffc91248bf761 in term 6.
@ 0x7f0a28231f1a std::__1::__invoke_void_return_wrapper<>::__call<>() at ??:0
I20260925 09:58:30.732228 14468 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 }, Status: INVALID_TERM, Last received: 5.218, Next index: 219, Last known committed idx: 218, Time since last communication: 0.004s
I20260925 09:58:30.736243 13955 leader_election.cc:302] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61e29afab0cf46819cbffc91248bf761, b7d0c1c2f1ed4e7a9ed54e73869d53b0; no voters: 70718e3f6f9f489089c0d8354380ed6c
I20260925 09:58:30.737185 14467 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 6 FOLLOWER]: Leader election won for term 6
I20260925 09:58:30.751874 14468 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 5 LEADER]: Stepping down as leader of term 5
I20260925 09:58:30.752701 14468 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 70718e3f6f9f489089c0d8354380ed6c, State: Running, Role: LEADER
I20260925 09:58:30.757078 13091 raft_consensus.cc:1226] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 6 FOLLOWER]: Rejecting Update request from peer 70718e3f6f9f489089c0d8354380ed6c for earlier term 5. Current term is 6. Ops: []
I20260925 09:58:30.758141 14467 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [term 6 LEADER]: Becoming Leader. State: Replica: 61e29afab0cf46819cbffc91248bf761, State: Running, Role: LEADER
I20260925 09:58:30.759107 14467 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 218, Committed index: 218, Last appended: 5.218, Last appended by leader: 218, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:30.759429 14468 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 218, Committed index: 218, Last appended: 5.218, Last appended by leader: 218, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } }
I20260925 09:58:30.763221 14468 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [term 5 FOLLOWER]: Advancing to term 6
I20260925 09:58:30.764293 14472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61e29afab0cf46819cbffc91248bf761. Latest consensus state: current_term: 6 leader_uuid: "61e29afab0cf46819cbffc91248bf761" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } } }
I20260925 09:58:30.765053 14472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [sys.catalog]: This master's current role is: LEADER
W20260925 09:58:30.766414 14473 consensus_queue.cc:1159] T 00000000000000000000000000000000 P 70718e3f6f9f489089c0d8354380ed6c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" responder_term: 6 status { last_received { term: 5 index: 218 } last_committed_idx: 218 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 70718e3f6f9f489089c0d8354380ed6c for earlier term 5. Current term is 6. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260925 09:58:30.784158 14474 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 09:58:30.798889 14416 consensus_queue.cc:1035] T 733492d93592485e920628c47d5c4251 P c4d724d4b83e4d1e817a3a505dc24eb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
@ 0x7f0a28231ee2 std::__1::__function::__alloc_func<>::operator()() at ??:0
I20260925 09:58:30.848121 14474 catalog_manager.cc:669] Loaded metadata for table default.table_5f3286f12ae947ef8c1dfa71d8dd21d2 [id=07c6108c574e407f827bc4ab4ffdf345]
I20260925 09:58:30.850121 14474 catalog_manager.cc:669] Loaded metadata for table default.table_8f71132b5a004ad89115f7a2278b95d6 [id=0cdc8d8fef5445668c0b0107243e0ae9]
I20260925 09:58:30.858175 14474 catalog_manager.cc:669] Loaded metadata for table default.table_1238f9df428247dc9b9ca7c59823bfe4 [id=5eb4ef6bf2c344bdb26e429f484c59a4]
I20260925 09:58:30.874794 14474 catalog_manager.cc:669] Loaded metadata for table default.table_47c0ac37d06f4f45a701b0910a42c579 [id=714d733484104c508e7ced7cb14148bf]
I20260925 09:58:30.881387 14474 catalog_manager.cc:669] Loaded metadata for table default.table_41304460d5a345c88866a7429d816aca [id=77d343c035fa4fd2b3faadcfc4fec52f]
I20260925 09:58:30.883056 14474 catalog_manager.cc:669] Loaded metadata for table default.table_aa3ad1f6adf841ae9a2efad3eed2f203 [id=7c9c081163934b5197aaeedda0de597c]
@ 0x7f0a282311de std::__1::__function::__func<>::operator()() at ??:0
I20260925 09:58:30.888643 14474 catalog_manager.cc:669] Loaded metadata for table default.table_eda42a4c592f447b8084b26441b81b7c [id=7e0502ffa5544a5bb252a72020649cd7]
I20260925 09:58:30.899869 14474 catalog_manager.cc:669] Loaded metadata for table default.table_f83185097e7343848b564d3d5881c9ab [id=7f3f3dd4c3d74f21878caad38c9c25b9]
I20260925 09:58:30.901304 14474 catalog_manager.cc:669] Loaded metadata for table default.table_840c9ea5fcc04f1b8811754058184452 [id=84e4b25904ae485fbf3816bd06643b42]
I20260925 09:58:30.902802 14474 catalog_manager.cc:669] Loaded metadata for table default.table_76fed9ed8d204d969883e54e5c7330c7 [id=929b4a72ba9b4357929f4354783971d6]
@ 0x580785 std::__1::__function::__value_func<>::operator()() at ??:?
@ 0x5805ba std::__1::function<>::operator()() at ??:?
I20260925 09:58:30.916523 14474 catalog_manager.cc:669] Loaded metadata for table default.table_afc7cdc28ab84dff9a8782192aa34919 [id=a74c28d6faf4436385410132d67371eb]
I20260925 09:58:30.918304 14474 catalog_manager.cc:669] Loaded metadata for table default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477]
I20260925 09:58:30.920097 14474 catalog_manager.cc:669] Loaded metadata for table default.table_688d7c9730444646b6d7a901d5537807 [id=c06b5e37a4f14eb5b653ada7e7f73b4f]
W20260925 09:58:30.924736 13067 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 consensus state has no leader
I20260925 09:58:30.935227 14474 catalog_manager.cc:669] Loaded metadata for table default.table_ca5cce29b409479f9191e1b06eb5bac0 [id=db33e4ea5b1d4f0abedf9b9dde8614e2]
I20260925 09:58:30.937206 14474 catalog_manager.cc:669] Loaded metadata for table default.table_1310f782dc864b068b28842f01c96b9c [id=dc9c88f862a64597aab300baf32f6f76]
I20260925 09:58:30.938858 14474 catalog_manager.cc:669] Loaded metadata for table default.table_d4409ab5331140e9a4bb995d57be4ec4 [id=f196d9d3750b4857905c1332d3ac763e]
@ 0x7f0a1f344dc7 kudu::Thread::SuperviseThread() at ??:0
@ 0x4230f0 __tsan_thread_start_func at ??:?
I20260925 09:58:31.021476 14482 consensus_queue.cc:1035] T fe39e8e4c3ad47f393565dde2cded405 P b3082cf15dc74c0480e5d33ecf592843 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 09:58:31.023980 13505 raft_consensus.cc:1203] T fafc2b657ace4a4381452ce43fad39b6 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 0.0->[1.1-1.1] Dedup: 1.1->[]
I20260925 09:58:31.039193 14474 tablet_loader.cc:96] loaded metadata for tablet 0090f685d0184822b41fd3e731874567 (table default.table_76fed9ed8d204d969883e54e5c7330c7 [id=929b4a72ba9b4357929f4354783971d6])
W20260925 09:58:31.043740 13297 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.3:40731, user_credentials={real_user=slave}} blocked reactor thread for 50868.7us
@ 0x7f0a209b26db start_thread at ??:0
I20260925 09:58:31.062316 14474 tablet_loader.cc:96] loaded metadata for tablet 0e3f1d9381b4450cb6c15525699efbc9 (table default.table_afc7cdc28ab84dff9a8782192aa34919 [id=a74c28d6faf4436385410132d67371eb])
I20260925 09:58:31.071826 14474 tablet_loader.cc:96] loaded metadata for tablet 11f8c7b5918a431aa9c4975f28ea89cb (table default.table_eda42a4c592f447b8084b26441b81b7c [id=7e0502ffa5544a5bb252a72020649cd7])
I20260925 09:58:31.073259 14474 tablet_loader.cc:96] loaded metadata for tablet 3d18f12528cf40a49a7cd0c5c97001bf (table default.table_1238f9df428247dc9b9ca7c59823bfe4 [id=5eb4ef6bf2c344bdb26e429f484c59a4])
I20260925 09:58:31.074633 14474 tablet_loader.cc:96] loaded metadata for tablet 48ffc9cb12e346e7ae20c7f916ac33d4 (table default.table_8f71132b5a004ad89115f7a2278b95d6 [id=0cdc8d8fef5445668c0b0107243e0ae9])
I20260925 09:58:31.075858 14474 tablet_loader.cc:96] loaded metadata for tablet 4b61aaac33514c999482b824e0839a72 (table default.table_f83185097e7343848b564d3d5881c9ab [id=7f3f3dd4c3d74f21878caad38c9c25b9])
I20260925 09:58:31.076985 14474 tablet_loader.cc:96] loaded metadata for tablet 62d6d3750baa41c193c39ca39c4486a1 (table default.table_d4409ab5331140e9a4bb995d57be4ec4 [id=f196d9d3750b4857905c1332d3ac763e])
I20260925 09:58:31.078119 14474 tablet_loader.cc:96] loaded metadata for tablet 733492d93592485e920628c47d5c4251 (table default.table_aa3ad1f6adf841ae9a2efad3eed2f203 [id=7c9c081163934b5197aaeedda0de597c])
I20260925 09:58:31.079219 14474 tablet_loader.cc:96] loaded metadata for tablet 90d39744f6334e42bd32c26d13e6912c (table default.table_ca5cce29b409479f9191e1b06eb5bac0 [id=db33e4ea5b1d4f0abedf9b9dde8614e2])
I20260925 09:58:31.096843 14474 tablet_loader.cc:96] loaded metadata for tablet ace8260d4ac94cafbb9147ebeb01ed85 (table default.table_41304460d5a345c88866a7429d816aca [id=77d343c035fa4fd2b3faadcfc4fec52f])
I20260925 09:58:31.098371 14474 tablet_loader.cc:96] loaded metadata for tablet b9cf55829fed4c93a8eab92cee8920c9 (table default.table_840c9ea5fcc04f1b8811754058184452 [id=84e4b25904ae485fbf3816bd06643b42])
I20260925 09:58:31.099815 14474 tablet_loader.cc:96] loaded metadata for tablet d3efa619592045b8be65e6b789b8291a (table default.table_832367e542164f49b92f562d7a77699f [id=ac75711cba134e34919b4106c864f477])
I20260925 09:58:31.101264 14474 tablet_loader.cc:96] loaded metadata for tablet e370ffe0dec24f40a3b446ed4cdc831b (table default.table_5f3286f12ae947ef8c1dfa71d8dd21d2 [id=07c6108c574e407f827bc4ab4ffdf345])
I20260925 09:58:31.102593 14474 tablet_loader.cc:96] loaded metadata for tablet e505eea45c414e82a69d7d965de6c814 (table default.table_1310f782dc864b068b28842f01c96b9c [id=dc9c88f862a64597aab300baf32f6f76])
I20260925 09:58:31.113155 14269 consensus_queue.cc:1035] T fe39e8e4c3ad47f393565dde2cded405 P b3082cf15dc74c0480e5d33ecf592843 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
@ 0x7f0a1be3861f clone at ??:0
I20260925 09:58:31.128695 14474 tablet_loader.cc:96] loaded metadata for tablet fafc2b657ace4a4381452ce43fad39b6 (table default.table_47c0ac37d06f4f45a701b0910a42c579 [id=714d733484104c508e7ced7cb14148bf])
I20260925 09:58:31.129772 14474 tablet_loader.cc:96] loaded metadata for tablet fe39e8e4c3ad47f393565dde2cded405 (table default.table_688d7c9730444646b6d7a901d5537807 [id=c06b5e37a4f14eb5b653ada7e7f73b4f])
I20260925 09:58:31.136610 14474 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 09:58:31.182788 14474 catalog_manager.cc:1259] Loaded cluster ID: 85565ad29e6546888a40444865b055de
I20260925 09:58:31.183071 14474 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 09:58:31.222610 14474 catalog_manager.cc:1504] Loading token signing keys...
I20260925 09:58:31.248040 14474 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761: Loaded TSK: 0
I20260925 09:58:31.249598 14474 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 09:58:31.267242 14474 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260925 09:58:31.289634 14474 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: 35
I20260925 09:58:31.379660 13091 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [term 6 FOLLOWER]: Refusing update from remote peer 61e29afab0cf46819cbffc91248bf761: Log matching property violated. Preceding OpId in replica: term: 5 index: 218. Preceding OpId from leader: term: 6 index: 219. (index mismatch)
I20260925 09:58:31.381171 14472 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 219, Last known committed idx: 218, Time since last communication: 0.000s
W20260925 09:58:31.402809 13603 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.1:42161, user_credentials={real_user=slave}} blocked reactor thread for 45045.3us
I20260925 09:58:31.414811 13505 raft_consensus.cc:1203] T 0090f685d0184822b41fd3e731874567 P c4d724d4b83e4d1e817a3a505dc24eb9 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 0.0->[1.1-1.1] Dedup: 1.1->[]
I20260925 09:58:31.432231 14493 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61e29afab0cf46819cbffc91248bf761. Latest consensus state: current_term: 6 leader_uuid: "61e29afab0cf46819cbffc91248bf761" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } } }
I20260925 09:58:31.433080 14493 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 09:58:31.462998 14495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 6 leader_uuid: "61e29afab0cf46819cbffc91248bf761" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } } }
I20260925 09:58:31.464427 14495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 [sys.catalog]: This master's current role is: LEADER
I20260925 09:58:31.478379 14493 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "61e29afab0cf46819cbffc91248bf761" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70718e3f6f9f489089c0d8354380ed6c" member_type: VOTER last_known_addr { host: "127.10.9.62" port: 43437 } } peers { permanent_uuid: "b7d0c1c2f1ed4e7a9ed54e73869d53b0" member_type: VOTER last_known_addr { host: "127.10.9.61" port: 40425 } } peers { permanent_uuid: "61e29afab0cf46819cbffc91248bf761" member_type: VOTER last_known_addr { host: "127.10.9.60" port: 42083 } } }
I20260925 09:58:31.479123 14493 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7d0c1c2f1ed4e7a9ed54e73869d53b0 [sys.catalog]: This master's current role is: FOLLOWER
F20260925 09:58:31.482987 14219 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 36): 36 DROP_TABLE default.table_41304460d5a345c88866a7429d816aca
*** Check failure stack trace: ***
@ 0x7f467df4c9a0 google::LogMessage::Flush() at ??:0
@ 0x7f467df4c70f google::LogMessage::~LogMessage() at ??:0
@ 0x7f4688fe7aea kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20260925 09:58:31.682897 13716 heartbeater.cc:503] Master 127.10.9.60:42083 was elected leader, sending a full tablet report...
I20260925 09:58:31.687507 13550 heartbeater.cc:503] Master 127.10.9.60:42083 was elected leader, sending a full tablet report...
@ 0x7f4688fe733c kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
W20260925 09:58:31.714457 14015 catalog_manager.cc:5328] Ignoring report from unknown tablet b103b022a84040ea97247d6503002f47
@ 0x7f4688feafd2 kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
W20260925 09:58:31.836771 13606 connection.cc:535] client connection to 127.10.9.62:43437 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
@ 0x7f4688feaf8a _ZNSt3__18__invokeIRZN4kudu6master30HmsNotificationLogListenerTask4InitEvE3$_0JEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOS6_DpOS7_ at ??:0
W20260925 09:58:31.838402 13729 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.10.9.62:43437)
W20260925 09:58:31.845763 13441 connection.cc:535] client connection to 127.10.9.62:43437 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 09:58:31.847201 13957 connection.cc:535] client connection to 127.10.9.62:43437 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 09:58:31.847664 13957 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 09:58:31.849463 13298 connection.cc:535] client connection to 127.10.9.62:43437 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 09:58:31.873991 14505 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:58:31.839666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.9.62:43437 (local address 127.10.9.3:36461)
0925 09:58:31.872831 (+ 33165us) negotiation.cc:107] Waiting for socket to connect
0925 09:58:31.873090 (+ 259us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.9.62:43437: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":32546,"threads_started":1}
W20260925 09:58:31.887567 14508 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:58:31.839801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.9.62:43437 (local address 127.0.0.1:47830)
0925 09:58:31.866250 (+ 26449us) negotiation.cc:107] Waiting for socket to connect
0925 09:58:31.866733 (+ 483us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.9.62:43437: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":25809,"thread_start_us":24048,"threads_started":1}
@ 0x7f4688feaf1a std::__1::__invoke_void_return_wrapper<>::__call<>() at ??:0
W20260925 09:58:31.908872 13957 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.9.62:43437, user_credentials={real_user=slave}} blocked reactor thread for 61251.4us
@ 0x7f4688feaee2 std::__1::__function::__alloc_func<>::operator()() at ??:0
W20260925 09:58:31.953146 13957 consensus_peers.cc:487] T 00000000000000000000000000000000 P 61e29afab0cf46819cbffc91248bf761 -> Peer 70718e3f6f9f489089c0d8354380ed6c (127.10.9.62:43437): Couldn't send request to peer 70718e3f6f9f489089c0d8354380ed6c. Status: Network error: Client connection negotiation failed: client connection to 127.10.9.62:43437: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 09:58:32.006564 13409 heartbeater.cc:503] Master 127.10.9.60:42083 was elected leader, sending a full tablet report...
@ 0x7f4688fea1de std::__1::__function::__func<>::operator()() at ??:0
@ 0x580785 std::__1::__function::__value_func<>::operator()() at ??:?
@ 0x5805ba std::__1::function<>::operator()() at ??:?
W20260925 09:58:32.022459 14006 catalog_manager.cc:5328] Ignoring report from unknown tablet af6ace3011c84603b32d0a409d5b8c1c
@ 0x7f46800fddc7 kudu::Thread::SuperviseThread() at ??:0
@ 0x4230f0 __tsan_thread_start_func at ??:?
@ 0x7f468176b6db start_thread at ??:0
@ 0x7f467cbf161f clone at ??:0
W20260925 09:58:32.373324 13604 connection.cc:535] client connection to 127.10.9.60:42083 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 09:58:32.374395 13439 connection.cc:535] client connection to 127.10.9.60:42083 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 09:58:32.378926 14508 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 09:58:32.377183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.9.60:42083 (local address 127.0.0.1:42268)
0925 09:58:32.377738 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0925 09:58:32.377989 (+ 251us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.9.60:42083: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":78}
I20260925 09:58:33.085327 14492 raft_consensus.cc:482] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:33.085865 14492 raft_consensus.cc:504] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:33.087894 14492 leader_election.cc:288] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
W20260925 09:58:33.089370 13603 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
W20260925 09:58:33.089823 13606 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
I20260925 09:58:33.090173 13606 leader_election.cc:302] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237; no voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:58:33.090837 14492 raft_consensus.cc:2726] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:58:33.158592 13066 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet a3562966c8c34046af271a7c617cdb6b (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet a3562966c8c34046af271a7c617cdb6b consensus state has no leader
W20260925 09:58:35.025512 13067 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 consensus state has no leader
I20260925 09:58:36.510289 14512 raft_consensus.cc:482] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:36.511070 14512 raft_consensus.cc:504] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:36.513470 14512 leader_election.cc:288] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
W20260925 09:58:36.514940 13298 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
W20260925 09:58:36.515595 13297 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
I20260925 09:58:36.515877 13297 leader_election.cc:302] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:58:36.516564 14512 raft_consensus.cc:2726] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:58:41.370853 13066 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet a3562966c8c34046af271a7c617cdb6b (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet a3562966c8c34046af271a7c617cdb6b consensus state has no leader
I20260925 09:58:43.116802 14559 raft_consensus.cc:482] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:43.117231 14559 raft_consensus.cc:504] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:43.119210 14559 leader_election.cc:288] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
W20260925 09:58:43.120757 13298 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
W20260925 09:58:43.121304 13297 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
I20260925 09:58:43.121661 13297 leader_election.cc:302] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:58:43.122316 14559 raft_consensus.cc:2726] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:58:43.238520 13067 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 consensus state has no leader
I20260925 09:58:46.834712 14559 raft_consensus.cc:482] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:46.835348 14559 raft_consensus.cc:504] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:46.838121 14559 leader_election.cc:288] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
W20260925 09:58:46.839520 13298 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
W20260925 09:58:46.840083 13297 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
I20260925 09:58:46.840361 13297 leader_election.cc:302] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:58:46.840939 14559 raft_consensus.cc:2726] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:58:46.903553 14570 raft_consensus.cc:482] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:46.904184 14570 raft_consensus.cc:504] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:46.906998 14570 leader_election.cc:288] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
W20260925 09:58:46.908546 13603 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
W20260925 09:58:46.909132 13606 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
I20260925 09:58:46.909441 13606 leader_election.cc:302] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237; no voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:58:46.910029 14570 raft_consensus.cc:2726] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:58:54.996443 14559 raft_consensus.cc:482] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:58:54.996918 14559 raft_consensus.cc:504] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:58:54.998286 14559 leader_election.cc:288] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
W20260925 09:58:55.000164 13298 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
W20260925 09:58:55.000627 13297 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
I20260925 09:58:55.000978 13297 leader_election.cc:302] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:58:55.001591 14559 raft_consensus.cc:2726] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:58:57.774893 13066 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet a3562966c8c34046af271a7c617cdb6b (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet a3562966c8c34046af271a7c617cdb6b consensus state has no leader
W20260925 09:58:59.631935 13067 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 consensus state has no leader
I20260925 09:59:01.666950 14660 raft_consensus.cc:482] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:59:01.667768 14660 raft_consensus.cc:504] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:59:01.669660 14660 leader_election.cc:288] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
W20260925 09:59:01.671442 13603 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
W20260925 09:59:01.671983 13606 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
I20260925 09:59:01.672287 13606 leader_election.cc:302] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237; no voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:59:01.672884 14660 raft_consensus.cc:2726] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:59:05.207069 13546 debug-util.cc:398] Leaking SignalData structure 0x7b08000f78c0 after lost signal to thread 13421
W20260925 09:59:05.208381 13546 debug-util.cc:398] Leaking SignalData structure 0x7b08000f5ec0 after lost signal to thread 13549
I20260925 09:59:05.918593 14660 raft_consensus.cc:482] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:59:05.919090 14660 raft_consensus.cc:504] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:59:05.921458 14660 leader_election.cc:288] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
W20260925 09:59:05.924263 13606 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
W20260925 09:59:05.924717 13603 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
I20260925 09:59:05.925040 13603 leader_election.cc:302] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237; no voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:59:05.926108 14696 raft_consensus.cc:2726] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:59:10.572839 13405 debug-util.cc:398] Leaking SignalData structure 0x7b08000f1200 after lost signal to thread 13274
W20260925 09:59:10.573992 13405 debug-util.cc:398] Leaking SignalData structure 0x7b0800122bc0 after lost signal to thread 13408
I20260925 09:59:11.009668 14660 raft_consensus.cc:482] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:59:11.010191 14660 raft_consensus.cc:504] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:59:11.012358 14660 leader_election.cc:288] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
W20260925 09:59:11.015605 13603 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
W20260925 09:59:11.016528 13606 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
I20260925 09:59:11.016909 13606 leader_election.cc:302] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237; no voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:59:11.017657 14660 raft_consensus.cc:2726] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:59:11.953076 14635 raft_consensus.cc:482] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:59:11.953826 14635 raft_consensus.cc:504] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
W20260925 09:59:11.957855 13298 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
I20260925 09:59:11.957922 14635 leader_election.cc:288] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
W20260925 09:59:11.959643 13297 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
I20260925 09:59:11.959970 13297 leader_election.cc:302] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:59:11.960561 14635 raft_consensus.cc:2726] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:59:12.351282 13546 debug-util.cc:398] Leaking SignalData structure 0x7b080010c1e0 after lost signal to thread 13421
W20260925 09:59:12.352351 13546 debug-util.cc:398] Leaking SignalData structure 0x7b08000e5a40 after lost signal to thread 13549
W20260925 09:59:12.352603 13546 debug-util.cc:398] Leaking SignalData structure 0x7b0800041160 after lost signal to thread 13551
W20260925 09:59:13.130386 13552 heartbeater.cc:644] Failed to heartbeat to 127.10.9.62:43437 (83 consecutive failures): Network error: Failed to ping master at 127.10.9.62:43437: Client connection negotiation failed: client connection to 127.10.9.62:43437: connect: Connection refused (error 111)
W20260925 09:59:13.342154 13409 heartbeater.cc:644] Failed to heartbeat to 127.10.9.60:42083 (82 consecutive failures): Network error: Failed to ping master at 127.10.9.60:42083: Client connection negotiation failed: client connection to 127.10.9.60:42083: connect: Connection refused (error 111)
W20260925 09:59:13.596114 13718 heartbeater.cc:644] Failed to heartbeat to 127.10.9.62:43437 (84 consecutive failures): Network error: Failed to ping master at 127.10.9.62:43437: Client connection negotiation failed: client connection to 127.10.9.62:43437: connect: Connection refused (error 111)
W20260925 09:59:17.037642 13113 debug-util.cc:398] Leaking SignalData structure 0x7b080004e420 after lost signal to thread 13047
W20260925 09:59:17.038414 13113 debug-util.cc:398] Leaking SignalData structure 0x7b08000b4860 after lost signal to thread 13116
I20260925 09:59:21.702909 14724 raft_consensus.cc:482] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:59:21.703739 14724 raft_consensus.cc:504] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:59:21.706933 14724 leader_election.cc:288] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
W20260925 09:59:21.708547 13603 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
W20260925 09:59:21.709131 13606 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
I20260925 09:59:21.709422 13606 leader_election.cc:302] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237; no voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:59:21.710146 14724 raft_consensus.cc:2726] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:59:24.150554 13728 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.10.9.60:42083)
I20260925 09:59:28.572898 14720 raft_consensus.cc:482] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:59:28.573460 14720 raft_consensus.cc:504] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:59:28.576076 14720 leader_election.cc:288] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
W20260925 09:59:28.577576 13298 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
W20260925 09:59:28.578153 13297 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
I20260925 09:59:28.578465 13297 leader_election.cc:302] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:59:28.578994 14720 raft_consensus.cc:2726] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:59:30.559851 13066 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet a3562966c8c34046af271a7c617cdb6b (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet a3562966c8c34046af271a7c617cdb6b consensus state has no leader
I20260925 09:59:30.625352 14724 raft_consensus.cc:482] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:59:30.625814 14724 raft_consensus.cc:504] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:59:30.627830 14724 leader_election.cc:288] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
W20260925 09:59:30.629258 13603 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
W20260925 09:59:30.629660 13606 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
I20260925 09:59:30.630000 13606 leader_election.cc:302] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237; no voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:59:30.630581 14724 raft_consensus.cc:2726] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:59:32.441592 13067 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 consensus state has no leader
I20260925 09:59:42.516644 14833 raft_consensus.cc:482] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:59:42.517251 14833 raft_consensus.cc:504] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:59:42.519349 14833 leader_election.cc:288] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
W20260925 09:59:42.521387 13297 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
W20260925 09:59:42.521914 13298 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
I20260925 09:59:42.522243 13298 leader_election.cc:302] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:59:42.522923 14833 raft_consensus.cc:2726] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 09:59:43.067502 13712 debug-util.cc:398] Leaking SignalData structure 0x7b0800112cc0 after lost signal to thread 13569
W20260925 09:59:43.068516 13712 debug-util.cc:398] Leaking SignalData structure 0x7b080006d060 after lost signal to thread 13715
I20260925 09:59:48.767347 14888 raft_consensus.cc:482] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:59:48.767954 14888 raft_consensus.cc:504] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:59:48.770886 14888 leader_election.cc:288] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
W20260925 09:59:48.772183 13606 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
W20260925 09:59:48.772693 13603 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
I20260925 09:59:48.773031 13603 leader_election.cc:302] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237; no voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:59:48.773689 14888 raft_consensus.cc:2726] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:59:52.438381 14888 raft_consensus.cc:482] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:59:52.438947 14888 raft_consensus.cc:504] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:59:52.441298 14888 leader_election.cc:288] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
W20260925 09:59:52.443192 13606 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
W20260925 09:59:52.443889 13603 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
I20260925 09:59:52.444180 13603 leader_election.cc:302] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237; no voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:59:52.444761 14888 raft_consensus.cc:2726] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 09:59:58.981810 14934 raft_consensus.cc:482] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 09:59:58.982439 14934 raft_consensus.cc:504] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 09:59:58.985666 14934 leader_election.cc:288] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
W20260925 09:59:58.986876 13298 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
W20260925 09:59:58.987515 13297 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
I20260925 09:59:58.987845 13297 leader_election.cc:302] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 09:59:58.988514 14934 raft_consensus.cc:2726] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 10:00:00.374006 14888 raft_consensus.cc:482] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:00:00.374459 14888 raft_consensus.cc:504] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 10:00:00.376138 14888 leader_election.cc:288] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
W20260925 10:00:00.378400 13603 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
W20260925 10:00:00.379136 13606 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
I20260925 10:00:00.379453 13606 leader_election.cc:302] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237; no voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 10:00:00.380038 14888 raft_consensus.cc:2726] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 10:00:04.699602 13071 maintenance_manager.cc:419] P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033358
I20260925 10:00:04.797222 13070 maintenance_manager.cc:643] P b7d0c1c2f1ed4e7a9ed54e73869d53b0: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.096s user 0.077s sys 0.017s Metrics: {"bytes_written":38337,"cfile_init":1,"drs_written":1,"lbm_read_time_us":300,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2577,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"rows_written":101,"thread_start_us":588,"threads_started":1,"wal-append.queue_time_us":963}
I20260925 10:00:04.799382 13071 maintenance_manager.cc:419] P b7d0c1c2f1ed4e7a9ed54e73869d53b0: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7420 bytes on disk
I20260925 10:00:04.801384 13070 maintenance_manager.cc:643] P b7d0c1c2f1ed4e7a9ed54e73869d53b0: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":274,"lbm_reads_lt_1ms":4}
W20260925 10:00:12.480813 13405 debug-util.cc:398] Leaking SignalData structure 0x7b080011f380 after lost signal to thread 13274
W20260925 10:00:12.481976 13405 debug-util.cc:398] Leaking SignalData structure 0x7b08000b3cc0 after lost signal to thread 13408
W20260925 10:00:12.985661 13712 debug-util.cc:398] Leaking SignalData structure 0x7b0800123520 after lost signal to thread 13569
W20260925 10:00:12.986579 13712 debug-util.cc:398] Leaking SignalData structure 0x7b08000d6c40 after lost signal to thread 13715
W20260925 10:00:13.541062 13550 heartbeater.cc:644] Failed to heartbeat to 127.10.9.60:42083 (202 consecutive failures): Network error: Failed to ping master at 127.10.9.60:42083: Client connection negotiation failed: client connection to 127.10.9.60:42083: connect: Connection refused (error 111)
W20260925 10:00:13.683689 13411 heartbeater.cc:644] Failed to heartbeat to 127.10.9.62:43437 (204 consecutive failures): Network error: Failed to ping master at 127.10.9.62:43437: Client connection negotiation failed: client connection to 127.10.9.62:43437: connect: Connection refused (error 111)
W20260925 10:00:14.009335 13718 heartbeater.cc:644] Failed to heartbeat to 127.10.9.62:43437 (204 consecutive failures): Network error: Failed to ping master at 127.10.9.62:43437: Client connection negotiation failed: client connection to 127.10.9.62:43437: connect: Connection refused (error 111)
I20260925 10:00:15.912401 15068 raft_consensus.cc:482] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:00:15.912992 15068 raft_consensus.cc:504] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 10:00:15.915676 15068 leader_election.cc:288] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
W20260925 10:00:15.917258 13298 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
W20260925 10:00:15.917987 13297 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
I20260925 10:00:15.918334 13297 leader_election.cc:302] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 10:00:15.918906 15068 raft_consensus.cc:2726] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 10:00:17.003371 15065 raft_consensus.cc:482] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:00:17.004066 15065 raft_consensus.cc:504] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 10:00:17.007629 15065 leader_election.cc:288] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
W20260925 10:00:17.008884 13603 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
W20260925 10:00:17.009464 13606 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
I20260925 10:00:17.009850 13606 leader_election.cc:302] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237; no voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 10:00:17.010524 15065 raft_consensus.cc:2726] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 10:00:19.068176 15088 raft_consensus.cc:482] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:00:19.068753 15088 raft_consensus.cc:504] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 10:00:19.071077 15088 leader_election.cc:288] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
W20260925 10:00:19.072404 13297 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
W20260925 10:00:19.074026 13298 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
I20260925 10:00:19.074298 13298 leader_election.cc:302] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 10:00:19.074899 15088 raft_consensus.cc:2726] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 10:00:19.986598 13405 debug-util.cc:398] Leaking SignalData structure 0x7b08000c38e0 after lost signal to thread 13274
W20260925 10:00:19.987787 13405 debug-util.cc:398] Leaking SignalData structure 0x7b0800103dc0 after lost signal to thread 13408
W20260925 10:00:20.184785 13546 debug-util.cc:398] Leaking SignalData structure 0x7b08000e0b60 after lost signal to thread 13421
W20260925 10:00:20.186172 13546 debug-util.cc:398] Leaking SignalData structure 0x7b08000e30e0 after lost signal to thread 13549
I20260925 10:00:22.110740 15085 raft_consensus.cc:482] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:00:22.111616 15085 raft_consensus.cc:504] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
W20260925 10:00:22.116571 13603 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
W20260925 10:00:22.117720 13606 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
I20260925 10:00:22.118096 13606 leader_election.cc:302] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237; no voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 10:00:22.118999 15085 leader_election.cc:288] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
I20260925 10:00:22.119551 15085 raft_consensus.cc:2726] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 10:00:29.414044 15160 raft_consensus.cc:482] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:00:29.414676 15160 raft_consensus.cc:504] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 10:00:29.416857 15160 leader_election.cc:288] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
W20260925 10:00:29.418351 13298 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
W20260925 10:00:29.418877 13297 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
I20260925 10:00:29.419168 13297 leader_election.cc:302] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 10:00:29.419744 15160 raft_consensus.cc:2726] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 10:00:30.610401 13066 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet a3562966c8c34046af271a7c617cdb6b (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet a3562966c8c34046af271a7c617cdb6b consensus state has no leader
W20260925 10:00:32.489283 13067 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 consensus state has no leader
I20260925 10:00:32.583060 15178 raft_consensus.cc:482] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:00:32.583632 15178 raft_consensus.cc:504] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 10:00:32.586462 15178 leader_election.cc:288] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
W20260925 10:00:32.587841 13603 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
W20260925 10:00:32.588428 13606 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
I20260925 10:00:32.588714 13606 leader_election.cc:302] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237; no voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 10:00:32.589320 15178 raft_consensus.cc:2726] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 10:00:43.939343 15277 raft_consensus.cc:482] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:00:43.940017 15277 raft_consensus.cc:504] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 10:00:43.942929 15277 leader_election.cc:288] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
W20260925 10:00:43.944238 13603 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
W20260925 10:00:43.945185 13606 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
I20260925 10:00:43.945541 13606 leader_election.cc:302] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237; no voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 10:00:43.946144 15277 raft_consensus.cc:2726] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 10:00:47.483597 15254 raft_consensus.cc:482] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:00:47.484198 15254 raft_consensus.cc:504] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 10:00:47.486647 15254 leader_election.cc:288] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
W20260925 10:00:47.488143 13298 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
W20260925 10:00:47.488829 13297 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
I20260925 10:00:47.489202 13297 leader_election.cc:302] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 10:00:47.489914 15254 raft_consensus.cc:2726] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 10:00:50.790612 13113 debug-util.cc:398] Leaking SignalData structure 0x7b080007a3c0 after lost signal to thread 13047
W20260925 10:00:50.791213 13113 debug-util.cc:398] Leaking SignalData structure 0x7b0800093fc0 after lost signal to thread 13116
I20260925 10:00:55.426980 15254 raft_consensus.cc:482] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:00:55.427371 15254 raft_consensus.cc:504] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 10:00:55.428825 15254 leader_election.cc:288] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
W20260925 10:00:55.430404 13298 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
W20260925 10:00:55.430893 13297 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
I20260925 10:00:55.431185 13297 leader_election.cc:302] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 10:00:55.431725 15254 raft_consensus.cc:2726] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 10:00:55.882258 13405 debug-util.cc:398] Leaking SignalData structure 0x7b08000b4220 after lost signal to thread 13274
W20260925 10:00:55.883240 13405 debug-util.cc:398] Leaking SignalData structure 0x7b080010f280 after lost signal to thread 13408
W20260925 10:00:56.975976 13712 debug-util.cc:398] Leaking SignalData structure 0x7b08000c60c0 after lost signal to thread 13569
W20260925 10:00:56.977393 13712 debug-util.cc:398] Leaking SignalData structure 0x7b0800115760 after lost signal to thread 13715
I20260925 10:01:02.934121 15353 raft_consensus.cc:482] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:01:02.934788 15353 raft_consensus.cc:504] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 10:01:02.937548 15353 leader_election.cc:288] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
W20260925 10:01:02.939163 13603 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
W20260925 10:01:02.939749 13606 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
I20260925 10:01:02.940021 13606 leader_election.cc:302] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237; no voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 10:01:02.940555 15353 raft_consensus.cc:2726] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 10:01:03.644333 15458 raft_consensus.cc:482] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:01:03.644995 15458 raft_consensus.cc:504] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 10:01:03.647301 15458 leader_election.cc:288] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
W20260925 10:01:03.648514 13298 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
W20260925 10:01:03.649081 13297 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
I20260925 10:01:03.649413 13297 leader_election.cc:302] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 10:01:03.649971 15458 raft_consensus.cc:2726] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 10:01:05.598100 13405 debug-util.cc:398] Leaking SignalData structure 0x7b08000fc020 after lost signal to thread 13274
W20260925 10:01:05.598912 13405 debug-util.cc:398] Leaking SignalData structure 0x7b08000e7ac0 after lost signal to thread 13408
W20260925 10:01:07.412642 13546 debug-util.cc:398] Leaking SignalData structure 0x7b0800084720 after lost signal to thread 13421
W20260925 10:01:07.413872 13546 debug-util.cc:398] Leaking SignalData structure 0x7b080011c280 after lost signal to thread 13549
W20260925 10:01:13.889499 13552 heartbeater.cc:644] Failed to heartbeat to 127.10.9.62:43437 (323 consecutive failures): Network error: Failed to ping master at 127.10.9.62:43437: Client connection negotiation failed: client connection to 127.10.9.62:43437: connect: Connection refused (error 111)
W20260925 10:01:14.086383 13411 heartbeater.cc:644] Failed to heartbeat to 127.10.9.62:43437 (324 consecutive failures): Network error: Failed to ping master at 127.10.9.62:43437: Client connection negotiation failed: client connection to 127.10.9.62:43437: connect: Connection refused (error 111)
W20260925 10:01:14.363745 13718 heartbeater.cc:644] Failed to heartbeat to 127.10.9.62:43437 (324 consecutive failures): Network error: Failed to ping master at 127.10.9.62:43437: Client connection negotiation failed: client connection to 127.10.9.62:43437: connect: Connection refused (error 111)
I20260925 10:01:14.666049 15509 raft_consensus.cc:482] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:01:14.666689 15509 raft_consensus.cc:504] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 10:01:14.668915 15509 leader_election.cc:288] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
W20260925 10:01:14.670441 13298 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
W20260925 10:01:14.671108 13297 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
I20260925 10:01:14.671523 13297 leader_election.cc:302] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 10:01:14.672362 15509 raft_consensus.cc:2726] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 10:01:18.626385 15552 raft_consensus.cc:482] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:01:18.626992 15552 raft_consensus.cc:504] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 10:01:18.629251 15552 leader_election.cc:288] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
W20260925 10:01:18.631096 13606 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
W20260925 10:01:18.631637 13603 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
I20260925 10:01:18.631899 13603 leader_election.cc:302] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237; no voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 10:01:18.632486 15552 raft_consensus.cc:2726] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 10:01:18.665450 15599 raft_consensus.cc:482] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:01:18.666028 15599 raft_consensus.cc:504] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 10:01:18.668329 15599 leader_election.cc:288] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
W20260925 10:01:18.669795 13298 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
W20260925 10:01:18.670307 13297 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
I20260925 10:01:18.670563 13297 leader_election.cc:302] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 10:01:18.671216 15599 raft_consensus.cc:2726] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 10:01:26.414800 15552 raft_consensus.cc:482] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:01:26.415504 15552 raft_consensus.cc:504] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 10:01:26.417078 15552 leader_election.cc:288] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
W20260925 10:01:26.418809 13603 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
W20260925 10:01:26.419349 13606 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
I20260925 10:01:26.419759 13606 leader_election.cc:302] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237; no voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 10:01:26.420408 15552 raft_consensus.cc:2726] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 10:01:30.635262 13066 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet a3562966c8c34046af271a7c617cdb6b (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet a3562966c8c34046af271a7c617cdb6b consensus state has no leader
I20260925 10:01:31.203009 15749 raft_consensus.cc:482] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:01:31.203680 15749 raft_consensus.cc:504] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 10:01:31.206562 15749 leader_election.cc:288] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
W20260925 10:01:31.207563 13298 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
W20260925 10:01:31.208047 13297 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
I20260925 10:01:31.208326 13297 leader_election.cc:302] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 10:01:31.208909 15749 raft_consensus.cc:2726] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 10:01:32.509059 13067 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 consensus state has no leader
I20260925 10:01:36.431226 15749 raft_consensus.cc:482] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:01:36.431743 15749 raft_consensus.cc:504] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 10:01:36.433954 15749 leader_election.cc:288] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
W20260925 10:01:36.435739 13298 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
W20260925 10:01:36.436717 13297 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
I20260925 10:01:36.437053 13297 leader_election.cc:302] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 10:01:36.437639 15749 raft_consensus.cc:2726] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 10:01:39.365865 15748 raft_consensus.cc:482] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:01:39.366474 15748 raft_consensus.cc:504] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 10:01:39.368778 15748 leader_election.cc:288] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
W20260925 10:01:39.370572 13606 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
W20260925 10:01:39.371079 13603 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
I20260925 10:01:39.371454 13603 leader_election.cc:302] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237; no voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 10:01:39.372011 15748 raft_consensus.cc:2726] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 10:01:39.978484 15749 raft_consensus.cc:482] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:01:39.979220 15749 raft_consensus.cc:504] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 10:01:39.981132 15749 leader_election.cc:288] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
W20260925 10:01:39.982929 13297 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
W20260925 10:01:39.983530 13298 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
I20260925 10:01:39.983784 13298 leader_election.cc:302] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 10:01:39.984268 15749 raft_consensus.cc:2726] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 10:01:46.457858 13546 debug-util.cc:398] Leaking SignalData structure 0x7b0800113480 after lost signal to thread 13421
W20260925 10:01:46.458900 13546 debug-util.cc:398] Leaking SignalData structure 0x7b08000feba0 after lost signal to thread 13549
W20260925 10:01:47.588824 13113 debug-util.cc:398] Leaking SignalData structure 0x7b0800051440 after lost signal to thread 13047
W20260925 10:01:47.589527 13113 debug-util.cc:398] Leaking SignalData structure 0x7b08000a09a0 after lost signal to thread 13116
I20260925 10:01:47.704629 15749 raft_consensus.cc:482] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:01:47.705235 15749 raft_consensus.cc:504] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 10:01:47.707012 15749 leader_election.cc:288] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
W20260925 10:01:47.708386 13298 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
W20260925 10:01:47.709021 13297 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
I20260925 10:01:47.709298 13297 leader_election.cc:302] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 10:01:47.709949 15749 raft_consensus.cc:2726] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 10:01:54.171226 15914 raft_consensus.cc:482] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:01:54.171742 15914 raft_consensus.cc:504] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 10:01:54.174206 15914 leader_election.cc:288] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
W20260925 10:01:54.175843 13298 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
W20260925 10:01:54.176563 13297 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
I20260925 10:01:54.176887 13297 leader_election.cc:302] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 10:01:54.177510 15914 raft_consensus.cc:2726] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 10:01:54.775559 15948 raft_consensus.cc:482] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:01:54.776202 15948 raft_consensus.cc:504] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 10:01:54.778745 15948 leader_election.cc:288] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
W20260925 10:01:54.780465 13603 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
W20260925 10:01:54.781246 13606 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
I20260925 10:01:54.781610 13606 leader_election.cc:302] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237; no voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 10:01:54.782294 15948 raft_consensus.cc:2726] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 10:01:57.567276 13712 debug-util.cc:398] Leaking SignalData structure 0x7b0800106e20 after lost signal to thread 13569
W20260925 10:01:57.568234 13712 debug-util.cc:398] Leaking SignalData structure 0x7b0800041ca0 after lost signal to thread 13715
I20260925 10:02:03.022938 16080 raft_consensus.cc:482] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:02:03.023692 16080 raft_consensus.cc:504] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 10:02:03.025528 16080 leader_election.cc:288] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
W20260925 10:02:03.027012 13298 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
W20260925 10:02:03.027597 13297 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
I20260925 10:02:03.027935 13297 leader_election.cc:302] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 10:02:03.028558 16080 raft_consensus.cc:2726] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 10:02:10.308977 16135 raft_consensus.cc:482] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:02:10.309722 16135 raft_consensus.cc:504] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 10:02:10.312147 16135 leader_election.cc:288] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
W20260925 10:02:10.314195 13606 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
W20260925 10:02:10.314775 13603 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
I20260925 10:02:10.315136 13603 leader_election.cc:302] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237; no voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 10:02:10.315853 16135 raft_consensus.cc:2726] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 10:02:14.154865 13551 heartbeater.cc:644] Failed to heartbeat to 127.10.9.61:40425 (153 consecutive failures): Timed out: Failed to ping master at 127.10.9.61:40425: Ping RPC to 127.10.9.61:40425 timed out after 1.000s (SENT)
I20260925 10:02:14.291162 16126 raft_consensus.cc:482] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:02:14.291759 16126 raft_consensus.cc:504] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 10:02:14.294222 16126 leader_election.cc:288] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
W20260925 10:02:14.295768 13298 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
W20260925 10:02:14.296597 13297 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
I20260925 10:02:14.296847 13297 leader_election.cc:302] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 10:02:14.297506 16126 raft_consensus.cc:2726] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 10:02:14.409813 13410 heartbeater.cc:644] Failed to heartbeat to 127.10.9.61:40425 (153 consecutive failures): Timed out: Failed to ping master at 127.10.9.61:40425: Ping RPC to 127.10.9.61:40425 timed out after 1.000s (SENT)
W20260925 10:02:14.698004 13716 heartbeater.cc:644] Failed to heartbeat to 127.10.9.60:42083 (443 consecutive failures): Network error: Failed to ping master at 127.10.9.60:42083: Client connection negotiation failed: client connection to 127.10.9.60:42083: connect: Connection refused (error 111)
I20260925 10:02:15.114126 16135 raft_consensus.cc:482] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:02:15.114813 16135 raft_consensus.cc:504] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 10:02:15.117132 16135 leader_election.cc:288] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
W20260925 10:02:15.118682 13606 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
W20260925 10:02:15.119138 13603 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
I20260925 10:02:15.119393 13603 leader_election.cc:302] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237; no voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 10:02:15.119961 16135 raft_consensus.cc:2726] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 10:02:22.896842 13113 debug-util.cc:398] Leaking SignalData structure 0x7b080004aa20 after lost signal to thread 13047
W20260925 10:02:22.897464 13113 debug-util.cc:398] Leaking SignalData structure 0x7b08000b70a0 after lost signal to thread 13116
W20260925 10:02:28.362221 13546 debug-util.cc:398] Leaking SignalData structure 0x7b080010db00 after lost signal to thread 13421
W20260925 10:02:28.363268 13546 debug-util.cc:398] Leaking SignalData structure 0x7b080011c960 after lost signal to thread 13549
I20260925 10:02:28.775250 16205 raft_consensus.cc:482] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:02:28.776072 16205 raft_consensus.cc:504] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 10:02:28.778373 16205 leader_election.cc:288] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
W20260925 10:02:28.780443 13298 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
W20260925 10:02:28.780992 13297 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
I20260925 10:02:28.781332 13297 leader_election.cc:302] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 10:02:28.782016 16205 raft_consensus.cc:2726] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 10:02:29.829993 13546 debug-util.cc:398] Leaking SignalData structure 0x7b080010d4a0 after lost signal to thread 13421
W20260925 10:02:29.831120 13546 debug-util.cc:398] Leaking SignalData structure 0x7b08000af860 after lost signal to thread 13549
W20260925 10:02:30.682173 13066 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet a3562966c8c34046af271a7c617cdb6b (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet a3562966c8c34046af271a7c617cdb6b consensus state has no leader
I20260925 10:02:32.253508 16237 raft_consensus.cc:482] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:02:32.254352 16237 raft_consensus.cc:504] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 10:02:32.257592 16237 leader_election.cc:288] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
W20260925 10:02:32.259181 13603 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
W20260925 10:02:32.259775 13606 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
I20260925 10:02:32.260071 13606 leader_election.cc:302] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237; no voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 10:02:32.260708 16237 raft_consensus.cc:2726] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 10:02:32.556809 13067 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 (table default.table_e59da12e7b95405787a2f9f73efdc76b [id=7e0502ffa5544a5bb252a72020649cd7], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 3970efec7a0d4e7c9275c1f6b88ff4a9 consensus state has no leader
W20260925 10:02:35.338156 13546 debug-util.cc:398] Leaking SignalData structure 0x7b08000dc100 after lost signal to thread 13421
W20260925 10:02:35.339419 13546 debug-util.cc:398] Leaking SignalData structure 0x7b080010ffe0 after lost signal to thread 13549
I20260925 10:02:38.212301 16205 raft_consensus.cc:482] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:02:38.212867 16205 raft_consensus.cc:504] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 10:02:38.215034 16205 leader_election.cc:288] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
W20260925 10:02:38.216539 13298 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
W20260925 10:02:38.217322 13297 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
I20260925 10:02:38.217653 13297 leader_election.cc:302] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 10:02:38.218308 16205 raft_consensus.cc:2726] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 10:02:44.534416 16205 raft_consensus.cc:482] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:02:44.534885 16205 raft_consensus.cc:504] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 10:02:44.536617 16205 leader_election.cc:288] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
W20260925 10:02:44.538234 13298 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
W20260925 10:02:44.538688 13297 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
I20260925 10:02:44.538985 13297 leader_election.cc:302] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 10:02:44.539515 16205 raft_consensus.cc:2726] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 10:02:45.841843 13113 debug-util.cc:398] Leaking SignalData structure 0x7b08000b5180 after lost signal to thread 13047
W20260925 10:02:45.842564 13113 debug-util.cc:398] Leaking SignalData structure 0x7b08000915e0 after lost signal to thread 13116
I20260925 10:02:49.905385 16418 raft_consensus.cc:482] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:02:49.906008 16418 raft_consensus.cc:504] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 10:02:49.908457 16418 leader_election.cc:288] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
W20260925 10:02:49.909933 13606 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
W20260925 10:02:49.910441 13603 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
I20260925 10:02:49.910705 13603 leader_election.cc:302] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237; no voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 10:02:49.911301 16418 raft_consensus.cc:2726] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 10:02:53.354816 13546 debug-util.cc:398] Leaking SignalData structure 0x7b0800084700 after lost signal to thread 13421
W20260925 10:02:53.355813 13546 debug-util.cc:398] Leaking SignalData structure 0x7b080011b480 after lost signal to thread 13549
W20260925 10:02:55.209007 13405 debug-util.cc:398] Leaking SignalData structure 0x7b08000e7dc0 after lost signal to thread 13274
W20260925 10:02:55.210424 13405 debug-util.cc:398] Leaking SignalData structure 0x7b0800104be0 after lost signal to thread 13408
I20260925 10:02:56.026966 16485 raft_consensus.cc:482] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:02:56.027670 16485 raft_consensus.cc:504] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 10:02:56.030025 16485 leader_election.cc:288] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
W20260925 10:02:56.031703 13603 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
W20260925 10:02:56.032209 13606 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
I20260925 10:02:56.032568 13606 leader_election.cc:302] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237; no voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 10:02:56.033253 16485 raft_consensus.cc:2726] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 10:03:02.665378 16475 raft_consensus.cc:482] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:03:02.666064 16475 raft_consensus.cc:504] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 10:03:02.668919 16475 leader_election.cc:288] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
W20260925 10:03:02.671324 13298 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
W20260925 10:03:02.671921 13297 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
I20260925 10:03:02.672206 13297 leader_election.cc:302] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 10:03:02.672760 16475 raft_consensus.cc:2726] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 10:03:08.511438 16564 raft_consensus.cc:482] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:03:08.512068 16564 raft_consensus.cc:504] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 10:03:08.514475 16564 leader_election.cc:288] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
W20260925 10:03:08.515960 13603 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
W20260925 10:03:08.516475 13606 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
I20260925 10:03:08.516755 13606 leader_election.cc:302] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237; no voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 10:03:08.517225 16564 raft_consensus.cc:2726] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 10:03:10.272501 13546 debug-util.cc:398] Leaking SignalData structure 0x7b0800107420 after lost signal to thread 13421
W20260925 10:03:10.274538 13546 debug-util.cc:398] Leaking SignalData structure 0x7b08001137a0 after lost signal to thread 13549
I20260925 10:03:13.577857 16614 raft_consensus.cc:482] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:03:13.578436 16614 raft_consensus.cc:504] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 10:03:13.580364 16614 leader_election.cc:288] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
W20260925 10:03:13.581894 13298 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
W20260925 10:03:13.582423 13297 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
I20260925 10:03:13.582669 13297 leader_election.cc:302] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 10:03:13.583313 16614 raft_consensus.cc:2726] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 10:03:14.251487 13551 heartbeater.cc:644] Failed to heartbeat to 127.10.9.61:40425 (193 consecutive failures): Timed out: Failed to ping master at 127.10.9.61:40425: Ping RPC to 127.10.9.61:40425 timed out after 1.000s (SENT)
W20260925 10:03:14.494884 13410 heartbeater.cc:644] Failed to heartbeat to 127.10.9.61:40425 (193 consecutive failures): Timed out: Failed to ping master at 127.10.9.61:40425: Ping RPC to 127.10.9.61:40425 timed out after 1.000s (SENT)
I20260925 10:03:14.811173 16564 raft_consensus.cc:482] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:03:14.811787 16564 raft_consensus.cc:504] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 10:03:14.814100 16564 leader_election.cc:288] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
W20260925 10:03:14.816726 13606 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
W20260925 10:03:14.818444 13717 heartbeater.cc:644] Failed to heartbeat to 127.10.9.61:40425 (193 consecutive failures): Timed out: Failed to ping master at 127.10.9.61:40425: Ping RPC to 127.10.9.61:40425 timed out after 1.000s (SENT)
W20260925 10:03:14.817312 13603 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
I20260925 10:03:14.818987 13603 leader_election.cc:302] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237; no voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 10:03:14.819864 16639 raft_consensus.cc:2726] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 10:03:18.005878 16564 raft_consensus.cc:482] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:03:18.006287 16564 raft_consensus.cc:504] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 10:03:18.008307 16564 leader_election.cc:288] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161), c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327)
W20260925 10:03:18.010047 13603 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b3082cf15dc74c0480e5d33ecf592843 (127.10.9.1:42161): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
W20260925 10:03:18.010524 13606 leader_election.cc:341] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: b103b022a84040ea97247d6503002f47
I20260925 10:03:18.010831 13606 leader_election.cc:302] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06c3ef7a6a94f7cbcc03e7cd232f237; no voters: b3082cf15dc74c0480e5d33ecf592843, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 10:03:18.011334 16564 raft_consensus.cc:2726] T b103b022a84040ea97247d6503002f47 P a06c3ef7a6a94f7cbcc03e7cd232f237 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 10:03:18.060696 16650 raft_consensus.cc:482] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 10:03:18.061365 16650 raft_consensus.cc:504] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3082cf15dc74c0480e5d33ecf592843" member_type: VOTER last_known_addr { host: "127.10.9.1" port: 42161 } } peers { permanent_uuid: "c4d724d4b83e4d1e817a3a505dc24eb9" member_type: VOTER last_known_addr { host: "127.10.9.2" port: 44327 } } peers { permanent_uuid: "a06c3ef7a6a94f7cbcc03e7cd232f237" member_type: VOTER last_known_addr { host: "127.10.9.3" port: 40731 } }
I20260925 10:03:18.063679 16650 leader_election.cc:288] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327), a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731)
W20260925 10:03:18.064934 13298 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4d724d4b83e4d1e817a3a505dc24eb9 (127.10.9.2:44327): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
W20260925 10:03:18.065510 13297 leader_election.cc:341] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a06c3ef7a6a94f7cbcc03e7cd232f237 (127.10.9.3:40731): Not found: Tablet not found: af6ace3011c84603b32d0a409d5b8c1c
I20260925 10:03:18.065814 13297 leader_election.cc:302] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3082cf15dc74c0480e5d33ecf592843; no voters: a06c3ef7a6a94f7cbcc03e7cd232f237, c4d724d4b83e4d1e817a3a505dc24eb9
I20260925 10:03:18.066614 16650 raft_consensus.cc:2726] T af6ace3011c84603b32d0a409d5b8c1c P b3082cf15dc74c0480e5d33ecf592843 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260925 10:03:26.539721 13727 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.10.9.62:43437,127.10.9.61:40425,127.10.9.60:42083, num_attempts: 759) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.10.9.60:42083: connect: Connection reset by peer (error 104)
W20260925 10:03:26.540311 13727 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.10.9.62:43437,127.10.9.61:40425,127.10.9.60:42083, num_attempts: 759) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.10.9.60:42083: connect: Connection reset by peer (error 104)
F20260925 10:03:26.552747 13741 master-stress-test.cc:352] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.10.9.62:43437,127.10.9.61:40425,127.10.9.60:42083, num_attempts: 759) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.10.9.60:42083: connect: Connection reset by peer (error 104)
*** Check failure stack trace: ***
*** Aborted at 1790330606 (unix time) try "date -d @1790330606" if you are using GNU date ***
F20260925 10:03:26.557917 13735 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1b826b3631274aca84923893f7383ee3 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.10.9.62:43437,127.10.9.61:40425,127.10.9.60:42083, num_attempts: 759) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.10.9.60:42083: connect: Connection reset by peer (error 104)
*** Check failure stack trace: ***
PC: @ 0x0 (unknown)
*** SIGABRT (@0x3e800002824) received by PID 10276 (TID 0x7ff8bdcff700) from PID 10276; stack trace: ***
@ 0x2b0af0 __tsan::CallUserSignalHandler() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1915
@ 0x2b31e0 rtl_sigaction() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:2005
@ 0x7ff8d1a43980 (unknown) at ??:0
@ 0x7ff8ccd00e87 gsignal at ??:0
@ 0x7ff8ccd027f1 abort at ??:0
@ 0x2b4f37 __interceptor_abort at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:263
@ 0x7ff8cfd34df4 google::LogMessage::SendToLog() at ??:0
@ 0x7ff8cfd359a0 google::LogMessage::Flush() at ??:0
@ 0x7ff8cfd3aadb google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x357ade kudu::MasterStressTest::DeleteTableThread() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/master-stress-test.cc:?
@ 0x3874ce _ZNSt3__18__invokeIMN4kudu16MasterStressTestEFvvEPNS1_26MasterStressTest_Test_TestEJEvEEDTcldsdeclsr3std3__1E7forwardIT0_Efp0_Efp_spclsr3std3__1E7forwardIT1_Efp1_EEEOT_OS7_DpOS8_ at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/type_traits:3840
@ 0x387381 _ZNSt3__116__thread_executeINS_10unique_ptrINS_15__thread_structENS_14default_deleteIS2_EEEEMN4kudu16MasterStressTestEFvvEJPNS6_26MasterStressTest_Test_TestEEJLm2EEEEvRNS_5tupleIJT_T0_DpT1_EEENS_15__tuple_indicesIJXspT2_EEEE at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/thread:281
@ 0x386acb std::__1::__thread_proxy<>() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/thread:?
@ 0x2b11b0 __tsan_thread_start_func at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:959
@ 0x7ff8d1a386db start_thread at ??:0
@ 0x7ff8ccde361f clone at ??:0